[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:22:07.304375  2067 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.4.254:32837
I20260709 12:22:07.306793  2067 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:22:07.308264  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:07.331948  2074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:07.332916  2076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:07.331951  2073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:07.334184  2067 server_base.cc:1061] running on GCE node
I20260709 12:22:08.605495  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:08.605899  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:08.606079  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599728606058 us; error 0 us; skew 500 ppm
I20260709 12:22:08.606998  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:08.613862  2067 webserver.cc:533] Webserver started at http://127.2.4.254:37385/ using document root <none> and password file <none>
I20260709 12:22:08.614928  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:08.615159  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:08.615669  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:08.622870  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/master-0-root/instance:
uuid: "371363936f9f4bb88d016afa314471ca"
format_stamp: "Formatted at 2026-07-09 12:22:08 on dist-test-slave-4qvp"
I20260709 12:22:08.633347  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:22:08.640403  2082 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:08.642174  2067 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 12:22:08.642669  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/master-0-root
uuid: "371363936f9f4bb88d016afa314471ca"
format_stamp: "Formatted at 2026-07-09 12:22:08 on dist-test-slave-4qvp"
I20260709 12:22:08.643088  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:08.709965  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:08.711944  2067 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:22:08.712517  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:08.811655  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.254:32837
I20260709 12:22:08.811748  2134 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.254:32837 every 8 connection(s)
I20260709 12:22:08.822381  2135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:08.860960  2135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca: Bootstrap starting.
I20260709 12:22:08.869237  2135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:08.872124  2135 log.cc:826] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca: Log is configured to *not* fsync() on all Append() calls
I20260709 12:22:08.880182  2135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca: No bootstrap required, opened a new log
I20260709 12:22:08.905603  2135 raft_consensus.cc:359] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371363936f9f4bb88d016afa314471ca" member_type: VOTER }
I20260709 12:22:08.906378  2135 raft_consensus.cc:385] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:08.906769  2135 raft_consensus.cc:740] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 371363936f9f4bb88d016afa314471ca, State: Initialized, Role: FOLLOWER
I20260709 12:22:08.908099  2135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371363936f9f4bb88d016afa314471ca" member_type: VOTER }
I20260709 12:22:08.908735  2135 raft_consensus.cc:399] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:08.909133  2135 raft_consensus.cc:493] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:08.909549  2135 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:08.914737  2135 raft_consensus.cc:515] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371363936f9f4bb88d016afa314471ca" member_type: VOTER }
I20260709 12:22:08.915807  2135 leader_election.cc:304] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 371363936f9f4bb88d016afa314471ca; no voters: 
I20260709 12:22:08.918066  2135 leader_election.cc:290] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:08.918524  2138 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:08.921778  2138 raft_consensus.cc:697] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [term 1 LEADER]: Becoming Leader. State: Replica: 371363936f9f4bb88d016afa314471ca, State: Running, Role: LEADER
I20260709 12:22:08.922936  2138 consensus_queue.cc:237] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371363936f9f4bb88d016afa314471ca" member_type: VOTER }
I20260709 12:22:08.924499  2135 sys_catalog.cc:565] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:08.939782  2139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "371363936f9f4bb88d016afa314471ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371363936f9f4bb88d016afa314471ca" member_type: VOTER } }
I20260709 12:22:08.941191  2139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:08.941495  2140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 371363936f9f4bb88d016afa314471ca. Latest consensus state: current_term: 1 leader_uuid: "371363936f9f4bb88d016afa314471ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371363936f9f4bb88d016afa314471ca" member_type: VOTER } }
I20260709 12:22:08.942420  2140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:08.946269  2148 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:08.965164  2148 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:08.968508  2067 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:08.988468  2148 catalog_manager.cc:1383] Generated new cluster ID: 46469e98b8b74f44932bef2edd190b74
I20260709 12:22:08.988835  2148 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:09.026837  2148 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:09.028988  2148 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:09.058449  2148 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca: Generated new TSK 0
I20260709 12:22:09.059836  2148 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:09.100664  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:09.111825  2157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:09.115414  2158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:09.122263  2160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:09.122922  2067 server_base.cc:1061] running on GCE node
I20260709 12:22:09.123993  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:09.124262  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:09.124483  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599729124458 us; error 0 us; skew 500 ppm
I20260709 12:22:09.125218  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:09.130909  2067 webserver.cc:533] Webserver started at http://127.2.4.193:37017/ using document root <none> and password file <none>
I20260709 12:22:09.131790  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:09.132150  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:09.132507  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:09.134353  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/ts-0-root/instance:
uuid: "c2759c2eeae9433d807b067536179bfb"
format_stamp: "Formatted at 2026-07-09 12:22:09 on dist-test-slave-4qvp"
I20260709 12:22:09.141744  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:22:09.147231  2165 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:09.148451  2067 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 12:22:09.148895  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/ts-0-root
uuid: "c2759c2eeae9433d807b067536179bfb"
format_stamp: "Formatted at 2026-07-09 12:22:09 on dist-test-slave-4qvp"
I20260709 12:22:09.149282  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:09.181145  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:09.183113  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:09.187096  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:09.192430  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:09.192770  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:09.193116  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:09.193336  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:09.311852  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.193:37261
I20260709 12:22:09.311959  2228 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.193:37261 every 8 connection(s)
I20260709 12:22:09.321223  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:09.345813  2233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:09.360699  2234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:09.361979  2236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:09.367992  2067 server_base.cc:1061] running on GCE node
I20260709 12:22:09.369107  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:09.369408  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:09.369652  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599729369603 us; error 0 us; skew 500 ppm
I20260709 12:22:09.370050  2229 heartbeater.cc:344] Connected to a master server at 127.2.4.254:32837
I20260709 12:22:09.370769  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:09.370961  2229 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:09.372711  2229 heartbeater.cc:507] Master 127.2.4.254:32837 requested a full tablet report, sending...
I20260709 12:22:09.374306  2067 webserver.cc:533] Webserver started at http://127.2.4.194:40797/ using document root <none> and password file <none>
I20260709 12:22:09.375021  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:09.375303  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:09.375667  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:09.376974  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/ts-1-root/instance:
uuid: "b642f379bf7c4533aba8f266d7d0fe53"
format_stamp: "Formatted at 2026-07-09 12:22:09 on dist-test-slave-4qvp"
I20260709 12:22:09.377956  2099 ts_manager.cc:194] Registered new tserver with Master: c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261)
I20260709 12:22:09.381843  2099 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59582
I20260709 12:22:09.383126  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.004s
I20260709 12:22:09.388139  2241 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:09.389798  2067 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 12:22:09.390208  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/ts-1-root
uuid: "b642f379bf7c4533aba8f266d7d0fe53"
format_stamp: "Formatted at 2026-07-09 12:22:09 on dist-test-slave-4qvp"
I20260709 12:22:09.390489  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:09.408439  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:09.409982  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:09.412755  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:09.416113  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:09.416325  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:09.416647  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:09.416795  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:09.535604  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.194:44805
I20260709 12:22:09.535801  2303 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.194:44805 every 8 connection(s)
I20260709 12:22:09.546861  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:09.562206  2308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:09.563632  2304 heartbeater.cc:344] Connected to a master server at 127.2.4.254:32837
W20260709 12:22:09.557952  2307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:09.564597  2304 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:09.566025  2304 heartbeater.cc:507] Master 127.2.4.254:32837 requested a full tablet report, sending...
I20260709 12:22:09.569836  2099 ts_manager.cc:194] Registered new tserver with Master: b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805)
W20260709 12:22:09.570538  2310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:09.570883  2067 server_base.cc:1061] running on GCE node
I20260709 12:22:09.572276  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:22:09.572233  2099 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59596
W20260709 12:22:09.572789  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:09.573063  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599729573044 us; error 0 us; skew 500 ppm
I20260709 12:22:09.573976  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:09.577325  2067 webserver.cc:533] Webserver started at http://127.2.4.195:37181/ using document root <none> and password file <none>
I20260709 12:22:09.578018  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:09.578281  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:09.578604  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:09.579883  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/ts-2-root/instance:
uuid: "91e3a0484e714200a61187a599eca1a1"
format_stamp: "Formatted at 2026-07-09 12:22:09 on dist-test-slave-4qvp"
I20260709 12:22:09.586051  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:22:09.590392  2315 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:09.591596  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:22:09.592001  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/ts-2-root
uuid: "91e3a0484e714200a61187a599eca1a1"
format_stamp: "Formatted at 2026-07-09 12:22:09 on dist-test-slave-4qvp"
I20260709 12:22:09.592439  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599727094521-2067-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:09.614331  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:09.615809  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:09.618196  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:09.622164  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:09.622437  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:09.622934  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:09.623162  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:09.737504  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.195:40031
I20260709 12:22:09.737797  2378 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.195:40031 every 8 connection(s)
I20260709 12:22:09.759819  2379 heartbeater.cc:344] Connected to a master server at 127.2.4.254:32837
I20260709 12:22:09.760397  2379 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:09.761767  2379 heartbeater.cc:507] Master 127.2.4.254:32837 requested a full tablet report, sending...
I20260709 12:22:09.764711  2099 ts_manager.cc:194] Registered new tserver with Master: 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:09.765699  2067 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.021923213s
I20260709 12:22:09.766589  2099 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54726
I20260709 12:22:09.825758  2099 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54738:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:09.830236  2099 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260709 12:22:09.949406  2147 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:09.965981  2267 tablet_service.cc:1511] Processing CreateTablet for tablet 7effe2c171054fd0a4fdbd5216c88e03 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:09.962018  2269 tablet_service.cc:1511] Processing CreateTablet for tablet 1b92f57dc0754ad8b04b6fe89d7a9fe4 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:09.966606  2268 tablet_service.cc:1511] Processing CreateTablet for tablet d3f2033c07ca45a183a62bac15cefbdc (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:09.971275  2266 tablet_service.cc:1511] Processing CreateTablet for tablet 3c011ea4eb8c4263918f63ca65871907 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:09.972362  2269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b92f57dc0754ad8b04b6fe89d7a9fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:09.978823  2267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7effe2c171054fd0a4fdbd5216c88e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:09.980719  2266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c011ea4eb8c4263918f63ca65871907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:09.982954  2268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3f2033c07ca45a183a62bac15cefbdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:09.988880  2265 tablet_service.cc:1511] Processing CreateTablet for tablet 91323316a13b463db86243d302b1fbe5 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:09.992892  2265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91323316a13b463db86243d302b1fbe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.018599  2340 tablet_service.cc:1511] Processing CreateTablet for tablet 91323316a13b463db86243d302b1fbe5 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:10.024894  2343 tablet_service.cc:1511] Processing CreateTablet for tablet d3f2033c07ca45a183a62bac15cefbdc (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:10.024878  2341 tablet_service.cc:1511] Processing CreateTablet for tablet 3c011ea4eb8c4263918f63ca65871907 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:10.035367  2340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91323316a13b463db86243d302b1fbe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.039983  2194 tablet_service.cc:1511] Processing CreateTablet for tablet 1b92f57dc0754ad8b04b6fe89d7a9fe4 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:10.037801  2342 tablet_service.cc:1511] Processing CreateTablet for tablet 7effe2c171054fd0a4fdbd5216c88e03 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:10.044589  2264 tablet_service.cc:1511] Processing CreateTablet for tablet f7b8bf01edfc401c8c9269e27f1f48e1 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:10.048849  2339 tablet_service.cc:1511] Processing CreateTablet for tablet f7b8bf01edfc401c8c9269e27f1f48e1 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:10.037557  2344 tablet_service.cc:1511] Processing CreateTablet for tablet 1b92f57dc0754ad8b04b6fe89d7a9fe4 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:10.047652  2193 tablet_service.cc:1511] Processing CreateTablet for tablet d3f2033c07ca45a183a62bac15cefbdc (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:10.060878  2343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3f2033c07ca45a183a62bac15cefbdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.062415  2264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7b8bf01edfc401c8c9269e27f1f48e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.070961  2194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b92f57dc0754ad8b04b6fe89d7a9fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.070820  2192 tablet_service.cc:1511] Processing CreateTablet for tablet 7effe2c171054fd0a4fdbd5216c88e03 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:10.069201  2190 tablet_service.cc:1511] Processing CreateTablet for tablet 91323316a13b463db86243d302b1fbe5 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:10.074198  2341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c011ea4eb8c4263918f63ca65871907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.059022  2191 tablet_service.cc:1511] Processing CreateTablet for tablet 3c011ea4eb8c4263918f63ca65871907 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:10.069043  2189 tablet_service.cc:1511] Processing CreateTablet for tablet f7b8bf01edfc401c8c9269e27f1f48e1 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:10.087390  2193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3f2033c07ca45a183a62bac15cefbdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.087307  2263 tablet_service.cc:1511] Processing CreateTablet for tablet d159b73e1fed4f1eab8d8caf495fc751 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:10.091493  2188 tablet_service.cc:1511] Processing CreateTablet for tablet d159b73e1fed4f1eab8d8caf495fc751 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:10.096060  2344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b92f57dc0754ad8b04b6fe89d7a9fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.097179  2263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d159b73e1fed4f1eab8d8caf495fc751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.095093  2191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c011ea4eb8c4263918f63ca65871907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.105911  2339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7b8bf01edfc401c8c9269e27f1f48e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.108702  2338 tablet_service.cc:1511] Processing CreateTablet for tablet d159b73e1fed4f1eab8d8caf495fc751 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:10.111867  2342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7effe2c171054fd0a4fdbd5216c88e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.114523  2190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91323316a13b463db86243d302b1fbe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.119220  2189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7b8bf01edfc401c8c9269e27f1f48e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.122663  2188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d159b73e1fed4f1eab8d8caf495fc751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.128600  2192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7effe2c171054fd0a4fdbd5216c88e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.129185  2338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d159b73e1fed4f1eab8d8caf495fc751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.137804  2187 tablet_service.cc:1511] Processing CreateTablet for tablet 1f984985863e44299701c48dba1f4315 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:10.159704  2187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f984985863e44299701c48dba1f4315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.176796  2397 tablet_bootstrap.cc:492] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53: Bootstrap starting.
I20260709 12:22:10.203502  2265 tablet_service.cc:1511] Processing CreateTablet for tablet 1f984985863e44299701c48dba1f4315 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:10.211325  2337 tablet_service.cc:1511] Processing CreateTablet for tablet 1f984985863e44299701c48dba1f4315 (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:10.217470  2265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f984985863e44299701c48dba1f4315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.229712  2185 tablet_service.cc:1511] Processing CreateTablet for tablet 0b055ff9d3d7458faaaef52610877acf (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:10.238871  2185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b055ff9d3d7458faaaef52610877acf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.236925  2266 tablet_service.cc:1511] Processing CreateTablet for tablet 0b055ff9d3d7458faaaef52610877acf (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:10.242501  2266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b055ff9d3d7458faaaef52610877acf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.246115  2337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f984985863e44299701c48dba1f4315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.269781  2336 tablet_service.cc:1511] Processing CreateTablet for tablet 0b055ff9d3d7458faaaef52610877acf (DEFAULT_TABLE table=test-workload [id=ecc8f3727e4743e3a38b955f9cd8e87b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:10.271481  2336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b055ff9d3d7458faaaef52610877acf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:10.326704  2398 tablet_bootstrap.cc:492] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1: Bootstrap starting.
I20260709 12:22:10.336397  2399 tablet_bootstrap.cc:492] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb: Bootstrap starting.
I20260709 12:22:10.345799  2397 tablet_bootstrap.cc:654] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.361846  2399 tablet_bootstrap.cc:654] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.387230  2229 heartbeater.cc:499] Master 127.2.4.254:32837 was elected leader, sending a full tablet report...
I20260709 12:22:10.392118  2398 tablet_bootstrap.cc:654] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.395442  2397 tablet_bootstrap.cc:492] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53: No bootstrap required, opened a new log
I20260709 12:22:10.396154  2397 ts_tablet_manager.cc:1403] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53: Time spent bootstrapping tablet: real 0.220s	user 0.008s	sys 0.015s
I20260709 12:22:10.396322  2399 tablet_bootstrap.cc:492] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb: No bootstrap required, opened a new log
I20260709 12:22:10.397423  2399 ts_tablet_manager.cc:1403] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb: Time spent bootstrapping tablet: real 0.084s	user 0.011s	sys 0.008s
I20260709 12:22:10.402012  2399 raft_consensus.cc:359] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:10.401824  2397 raft_consensus.cc:359] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.402787  2399 raft_consensus.cc:385] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.402945  2397 raft_consensus.cc:385] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.403149  2399 raft_consensus.cc:740] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.403369  2397 raft_consensus.cc:740] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.404026  2399 consensus_queue.cc:260] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:10.404548  2397 consensus_queue.cc:260] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.411480  2399 ts_tablet_manager.cc:1434] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.004s
I20260709 12:22:10.411908  2304 heartbeater.cc:499] Master 127.2.4.254:32837 was elected leader, sending a full tablet report...
I20260709 12:22:10.412662  2399 tablet_bootstrap.cc:492] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb: Bootstrap starting.
I20260709 12:22:10.415030  2397 ts_tablet_manager.cc:1434] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53: Time spent starting tablet: real 0.019s	user 0.011s	sys 0.000s
I20260709 12:22:10.416846  2397 tablet_bootstrap.cc:492] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53: Bootstrap starting.
I20260709 12:22:10.420603  2399 tablet_bootstrap.cc:654] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.430015  2398 tablet_bootstrap.cc:492] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1: No bootstrap required, opened a new log
I20260709 12:22:10.430689  2398 ts_tablet_manager.cc:1403] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1: Time spent bootstrapping tablet: real 0.114s	user 0.020s	sys 0.005s
I20260709 12:22:10.430951  2399 tablet_bootstrap.cc:492] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb: No bootstrap required, opened a new log
I20260709 12:22:10.433872  2397 tablet_bootstrap.cc:654] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.434175  2399 ts_tablet_manager.cc:1403] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.010s
I20260709 12:22:10.438311  2399 raft_consensus.cc:359] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.439167  2399 raft_consensus.cc:385] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.439491  2399 raft_consensus.cc:740] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.440305  2399 consensus_queue.cc:260] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.442366  2398 raft_consensus.cc:359] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.443159  2398 raft_consensus.cc:385] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.443480  2398 raft_consensus.cc:740] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.447651  2399 ts_tablet_manager.cc:1434] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 12:22:10.448443  2397 tablet_bootstrap.cc:492] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53: No bootstrap required, opened a new log
I20260709 12:22:10.448876  2399 tablet_bootstrap.cc:492] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb: Bootstrap starting.
I20260709 12:22:10.449044  2397 ts_tablet_manager.cc:1403] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.000s
I20260709 12:22:10.444604  2398 consensus_queue.cc:260] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.453922  2397 raft_consensus.cc:359] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.454833  2397 raft_consensus.cc:385] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.455163  2397 raft_consensus.cc:740] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.455998  2397 consensus_queue.cc:260] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.461095  2399 tablet_bootstrap.cc:654] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.463507  2404 raft_consensus.cc:493] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.464532  2398 ts_tablet_manager.cc:1434] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.005s
I20260709 12:22:10.464159  2404 raft_consensus.cc:515] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.465306  2379 heartbeater.cc:499] Master 127.2.4.254:32837 was elected leader, sending a full tablet report...
I20260709 12:22:10.467603  2404 leader_election.cc:290] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:10.471033  2397 ts_tablet_manager.cc:1434] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.002s
I20260709 12:22:10.473767  2397 tablet_bootstrap.cc:492] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53: Bootstrap starting.
I20260709 12:22:10.479075  2398 tablet_bootstrap.cc:492] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1: Bootstrap starting.
I20260709 12:22:10.472470  2408 raft_consensus.cc:493] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.483070  2397 tablet_bootstrap.cc:654] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.483916  2408 raft_consensus.cc:515] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
W20260709 12:22:10.497128  2380 tablet.cc:2406] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:22:10.506266  2397 tablet_bootstrap.cc:492] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53: No bootstrap required, opened a new log
I20260709 12:22:10.506963  2397 ts_tablet_manager.cc:1403] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53: Time spent bootstrapping tablet: real 0.034s	user 0.021s	sys 0.008s
I20260709 12:22:10.510493  2397 raft_consensus.cc:359] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.511703  2397 raft_consensus.cc:385] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.512089  2397 raft_consensus.cc:740] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.512996  2397 consensus_queue.cc:260] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.515769  2397 ts_tablet_manager.cc:1434] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:22:10.516953  2397 tablet_bootstrap.cc:492] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53: Bootstrap starting.
I20260709 12:22:10.521600  2398 tablet_bootstrap.cc:654] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.524367  2397 tablet_bootstrap.cc:654] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.536271  2404 raft_consensus.cc:493] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.536820  2404 raft_consensus.cc:515] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.537731  2397 tablet_bootstrap.cc:492] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53: No bootstrap required, opened a new log
I20260709 12:22:10.538532  2397 ts_tablet_manager.cc:1403] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53: Time spent bootstrapping tablet: real 0.022s	user 0.005s	sys 0.011s
I20260709 12:22:10.539047  2404 leader_election.cc:290] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:10.535890  2398 tablet_bootstrap.cc:492] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1: No bootstrap required, opened a new log
I20260709 12:22:10.540038  2398 ts_tablet_manager.cc:1403] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1: Time spent bootstrapping tablet: real 0.061s	user 0.014s	sys 0.010s
I20260709 12:22:10.543398  2398 raft_consensus.cc:359] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.544224  2398 raft_consensus.cc:385] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.544528  2398 raft_consensus.cc:740] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.546372  2397 raft_consensus.cc:359] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.548153  2397 raft_consensus.cc:385] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.548975  2397 raft_consensus.cc:740] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.551323  2397 consensus_queue.cc:260] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.554908  2397 ts_tablet_manager.cc:1434] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.001s
I20260709 12:22:10.556322  2397 tablet_bootstrap.cc:492] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53: Bootstrap starting.
I20260709 12:22:10.557178  2204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb" is_pre_election: true
I20260709 12:22:10.545310  2398 consensus_queue.cc:260] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.564018  2398 ts_tablet_manager.cc:1434] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:22:10.564603  2397 tablet_bootstrap.cc:654] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.565616  2398 tablet_bootstrap.cc:492] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1: Bootstrap starting.
I20260709 12:22:10.568183  2204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f2033c07ca45a183a62bac15cefbdc" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb" is_pre_election: true
I20260709 12:22:10.569141  2204 raft_consensus.cc:2468] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 0.
I20260709 12:22:10.573503  2317 leader_election.cc:304] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:10.575299  2408 leader_election.cc:290] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805)
I20260709 12:22:10.580821  2398 tablet_bootstrap.cc:654] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.580727  2407 raft_consensus.cc:2804] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:10.581430  2407 raft_consensus.cc:493] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:10.581915  2407 raft_consensus.cc:3060] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:10.587766  2399 tablet_bootstrap.cc:492] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb: No bootstrap required, opened a new log
I20260709 12:22:10.588585  2399 ts_tablet_manager.cc:1403] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb: Time spent bootstrapping tablet: real 0.140s	user 0.029s	sys 0.026s
I20260709 12:22:10.590507  2407 raft_consensus.cc:515] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.593223  2407 leader_election.cc:290] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 election: Requested vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805)
I20260709 12:22:10.594439  2399 raft_consensus.cc:359] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.595719  2399 raft_consensus.cc:385] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.596202  2399 raft_consensus.cc:740] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.597453  2399 consensus_queue.cc:260] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.601884  2204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f2033c07ca45a183a62bac15cefbdc" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:10.614600  2204 raft_consensus.cc:3060] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:10.614840  2399 ts_tablet_manager.cc:1434] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 12:22:10.616070  2399 tablet_bootstrap.cc:492] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb: Bootstrap starting.
I20260709 12:22:10.605424  2203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91323316a13b463db86243d302b1fbe5" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb" is_pre_election: true
I20260709 12:22:10.607721  2404 raft_consensus.cc:493] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.618172  2404 raft_consensus.cc:515] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.620607  2404 leader_election.cc:290] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
W20260709 12:22:10.608525  2243 leader_election.cc:343] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:10.627887  2399 tablet_bootstrap.cc:654] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.628546  2203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f2033c07ca45a183a62bac15cefbdc" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb" is_pre_election: true
I20260709 12:22:10.633661  2398 tablet_bootstrap.cc:492] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1: No bootstrap required, opened a new log
I20260709 12:22:10.634445  2398 ts_tablet_manager.cc:1403] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1: Time spent bootstrapping tablet: real 0.069s	user 0.016s	sys 0.009s
I20260709 12:22:10.639369  2398 raft_consensus.cc:359] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.640565  2398 raft_consensus.cc:385] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.640956  2398 raft_consensus.cc:740] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.642426  2398 consensus_queue.cc:260] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.643292  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1" is_pre_election: true
I20260709 12:22:10.644996  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91323316a13b463db86243d302b1fbe5" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1" is_pre_election: true
I20260709 12:22:10.646423  2353 raft_consensus.cc:2468] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 0.
I20260709 12:22:10.644991  2398 ts_tablet_manager.cc:1434] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:22:10.656628  2398 tablet_bootstrap.cc:492] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1: Bootstrap starting.
I20260709 12:22:10.658253  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f2033c07ca45a183a62bac15cefbdc" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1" is_pre_election: true
I20260709 12:22:10.658546  2279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f2033c07ca45a183a62bac15cefbdc" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53" is_pre_election: true
I20260709 12:22:10.659246  2353 raft_consensus.cc:2393] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b642f379bf7c4533aba8f266d7d0fe53 in current term 1: Already voted for candidate 91e3a0484e714200a61187a599eca1a1 in this term.
I20260709 12:22:10.659653  2279 raft_consensus.cc:2468] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 0.
W20260709 12:22:10.660518  2243 leader_election.cc:343] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:10.662971  2243 leader_election.cc:343] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:10.663470  2243 leader_election.cc:304] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53; no voters: 91e3a0484e714200a61187a599eca1a1, c2759c2eeae9433d807b067536179bfb
I20260709 12:22:10.661496  2278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f2033c07ca45a183a62bac15cefbdc" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:10.665261  2278 raft_consensus.cc:3060] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:10.665987  2404 raft_consensus.cc:2749] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:10.666182  2204 raft_consensus.cc:2468] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 1.
I20260709 12:22:10.666466  2243 leader_election.cc:304] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, b642f379bf7c4533aba8f266d7d0fe53; no voters: c2759c2eeae9433d807b067536179bfb
I20260709 12:22:10.667968  2404 raft_consensus.cc:2804] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:10.667493  2317 leader_election.cc:304] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:10.667810  2243 leader_election.cc:304] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53; no voters: 91e3a0484e714200a61187a599eca1a1, c2759c2eeae9433d807b067536179bfb
I20260709 12:22:10.668694  2404 raft_consensus.cc:493] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:10.669816  2407 raft_consensus.cc:2804] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:10.670208  2404 raft_consensus.cc:3060] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:10.670375  2407 raft_consensus.cc:697] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 1 LEADER]: Becoming Leader. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Running, Role: LEADER
I20260709 12:22:10.671859  2407 consensus_queue.cc:237] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.672930  2397 tablet_bootstrap.cc:492] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53: No bootstrap required, opened a new log
I20260709 12:22:10.673759  2397 ts_tablet_manager.cc:1403] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53: Time spent bootstrapping tablet: real 0.118s	user 0.042s	sys 0.056s
I20260709 12:22:10.679786  2399 tablet_bootstrap.cc:492] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb: No bootstrap required, opened a new log
I20260709 12:22:10.684649  2399 ts_tablet_manager.cc:1403] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb: Time spent bootstrapping tablet: real 0.069s	user 0.031s	sys 0.009s
I20260709 12:22:10.689421  2399 raft_consensus.cc:359] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.690423  2399 raft_consensus.cc:385] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.690825  2399 raft_consensus.cc:740] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.691357  2398 tablet_bootstrap.cc:654] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.692137  2399 consensus_queue.cc:260] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.694767  2278 raft_consensus.cc:2468] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 1.
I20260709 12:22:10.694495  2404 raft_consensus.cc:515] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.697232  2404 leader_election.cc:290] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 election: Requested vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:10.703205  2404 raft_consensus.cc:2749] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:10.704129  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91323316a13b463db86243d302b1fbe5" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:10.704826  2353 raft_consensus.cc:3060] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:10.702366  2204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91323316a13b463db86243d302b1fbe5" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:10.702698  2399 ts_tablet_manager.cc:1434] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb: Time spent starting tablet: real 0.018s	user 0.000s	sys 0.008s
I20260709 12:22:10.699486  2397 raft_consensus.cc:359] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.713495  2397 raft_consensus.cc:385] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.716599  2397 raft_consensus.cc:740] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Initialized, Role: FOLLOWER
W20260709 12:22:10.715399  2243 leader_election.cc:343] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:10.720490  2397 consensus_queue.cc:260] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.722587  2353 raft_consensus.cc:2468] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 1.
I20260709 12:22:10.721956  2399 tablet_bootstrap.cc:492] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb: Bootstrap starting.
I20260709 12:22:10.729754  2403 raft_consensus.cc:493] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.731074  2403 raft_consensus.cc:515] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:10.738283  2403 leader_election.cc:290] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:10.740685  2397 ts_tablet_manager.cc:1434] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53: Time spent starting tablet: real 0.066s	user 0.006s	sys 0.003s
I20260709 12:22:10.742179  2397 tablet_bootstrap.cc:492] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53: Bootstrap starting.
I20260709 12:22:10.745211  2398 tablet_bootstrap.cc:492] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1: No bootstrap required, opened a new log
I20260709 12:22:10.745941  2398 ts_tablet_manager.cc:1403] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1: Time spent bootstrapping tablet: real 0.090s	user 0.006s	sys 0.013s
I20260709 12:22:10.747722  2098 catalog_manager.cc:5697] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91e3a0484e714200a61187a599eca1a1 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "91e3a0484e714200a61187a599eca1a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:10.751700  2398 raft_consensus.cc:359] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.755725  2398 raft_consensus.cc:385] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.756050  2398 raft_consensus.cc:740] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.756901  2399 tablet_bootstrap.cc:654] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.758831  2398 consensus_queue.cc:260] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.765519  2397 tablet_bootstrap.cc:654] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.751585  2243 leader_election.cc:304] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, b642f379bf7c4533aba8f266d7d0fe53; no voters: c2759c2eeae9433d807b067536179bfb
I20260709 12:22:10.773123  2420 raft_consensus.cc:2804] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:10.773891  2420 raft_consensus.cc:697] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 LEADER]: Becoming Leader. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:10.775028  2398 ts_tablet_manager.cc:1434] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.001s
I20260709 12:22:10.774860  2420 consensus_queue.cc:237] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.781817  2398 tablet_bootstrap.cc:492] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1: Bootstrap starting.
I20260709 12:22:10.784617  2404 raft_consensus.cc:493] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.786653  2404 raft_consensus.cc:515] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.792296  2404 leader_election.cc:290] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:10.801100  2204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c011ea4eb8c4263918f63ca65871907" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb" is_pre_election: true
I20260709 12:22:10.804984  2278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53" is_pre_election: true
W20260709 12:22:10.807709  2243 leader_election.cc:343] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:10.812700  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c011ea4eb8c4263918f63ca65871907" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1" is_pre_election: true
W20260709 12:22:10.814019  2166 leader_election.cc:343] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:10.815621  2397 tablet_bootstrap.cc:492] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53: No bootstrap required, opened a new log
W20260709 12:22:10.815907  2243 leader_election.cc:343] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:10.816174  2397 ts_tablet_manager.cc:1403] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53: Time spent bootstrapping tablet: real 0.074s	user 0.010s	sys 0.007s
I20260709 12:22:10.816341  2243 leader_election.cc:304] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53; no voters: 91e3a0484e714200a61187a599eca1a1, c2759c2eeae9433d807b067536179bfb
I20260709 12:22:10.817116  2404 raft_consensus.cc:2749] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:10.822117  2397 raft_consensus.cc:359] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:10.823158  2397 raft_consensus.cc:385] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.823590  2397 raft_consensus.cc:740] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.826578  2398 tablet_bootstrap.cc:654] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.828522  2407 raft_consensus.cc:493] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.824748  2397 consensus_queue.cc:260] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:10.829334  2407 raft_consensus.cc:515] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.832155  2399 tablet_bootstrap.cc:492] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb: No bootstrap required, opened a new log
I20260709 12:22:10.832793  2399 ts_tablet_manager.cc:1403] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb: Time spent bootstrapping tablet: real 0.111s	user 0.015s	sys 0.014s
I20260709 12:22:10.835309  2407 leader_election.cc:290] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805)
I20260709 12:22:10.836545  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1" is_pre_election: true
I20260709 12:22:10.836040  2099 catalog_manager.cc:5697] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 reported cstate change: term changed from 0 to 1, leader changed from <none> to b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:10.838685  2278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53" is_pre_election: true
I20260709 12:22:10.839433  2278 raft_consensus.cc:2468] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 0.
W20260709 12:22:10.840123  2167 leader_election.cc:343] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:10.840620  2167 leader_election.cc:304] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c2759c2eeae9433d807b067536179bfb; no voters: 91e3a0484e714200a61187a599eca1a1, b642f379bf7c4533aba8f266d7d0fe53
I20260709 12:22:10.841833  2403 raft_consensus.cc:2749] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:10.841542  2204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb" is_pre_election: true
I20260709 12:22:10.846746  2316 leader_election.cc:304] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, b642f379bf7c4533aba8f266d7d0fe53; no voters: 
I20260709 12:22:10.847844  2407 raft_consensus.cc:2804] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:22:10.847970  2317 leader_election.cc:343] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:10.848292  2407 raft_consensus.cc:493] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:10.848726  2407 raft_consensus.cc:3060] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:10.848829  2420 raft_consensus.cc:493] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.849607  2420 raft_consensus.cc:515] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.854456  2420 leader_election.cc:290] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:10.856590  2204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b8bf01edfc401c8c9269e27f1f48e1" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb" is_pre_election: true
I20260709 12:22:10.857599  2204 raft_consensus.cc:2468] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 0.
I20260709 12:22:10.858726  2407 raft_consensus.cc:515] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.859891  2243 leader_election.cc:304] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:10.861068  2407 leader_election.cc:290] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 election: Requested vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805)
I20260709 12:22:10.843291  2399 raft_consensus.cc:359] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.863252  2403 raft_consensus.cc:493] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.866616  2403 raft_consensus.cc:515] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.867432  2420 raft_consensus.cc:2804] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:10.867832  2420 raft_consensus.cc:493] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:10.866050  2399 raft_consensus.cc:385] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.868983  2399 raft_consensus.cc:740] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.869863  2399 consensus_queue.cc:260] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.847503  2397 ts_tablet_manager.cc:1434] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:22:10.868175  2420 raft_consensus.cc:3060] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:10.869695  2278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d159b73e1fed4f1eab8d8caf495fc751" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53" is_pre_election: true
I20260709 12:22:10.871163  2279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:10.872565  2279 raft_consensus.cc:3060] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:10.868775  2403 leader_election.cc:290] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
W20260709 12:22:10.873328  2166 leader_election.cc:343] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:10.869822  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b8bf01edfc401c8c9269e27f1f48e1" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1" is_pre_election: true
I20260709 12:22:10.876832  2397 tablet_bootstrap.cc:492] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53: Bootstrap starting.
I20260709 12:22:10.874528  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d159b73e1fed4f1eab8d8caf495fc751" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1" is_pre_election: true
W20260709 12:22:10.880579  2243 leader_election.cc:343] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:10.881207  2398 tablet_bootstrap.cc:492] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1: No bootstrap required, opened a new log
I20260709 12:22:10.881805  2398 ts_tablet_manager.cc:1403] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1: Time spent bootstrapping tablet: real 0.102s	user 0.023s	sys 0.008s
I20260709 12:22:10.883088  2354 raft_consensus.cc:2468] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 0.
I20260709 12:22:10.885095  2167 leader_election.cc:304] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, c2759c2eeae9433d807b067536179bfb; no voters: b642f379bf7c4533aba8f266d7d0fe53
I20260709 12:22:10.886448  2403 raft_consensus.cc:2804] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:10.886876  2403 raft_consensus.cc:493] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:10.887248  2403 raft_consensus.cc:3060] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:10.885123  2204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb"
W20260709 12:22:10.890843  2317 leader_election.cc:343] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:10.892977  2398 raft_consensus.cc:359] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.893826  2398 raft_consensus.cc:385] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.894162  2398 raft_consensus.cc:740] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.909710  2398 consensus_queue.cc:260] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.906844  2403 raft_consensus.cc:515] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.916831  2279 raft_consensus.cc:2468] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 1.
I20260709 12:22:10.919296  2316 leader_election.cc:304] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, b642f379bf7c4533aba8f266d7d0fe53; no voters: c2759c2eeae9433d807b067536179bfb
I20260709 12:22:10.915657  2398 ts_tablet_manager.cc:1434] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1: Time spent starting tablet: real 0.034s	user 0.009s	sys 0.000s
I20260709 12:22:10.920622  2407 raft_consensus.cc:2804] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:10.920580  2420 raft_consensus.cc:515] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.921252  2407 raft_consensus.cc:697] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 1 LEADER]: Becoming Leader. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Running, Role: LEADER
I20260709 12:22:10.922104  2407 consensus_queue.cc:237] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.923801  2204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b8bf01edfc401c8c9269e27f1f48e1" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:10.924142  2397 tablet_bootstrap.cc:654] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.918058  2399 ts_tablet_manager.cc:1434] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb: Time spent starting tablet: real 0.085s	user 0.012s	sys 0.021s
I20260709 12:22:10.925086  2398 tablet_bootstrap.cc:492] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1: Bootstrap starting.
I20260709 12:22:10.924391  2204 raft_consensus.cc:3060] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:10.926167  2399 tablet_bootstrap.cc:492] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb: Bootstrap starting.
I20260709 12:22:10.926858  2403 leader_election.cc:290] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 election: Requested vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:10.928105  2420 leader_election.cc:290] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 election: Requested vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:10.928151  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d159b73e1fed4f1eab8d8caf495fc751" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:10.926795  2279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d159b73e1fed4f1eab8d8caf495fc751" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:10.935151  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b8bf01edfc401c8c9269e27f1f48e1" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:10.936131  2354 raft_consensus.cc:3060] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:10.936797  2166 leader_election.cc:343] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:10.936966  2243 leader_election.cc:343] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:10.947106  2204 raft_consensus.cc:2468] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 1.
I20260709 12:22:10.950254  2243 leader_election.cc:304] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb; no voters: 91e3a0484e714200a61187a599eca1a1
W20260709 12:22:10.951913  2147 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:10.952927  2420 raft_consensus.cc:2804] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:10.953375  2354 raft_consensus.cc:2468] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 1.
I20260709 12:22:10.955271  2420 raft_consensus.cc:697] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 LEADER]: Becoming Leader. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:10.958041  2167 leader_election.cc:304] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, c2759c2eeae9433d807b067536179bfb; no voters: b642f379bf7c4533aba8f266d7d0fe53
I20260709 12:22:10.947340  2097 catalog_manager.cc:5697] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91e3a0484e714200a61187a599eca1a1 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "91e3a0484e714200a61187a599eca1a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:10.960958  2403 raft_consensus.cc:2804] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:10.961676  2403 raft_consensus.cc:697] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 1 LEADER]: Becoming Leader. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Running, Role: LEADER
I20260709 12:22:10.960005  2420 consensus_queue.cc:237] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.962513  2397 tablet_bootstrap.cc:492] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53: No bootstrap required, opened a new log
I20260709 12:22:10.962808  2403 consensus_queue.cc:237] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.964166  2397 ts_tablet_manager.cc:1403] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53: Time spent bootstrapping tablet: real 0.088s	user 0.009s	sys 0.013s
I20260709 12:22:10.965209  2399 tablet_bootstrap.cc:654] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.988538  2397 raft_consensus.cc:359] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:10.990263  2397 raft_consensus.cc:385] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:10.990705  2399 tablet_bootstrap.cc:492] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb: No bootstrap required, opened a new log
I20260709 12:22:10.989957  2098 catalog_manager.cc:5697] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb reported cstate change: term changed from 0 to 1, leader changed from <none> to c2759c2eeae9433d807b067536179bfb (127.2.4.193). New cstate: current_term: 1 leader_uuid: "c2759c2eeae9433d807b067536179bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:10.990873  2397 raft_consensus.cc:740] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Initialized, Role: FOLLOWER
I20260709 12:22:10.991827  2399 ts_tablet_manager.cc:1403] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb: Time spent bootstrapping tablet: real 0.066s	user 0.014s	sys 0.006s
I20260709 12:22:10.994676  2398 tablet_bootstrap.cc:654] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:10.994900  2099 catalog_manager.cc:5697] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 reported cstate change: term changed from 0 to 1, leader changed from <none> to b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:11.000403  2397 consensus_queue.cc:260] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.011673  2397 ts_tablet_manager.cc:1434] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.006s
I20260709 12:22:11.016942  2397 tablet_bootstrap.cc:492] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53: Bootstrap starting.
I20260709 12:22:11.018091  2399 raft_consensus.cc:359] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.020550  2399 raft_consensus.cc:385] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:11.027140  2399 raft_consensus.cc:740] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Initialized, Role: FOLLOWER
I20260709 12:22:11.028008  2399 consensus_queue.cc:260] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.028780  2397 tablet_bootstrap.cc:654] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:11.028954  2398 tablet_bootstrap.cc:492] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1: No bootstrap required, opened a new log
I20260709 12:22:11.029661  2398 ts_tablet_manager.cc:1403] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1: Time spent bootstrapping tablet: real 0.107s	user 0.017s	sys 0.008s
I20260709 12:22:11.039317  2399 ts_tablet_manager.cc:1434] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb: Time spent starting tablet: real 0.046s	user 0.010s	sys 0.003s
I20260709 12:22:11.040719  2399 tablet_bootstrap.cc:492] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb: Bootstrap starting.
I20260709 12:22:11.041958  2397 tablet_bootstrap.cc:492] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53: No bootstrap required, opened a new log
I20260709 12:22:11.042521  2397 ts_tablet_manager.cc:1403] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.005s
I20260709 12:22:11.042858  2398 raft_consensus.cc:359] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.043839  2398 raft_consensus.cc:385] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:11.044206  2398 raft_consensus.cc:740] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Initialized, Role: FOLLOWER
I20260709 12:22:11.046108  2398 consensus_queue.cc:260] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.048592  2398 ts_tablet_manager.cc:1434] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.005s
I20260709 12:22:11.048584  2397 raft_consensus.cc:359] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.049757  2397 raft_consensus.cc:385] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:11.050477  2397 raft_consensus.cc:740] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Initialized, Role: FOLLOWER
I20260709 12:22:11.050135  2398 tablet_bootstrap.cc:492] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1: Bootstrap starting.
I20260709 12:22:11.058223  2397 consensus_queue.cc:260] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.059536  2398 tablet_bootstrap.cc:654] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:11.063891  2397 ts_tablet_manager.cc:1434] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.003s
I20260709 12:22:11.064965  2397 tablet_bootstrap.cc:492] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53: Bootstrap starting.
I20260709 12:22:11.071008  2397 tablet_bootstrap.cc:654] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:11.077548  2403 raft_consensus.cc:493] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.078155  2403 raft_consensus.cc:515] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.080317  2403 leader_election.cc:290] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:11.081813  2279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c011ea4eb8c4263918f63ca65871907" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53" is_pre_election: true
I20260709 12:22:11.082604  2279 raft_consensus.cc:2468] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 0.
I20260709 12:22:11.084455  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c011ea4eb8c4263918f63ca65871907" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1" is_pre_election: true
I20260709 12:22:11.085229  2354 raft_consensus.cc:2468] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 0.
I20260709 12:22:11.086431  2166 leader_election.cc:304] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:11.087387  2403 raft_consensus.cc:2804] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:11.087855  2403 raft_consensus.cc:493] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:11.088239  2403 raft_consensus.cc:3060] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.088447  2397 tablet_bootstrap.cc:492] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53: No bootstrap required, opened a new log
I20260709 12:22:11.088915  2397 ts_tablet_manager.cc:1403] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.005s
I20260709 12:22:11.091231  2398 tablet_bootstrap.cc:492] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1: No bootstrap required, opened a new log
I20260709 12:22:11.091892  2398 ts_tablet_manager.cc:1403] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.017s
I20260709 12:22:11.091768  2397 raft_consensus.cc:359] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.087404  2399 tablet_bootstrap.cc:654] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:11.092661  2397 raft_consensus.cc:385] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:11.093156  2397 raft_consensus.cc:740] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Initialized, Role: FOLLOWER
I20260709 12:22:11.093909  2397 consensus_queue.cc:260] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.095525  2403 raft_consensus.cc:515] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.096438  2398 raft_consensus.cc:359] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.097687  2403 leader_election.cc:290] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 election: Requested vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:11.099432  2397 ts_tablet_manager.cc:1434] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.004s
I20260709 12:22:11.099538  2279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c011ea4eb8c4263918f63ca65871907" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:11.100260  2279 raft_consensus.cc:3060] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.102099  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c011ea4eb8c4263918f63ca65871907" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:11.102662  2354 raft_consensus.cc:3060] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.103547  2399 tablet_bootstrap.cc:492] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb: No bootstrap required, opened a new log
I20260709 12:22:11.104086  2399 ts_tablet_manager.cc:1403] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb: Time spent bootstrapping tablet: real 0.064s	user 0.007s	sys 0.007s
I20260709 12:22:11.107430  2399 raft_consensus.cc:359] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.108347  2399 raft_consensus.cc:385] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:11.108675  2399 raft_consensus.cc:740] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Initialized, Role: FOLLOWER
I20260709 12:22:11.109396  2399 consensus_queue.cc:260] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.112025  2399 ts_tablet_manager.cc:1434] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.001s
I20260709 12:22:11.113078  2399 tablet_bootstrap.cc:492] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb: Bootstrap starting.
I20260709 12:22:11.113862  2279 raft_consensus.cc:2468] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 1.
I20260709 12:22:11.113891  2354 raft_consensus.cc:2468] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 1.
I20260709 12:22:11.115370  2166 leader_election.cc:304] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:11.116232  2403 raft_consensus.cc:2804] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:11.116784  2403 raft_consensus.cc:697] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 1 LEADER]: Becoming Leader. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Running, Role: LEADER
I20260709 12:22:11.117781  2398 raft_consensus.cc:385] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:11.118120  2398 raft_consensus.cc:740] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Initialized, Role: FOLLOWER
I20260709 12:22:11.117599  2403 consensus_queue.cc:237] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.123476  2407 consensus_queue.cc:1048] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:11.125069  2398 consensus_queue.cc:260] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.126588  2408 raft_consensus.cc:493] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.127102  2408 raft_consensus.cc:515] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.127934  2398 ts_tablet_manager.cc:1434] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 12:22:11.129601  2398 tablet_bootstrap.cc:492] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1: Bootstrap starting.
I20260709 12:22:11.130421  2204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f984985863e44299701c48dba1f4315" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb" is_pre_election: true
I20260709 12:22:11.129331  2408 leader_election.cc:290] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805)
I20260709 12:22:11.123832  2399 tablet_bootstrap.cc:654] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:11.132066  2277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f984985863e44299701c48dba1f4315" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53" is_pre_election: true
I20260709 12:22:11.132841  2277 raft_consensus.cc:2468] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 0.
I20260709 12:22:11.134418  2316 leader_election.cc:304] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, b642f379bf7c4533aba8f266d7d0fe53; no voters: 
I20260709 12:22:11.135412  2408 raft_consensus.cc:2804] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:11.135813  2408 raft_consensus.cc:493] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:11.136194  2408 raft_consensus.cc:3060] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.137017  2398 tablet_bootstrap.cc:654] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:11.131124  2204 raft_consensus.cc:2468] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 0.
I20260709 12:22:11.157492  2399 tablet_bootstrap.cc:492] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb: No bootstrap required, opened a new log
I20260709 12:22:11.158588  2407 consensus_queue.cc:1048] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:11.160523  2399 ts_tablet_manager.cc:1403] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.014s
I20260709 12:22:11.163833  2420 raft_consensus.cc:493] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.164650  2420 raft_consensus.cc:515] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:11.165091  2399 raft_consensus.cc:359] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.166278  2399 raft_consensus.cc:385] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:11.166716  2399 raft_consensus.cc:740] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Initialized, Role: FOLLOWER
I20260709 12:22:11.169610  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1" is_pre_election: true
W20260709 12:22:11.172242  2243 leader_election.cc:343] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:11.172991  2203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb" is_pre_election: true
I20260709 12:22:11.174259  2203 raft_consensus.cc:2468] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 0.
I20260709 12:22:11.176056  2420 leader_election.cc:290] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031), c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261)
I20260709 12:22:11.168301  2399 consensus_queue.cc:260] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.177340  2243 leader_election.cc:304] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb; no voters: 91e3a0484e714200a61187a599eca1a1
I20260709 12:22:11.179541  2404 raft_consensus.cc:2804] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:11.179776  2399 ts_tablet_manager.cc:1434] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:22:11.180240  2404 raft_consensus.cc:493] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:11.180812  2404 raft_consensus.cc:3060] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.181265  2399 tablet_bootstrap.cc:492] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb: Bootstrap starting.
I20260709 12:22:11.186115  2420 consensus_queue.cc:1048] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:11.189281  2399 tablet_bootstrap.cc:654] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:11.225437  2399 tablet_bootstrap.cc:492] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb: No bootstrap required, opened a new log
I20260709 12:22:11.225437  2398 tablet_bootstrap.cc:492] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1: No bootstrap required, opened a new log
I20260709 12:22:11.203351  2404 raft_consensus.cc:515] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:11.205109  2408 raft_consensus.cc:515] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.205845  2098 catalog_manager.cc:5697] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb reported cstate change: term changed from 0 to 1, leader changed from <none> to c2759c2eeae9433d807b067536179bfb (127.2.4.193). New cstate: current_term: 1 leader_uuid: "c2759c2eeae9433d807b067536179bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:11.227147  2399 ts_tablet_manager.cc:1403] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb: Time spent bootstrapping tablet: real 0.046s	user 0.006s	sys 0.007s
I20260709 12:22:11.237149  2203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f984985863e44299701c48dba1f4315" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:11.238072  2203 raft_consensus.cc:3060] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.231945  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:11.241006  2399 raft_consensus.cc:359] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.242046  2399 raft_consensus.cc:385] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:11.237171  2398 ts_tablet_manager.cc:1403] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1: Time spent bootstrapping tablet: real 0.108s	user 0.038s	sys 0.053s
I20260709 12:22:11.242395  2399 raft_consensus.cc:740] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Initialized, Role: FOLLOWER
I20260709 12:22:11.264489  2399 consensus_queue.cc:260] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.268841  2277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f984985863e44299701c48dba1f4315" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:11.269847  2277 raft_consensus.cc:3060] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.282353  2408 leader_election.cc:290] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 election: Requested vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805)
I20260709 12:22:11.283962  2398 raft_consensus.cc:359] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:11.285333  2398 raft_consensus.cc:385] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:11.285980  2398 raft_consensus.cc:740] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Initialized, Role: FOLLOWER
I20260709 12:22:11.288373  2399 ts_tablet_manager.cc:1434] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.013s
I20260709 12:22:11.289834  2398 consensus_queue.cc:260] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
W20260709 12:22:11.283636  2243 leader_election.cc:343] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:11.289291  2202 raft_consensus.cc:3060] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.277102  2404 leader_election.cc:290] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 election: Requested vote from peers 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031), c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261)
I20260709 12:22:11.304571  2201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:11.305270  2201 raft_consensus.cc:3060] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.307981  2398 ts_tablet_manager.cc:1434] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1: Time spent starting tablet: real 0.065s	user 0.010s	sys 0.000s
I20260709 12:22:11.315237  2398 tablet_bootstrap.cc:492] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1: Bootstrap starting.
I20260709 12:22:11.331269  2203 raft_consensus.cc:2468] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 1.
I20260709 12:22:11.332705  2317 leader_election.cc:304] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:11.354001  2451 raft_consensus.cc:2804] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:11.354610  2451 raft_consensus.cc:697] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 1 LEADER]: Becoming Leader. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Running, Role: LEADER
I20260709 12:22:11.355504  2451 consensus_queue.cc:237] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.361259  2279 raft_consensus.cc:3060] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.362412  2398 tablet_bootstrap.cc:654] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:11.366298  2203 raft_consensus.cc:3060] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.359544  2277 raft_consensus.cc:2468] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 1.
I20260709 12:22:11.381337  2201 raft_consensus.cc:2468] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 1.
I20260709 12:22:11.382912  2243 leader_election.cc:304] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb; no voters: 91e3a0484e714200a61187a599eca1a1
I20260709 12:22:11.384676  2404 raft_consensus.cc:2804] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:11.388096  2403 raft_consensus.cc:493] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.387324  2404 raft_consensus.cc:697] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 LEADER]: Becoming Leader. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:11.388942  2403 raft_consensus.cc:515] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.389781  2404 consensus_queue.cc:237] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:11.397917  2432 consensus_queue.cc:1048] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:11.395498  2420 raft_consensus.cc:493] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.402236  2420 raft_consensus.cc:515] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.409986  2420 leader_election.cc:290] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:11.410810  2420 consensus_queue.cc:1048] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:11.410609  2098 catalog_manager.cc:5697] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91e3a0484e714200a61187a599eca1a1 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "91e3a0484e714200a61187a599eca1a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:11.412402  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91323316a13b463db86243d302b1fbe5" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1" is_pre_election: true
I20260709 12:22:11.415441  2403 leader_election.cc:290] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:11.423652  2398 tablet_bootstrap.cc:492] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1: No bootstrap required, opened a new log
I20260709 12:22:11.424271  2398 ts_tablet_manager.cc:1403] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1: Time spent bootstrapping tablet: real 0.109s	user 0.017s	sys 0.004s
I20260709 12:22:11.427237  2279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91323316a13b463db86243d302b1fbe5" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53" is_pre_election: true
I20260709 12:22:11.428562  2398 raft_consensus.cc:359] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.428858  2166 leader_election.cc:304] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c2759c2eeae9433d807b067536179bfb; no voters: 91e3a0484e714200a61187a599eca1a1, b642f379bf7c4533aba8f266d7d0fe53
I20260709 12:22:11.429347  2398 raft_consensus.cc:385] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:11.429741  2398 raft_consensus.cc:740] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Initialized, Role: FOLLOWER
I20260709 12:22:11.430282  2403 raft_consensus.cc:2749] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:11.430217  2451 consensus_queue.cc:1048] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:11.430570  2398 consensus_queue.cc:260] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.436703  2398 ts_tablet_manager.cc:1434] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 12:22:11.441707  2202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d159b73e1fed4f1eab8d8caf495fc751" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb" is_pre_election: true
I20260709 12:22:11.445294  2451 consensus_queue.cc:1048] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:11.446002  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d159b73e1fed4f1eab8d8caf495fc751" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1" is_pre_election: true
I20260709 12:22:11.453389  2432 consensus_queue.cc:1048] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:22:11.459120  2243 leader_election.cc:304] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53; no voters: 91e3a0484e714200a61187a599eca1a1, c2759c2eeae9433d807b067536179bfb
I20260709 12:22:11.459220  2099 catalog_manager.cc:5697] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 reported cstate change: term changed from 0 to 1, leader changed from <none> to b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:11.468596  2451 raft_consensus.cc:493] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.469269  2404 consensus_queue.cc:1048] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:11.469362  2451 raft_consensus.cc:515] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.473966  2204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b8bf01edfc401c8c9269e27f1f48e1" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb" is_pre_election: true
I20260709 12:22:11.476364  2451 leader_election.cc:290] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805)
I20260709 12:22:11.478188  2279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b8bf01edfc401c8c9269e27f1f48e1" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53" is_pre_election: true
I20260709 12:22:11.483251  2459 raft_consensus.cc:2749] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:11.483445  2316 leader_election.cc:304] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1; no voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb
I20260709 12:22:11.484879  2451 raft_consensus.cc:2749] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:11.492969  2432 raft_consensus.cc:493] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.493818  2432 raft_consensus.cc:515] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.496596  2432 leader_election.cc:290] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:11.498415  2276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b055ff9d3d7458faaaef52610877acf" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53" is_pre_election: true
I20260709 12:22:11.498570  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b055ff9d3d7458faaaef52610877acf" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1" is_pre_election: true
I20260709 12:22:11.499461  2276 raft_consensus.cc:2468] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 0.
I20260709 12:22:11.499550  2353 raft_consensus.cc:2468] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 0.
I20260709 12:22:11.501173  2166 leader_election.cc:304] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:11.502792  2432 raft_consensus.cc:2804] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:11.505998  2432 raft_consensus.cc:493] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:11.506393  2432 raft_consensus.cc:3060] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.548233  2432 raft_consensus.cc:515] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.565994  2432 leader_election.cc:290] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 election: Requested vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:11.566246  2354 raft_consensus.cc:3060] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.567308  2278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b055ff9d3d7458faaaef52610877acf" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:11.568264  2278 raft_consensus.cc:3060] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.576467  2460 consensus_queue.cc:1048] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:11.589591  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b055ff9d3d7458faaaef52610877acf" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:11.591710  2354 raft_consensus.cc:3060] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.587131  2278 raft_consensus.cc:2468] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 1.
I20260709 12:22:11.596515  2166 leader_election.cc:304] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:11.597409  2432 raft_consensus.cc:2804] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:11.597924  2432 raft_consensus.cc:697] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 1 LEADER]: Becoming Leader. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Running, Role: LEADER
I20260709 12:22:11.603588  2354 raft_consensus.cc:2468] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 1.
I20260709 12:22:11.598770  2432 consensus_queue.cc:237] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.623215  2432 consensus_queue.cc:1048] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:11.626914  2099 catalog_manager.cc:5697] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb reported cstate change: term changed from 0 to 1, leader changed from <none> to c2759c2eeae9433d807b067536179bfb (127.2.4.193). New cstate: current_term: 1 leader_uuid: "c2759c2eeae9433d807b067536179bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:11.636332  2403 consensus_queue.cc:1048] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:11.656334  2449 raft_consensus.cc:493] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.656936  2449 raft_consensus.cc:515] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:11.659253  2449 leader_election.cc:290] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261)
I20260709 12:22:11.660566  2201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2759c2eeae9433d807b067536179bfb" is_pre_election: true
I20260709 12:22:11.661409  2201 raft_consensus.cc:2393] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91e3a0484e714200a61187a599eca1a1 in current term 1: Already voted for candidate b642f379bf7c4533aba8f266d7d0fe53 in this term.
I20260709 12:22:11.663833  2278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53" is_pre_election: true
I20260709 12:22:11.665273  2316 leader_election.cc:304] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1; no voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb
I20260709 12:22:11.666040  2449 raft_consensus.cc:3060] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:11.673183  2449 raft_consensus.cc:2749] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:11.741585  2278 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4"
dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
mode: GRACEFUL
new_leader_uuid: "c2759c2eeae9433d807b067536179bfb"
 from {username='slave'} at 127.0.0.1:60428
I20260709 12:22:11.742349  2278 raft_consensus.cc:606] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 LEADER]: Received request to transfer leadership to TS c2759c2eeae9433d807b067536179bfb
I20260709 12:22:11.760736  2354 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3f2033c07ca45a183a62bac15cefbdc"
dest_uuid: "91e3a0484e714200a61187a599eca1a1"
mode: GRACEFUL
new_leader_uuid: "c2759c2eeae9433d807b067536179bfb"
 from {username='slave'} at 127.0.0.1:53348
I20260709 12:22:11.761498  2354 raft_consensus.cc:606] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 1 LEADER]: Received request to transfer leadership to TS c2759c2eeae9433d807b067536179bfb
I20260709 12:22:11.767457  2354 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03"
dest_uuid: "91e3a0484e714200a61187a599eca1a1"
mode: GRACEFUL
new_leader_uuid: "c2759c2eeae9433d807b067536179bfb"
 from {username='slave'} at 127.0.0.1:53348
I20260709 12:22:11.768242  2354 raft_consensus.cc:606] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 1 LEADER]: Received request to transfer leadership to TS c2759c2eeae9433d807b067536179bfb
I20260709 12:22:11.793139  2449 consensus_queue.cc:1048] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:11.799336  2203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d159b73e1fed4f1eab8d8caf495fc751"
dest_uuid: "c2759c2eeae9433d807b067536179bfb"
mode: GRACEFUL
new_leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
 from {username='slave'} at 127.0.0.1:55784
I20260709 12:22:11.800087  2203 raft_consensus.cc:606] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 1 LEADER]: Received request to transfer leadership to TS b642f379bf7c4533aba8f266d7d0fe53
I20260709 12:22:11.809275  2354 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f984985863e44299701c48dba1f4315"
dest_uuid: "91e3a0484e714200a61187a599eca1a1"
mode: GRACEFUL
new_leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
 from {username='slave'} at 127.0.0.1:53348
I20260709 12:22:11.810571  2354 raft_consensus.cc:606] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 1 LEADER]: Received request to transfer leadership to TS b642f379bf7c4533aba8f266d7d0fe53
I20260709 12:22:11.814639  2449 consensus_queue.cc:1048] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:11.819072  2203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b055ff9d3d7458faaaef52610877acf"
dest_uuid: "c2759c2eeae9433d807b067536179bfb"
mode: GRACEFUL
new_leader_uuid: "91e3a0484e714200a61187a599eca1a1"
 from {username='slave'} at 127.0.0.1:55784
I20260709 12:22:11.819805  2203 raft_consensus.cc:606] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 1 LEADER]: Received request to transfer leadership to TS 91e3a0484e714200a61187a599eca1a1
I20260709 12:22:11.834434  2408 raft_consensus.cc:993] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1: : Instructing follower b642f379bf7c4533aba8f266d7d0fe53 to start an election
I20260709 12:22:11.835018  2449 raft_consensus.cc:1081] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 1 LEADER]: Signalling peer b642f379bf7c4533aba8f266d7d0fe53 to start an election
I20260709 12:22:11.837416  2278 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f984985863e44299701c48dba1f4315"
dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
 from {username='slave'} at 127.0.0.1:60420
I20260709 12:22:11.838073  2278 raft_consensus.cc:493] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:11.838411  2278 raft_consensus.cc:3060] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:11.846021  2278 raft_consensus.cc:515] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.848098  2278 leader_election.cc:290] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 2 election: Requested vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:11.849566  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f984985863e44299701c48dba1f4315" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:11.850200  2201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f984985863e44299701c48dba1f4315" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:11.850327  2354 raft_consensus.cc:3055] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:11.850832  2201 raft_consensus.cc:3060] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:11.850868  2354 raft_consensus.cc:740] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Running, Role: LEADER
I20260709 12:22:11.851999  2354 consensus_queue.cc:260] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.853856  2354 raft_consensus.cc:3060] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:11.860425  2201 raft_consensus.cc:2468] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 2.
I20260709 12:22:11.862460  2354 raft_consensus.cc:2468] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 2.
I20260709 12:22:11.862572  2243 leader_election.cc:304] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:11.863729  2460 raft_consensus.cc:2804] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:11.864238  2460 raft_consensus.cc:697] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 LEADER]: Becoming Leader. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:11.865089  2460 consensus_queue.cc:237] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.875716  2096 catalog_manager.cc:5697] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 reported cstate change: term changed from 1 to 2, leader changed from 91e3a0484e714200a61187a599eca1a1 (127.2.4.195) to b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:11.908484  2460 consensus_queue.cc:1048] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:11.917502  2459 raft_consensus.cc:993] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53: : Instructing follower c2759c2eeae9433d807b067536179bfb to start an election
I20260709 12:22:11.918501  2460 raft_consensus.cc:1081] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 LEADER]: Signalling peer c2759c2eeae9433d807b067536179bfb to start an election
I20260709 12:22:11.921388  2203 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4"
dest_uuid: "c2759c2eeae9433d807b067536179bfb"
 from {username='slave'} at 127.0.0.1:55768
I20260709 12:22:11.922096  2203 raft_consensus.cc:493] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:11.922472  2203 raft_consensus.cc:3060] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:11.926469  2460 consensus_queue.cc:1048] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:11.929896  2203 raft_consensus.cc:515] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:11.931308  2201 raft_consensus.cc:1240] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Rejecting Update request from peer b642f379bf7c4533aba8f266d7d0fe53 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:11.933261  2278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:11.933967  2278 raft_consensus.cc:3055] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:11.934340  2278 raft_consensus.cc:740] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:11.935290  2278 consensus_queue.cc:260] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:11.937319  2460 consensus_queue.cc:1059] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:11.939559  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:11.940775  2278 raft_consensus.cc:3060] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:11.938011  2203 leader_election.cc:290] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 2 election: Requested vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:11.950639  2278 raft_consensus.cc:2468] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 2.
I20260709 12:22:11.952443  2166 leader_election.cc:304] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb; no voters: 91e3a0484e714200a61187a599eca1a1
I20260709 12:22:11.953475  2403 raft_consensus.cc:2804] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:11.954255  2403 raft_consensus.cc:697] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Becoming Leader. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Running, Role: LEADER
I20260709 12:22:11.955163  2403 consensus_queue.cc:237] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:11.968348  2097 catalog_manager.cc:5697] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb reported cstate change: term changed from 1 to 2, leader changed from b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194) to c2759c2eeae9433d807b067536179bfb (127.2.4.193). New cstate: current_term: 2 leader_uuid: "c2759c2eeae9433d807b067536179bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:11.986369  2468 raft_consensus.cc:993] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb: : Instructing follower b642f379bf7c4533aba8f266d7d0fe53 to start an election
I20260709 12:22:11.986868  2468 raft_consensus.cc:1081] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 1 LEADER]: Signalling peer b642f379bf7c4533aba8f266d7d0fe53 to start an election
I20260709 12:22:11.988940  2278 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d159b73e1fed4f1eab8d8caf495fc751"
dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
 from {username='slave'} at 127.0.0.1:60426
I20260709 12:22:11.989673  2278 raft_consensus.cc:493] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:11.990057  2278 raft_consensus.cc:3060] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:11.997339  2278 raft_consensus.cc:515] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:11.999663  2278 leader_election.cc:290] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 2 election: Requested vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:12.000749  2203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d159b73e1fed4f1eab8d8caf495fc751" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:12.001135  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d159b73e1fed4f1eab8d8caf495fc751" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:12.001598  2203 raft_consensus.cc:3055] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:12.001704  2353 raft_consensus.cc:3060] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.002206  2203 raft_consensus.cc:740] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Running, Role: LEADER
I20260709 12:22:12.003233  2203 consensus_queue.cc:260] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:12.004630  2203 raft_consensus.cc:3060] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.012861  2203 raft_consensus.cc:2468] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 2.
I20260709 12:22:12.014292  2353 raft_consensus.cc:2468] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 2.
I20260709 12:22:12.014314  2243 leader_election.cc:304] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:12.015799  2420 raft_consensus.cc:2804] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:12.016333  2420 raft_consensus.cc:697] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 LEADER]: Becoming Leader. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:12.017165  2420 consensus_queue.cc:237] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:12.024674  2096 catalog_manager.cc:5697] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 reported cstate change: term changed from 1 to 2, leader changed from c2759c2eeae9433d807b067536179bfb (127.2.4.193) to b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:12.084066  2408 raft_consensus.cc:993] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1: : Instructing follower c2759c2eeae9433d807b067536179bfb to start an election
I20260709 12:22:12.084679  2449 raft_consensus.cc:1081] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 1 LEADER]: Signalling peer c2759c2eeae9433d807b067536179bfb to start an election
I20260709 12:22:12.086890  2203 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03"
dest_uuid: "c2759c2eeae9433d807b067536179bfb"
 from {username='slave'} at 127.0.0.1:55772
I20260709 12:22:12.088222  2203 raft_consensus.cc:493] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:12.089335  2203 raft_consensus.cc:3060] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.093612  2468 consensus_queue.cc:1048] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:12.099370  2203 raft_consensus.cc:515] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:12.103581  2276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:12.104357  2276 raft_consensus.cc:3060] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.105157  2203 leader_election.cc:290] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 2 election: Requested vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:12.106124  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:12.106859  2353 raft_consensus.cc:3055] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:12.107520  2353 raft_consensus.cc:740] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Running, Role: LEADER
I20260709 12:22:12.108574  2353 consensus_queue.cc:260] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:12.120754  2276 raft_consensus.cc:2468] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 2.
I20260709 12:22:12.124122  2166 leader_election.cc:304] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:12.125303  2432 raft_consensus.cc:2804] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:12.125984  2432 raft_consensus.cc:697] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Becoming Leader. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Running, Role: LEADER
I20260709 12:22:12.126435  2353 raft_consensus.cc:3060] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.127142  2432 consensus_queue.cc:237] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:12.138375  2353 raft_consensus.cc:2468] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 2.
I20260709 12:22:12.129698  2471 consensus_queue.cc:1048] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:22:12.152627  2097 catalog_manager.cc:5697] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb reported cstate change: term changed from 1 to 2, leader changed from 91e3a0484e714200a61187a599eca1a1 (127.2.4.195) to c2759c2eeae9433d807b067536179bfb (127.2.4.193). New cstate: current_term: 2 leader_uuid: "c2759c2eeae9433d807b067536179bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:12.156947  2432 raft_consensus.cc:993] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb: : Instructing follower 91e3a0484e714200a61187a599eca1a1 to start an election
I20260709 12:22:12.157493  2432 raft_consensus.cc:1081] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 1 LEADER]: Signalling peer 91e3a0484e714200a61187a599eca1a1 to start an election
I20260709 12:22:12.162633  2353 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b055ff9d3d7458faaaef52610877acf"
dest_uuid: "91e3a0484e714200a61187a599eca1a1"
 from {username='slave'} at 127.0.0.1:53336
I20260709 12:22:12.163314  2353 raft_consensus.cc:493] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:12.163765  2353 raft_consensus.cc:3060] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.175173  2353 raft_consensus.cc:515] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:12.177492  2353 leader_election.cc:290] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 2 election: Requested vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805)
I20260709 12:22:12.179466  2203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b055ff9d3d7458faaaef52610877acf" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:12.180683  2203 raft_consensus.cc:3055] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:12.181068  2203 raft_consensus.cc:740] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Running, Role: LEADER
I20260709 12:22:12.180889  2276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b055ff9d3d7458faaaef52610877acf" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:12.181936  2276 raft_consensus.cc:3060] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.182204  2203 consensus_queue.cc:260] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:12.184185  2203 raft_consensus.cc:3060] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.189986  2276 raft_consensus.cc:2468] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 2.
I20260709 12:22:12.191514  2316 leader_election.cc:304] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, b642f379bf7c4533aba8f266d7d0fe53; no voters: 
I20260709 12:22:12.192493  2449 raft_consensus.cc:2804] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:12.193192  2449 raft_consensus.cc:697] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [term 2 LEADER]: Becoming Leader. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Running, Role: LEADER
I20260709 12:22:12.195495  2203 raft_consensus.cc:2468] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 2.
I20260709 12:22:12.194308  2449 consensus_queue.cc:237] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:12.206226  2096 catalog_manager.cc:5697] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 reported cstate change: term changed from 1 to 2, leader changed from c2759c2eeae9433d807b067536179bfb (127.2.4.193) to 91e3a0484e714200a61187a599eca1a1 (127.2.4.195). New cstate: current_term: 2 leader_uuid: "91e3a0484e714200a61187a599eca1a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:12.291714  2408 raft_consensus.cc:993] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1: : Instructing follower c2759c2eeae9433d807b067536179bfb to start an election
I20260709 12:22:12.292327  2408 raft_consensus.cc:1081] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 1 LEADER]: Signalling peer c2759c2eeae9433d807b067536179bfb to start an election
I20260709 12:22:12.294384  2203 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d3f2033c07ca45a183a62bac15cefbdc"
dest_uuid: "c2759c2eeae9433d807b067536179bfb"
 from {username='slave'} at 127.0.0.1:55772
I20260709 12:22:12.295054  2203 raft_consensus.cc:493] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:12.295435  2203 raft_consensus.cc:3060] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.303032  2203 raft_consensus.cc:515] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:12.305322  2203 leader_election.cc:290] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 2 election: Requested vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:12.306524  2276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f2033c07ca45a183a62bac15cefbdc" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:12.306600  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f2033c07ca45a183a62bac15cefbdc" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:12.307184  2276 raft_consensus.cc:3060] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.307317  2353 raft_consensus.cc:3055] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:12.307807  2353 raft_consensus.cc:740] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Running, Role: LEADER
I20260709 12:22:12.308812  2353 consensus_queue.cc:260] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:12.310076  2353 raft_consensus.cc:3060] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.312230  2203 raft_consensus.cc:1275] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Refusing update from remote peer b642f379bf7c4533aba8f266d7d0fe53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:12.313836  2420 consensus_queue.cc:1048] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:12.315744  2276 raft_consensus.cc:2468] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 2.
I20260709 12:22:12.317133  2166 leader_election.cc:304] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:12.318197  2468 raft_consensus.cc:2804] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:12.318738  2468 raft_consensus.cc:697] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Becoming Leader. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Running, Role: LEADER
I20260709 12:22:12.319550  2468 consensus_queue.cc:237] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:12.323894  2354 raft_consensus.cc:1275] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Refusing update from remote peer b642f379bf7c4533aba8f266d7d0fe53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:12.325348  2353 raft_consensus.cc:2468] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 2.
I20260709 12:22:12.327332  2404 consensus_queue.cc:1048] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:12.343070  2097 catalog_manager.cc:5697] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb reported cstate change: term changed from 1 to 2, leader changed from 91e3a0484e714200a61187a599eca1a1 (127.2.4.195) to c2759c2eeae9433d807b067536179bfb (127.2.4.193). New cstate: current_term: 2 leader_uuid: "c2759c2eeae9433d807b067536179bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:12.365043  2353 raft_consensus.cc:3060] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.372617  2353 raft_consensus.cc:1275] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Refusing update from remote peer c2759c2eeae9433d807b067536179bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:12.374434  2468 consensus_queue.cc:1048] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:12.393620  2276 raft_consensus.cc:1275] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Refusing update from remote peer c2759c2eeae9433d807b067536179bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:12.396064  2468 consensus_queue.cc:1048] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:12.473389  2203 raft_consensus.cc:1275] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Refusing update from remote peer b642f379bf7c4533aba8f266d7d0fe53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:12.476943  2404 consensus_queue.cc:1048] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:12.490703  2353 raft_consensus.cc:1275] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Refusing update from remote peer b642f379bf7c4533aba8f266d7d0fe53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:12.492702  2460 consensus_queue.cc:1048] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:12.510577  2276 raft_consensus.cc:1275] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Refusing update from remote peer c2759c2eeae9433d807b067536179bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:12.512583  2468 consensus_queue.cc:1048] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:12.529528  2353 raft_consensus.cc:1275] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Refusing update from remote peer c2759c2eeae9433d807b067536179bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:12.531832  2403 consensus_queue.cc:1048] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:12.627143  2276 raft_consensus.cc:1275] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Refusing update from remote peer 91e3a0484e714200a61187a599eca1a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:12.629370  2408 consensus_queue.cc:1048] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:12.641089  2203 raft_consensus.cc:1275] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Refusing update from remote peer 91e3a0484e714200a61187a599eca1a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:12.643779  2449 consensus_queue.cc:1048] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:12.778685  2276 raft_consensus.cc:1275] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Refusing update from remote peer c2759c2eeae9433d807b067536179bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:12.780048  2432 consensus_queue.cc:1048] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:12.790130  2353 raft_consensus.cc:1275] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Refusing update from remote peer c2759c2eeae9433d807b067536179bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:12.792155  2471 consensus_queue.cc:1048] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:14.863185  2067 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:22:14.864140  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:14.890934  2276 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f984985863e44299701c48dba1f4315"
dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
mode: GRACEFUL
new_leader_uuid: "91e3a0484e714200a61187a599eca1a1"
 from {username='slave'} at 127.0.0.1:60438
I20260709 12:22:14.891533  2276 raft_consensus.cc:606] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 LEADER]: Received request to transfer leadership to TS 91e3a0484e714200a61187a599eca1a1
I20260709 12:22:14.905529  2203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03"
dest_uuid: "c2759c2eeae9433d807b067536179bfb"
mode: GRACEFUL
new_leader_uuid: "91e3a0484e714200a61187a599eca1a1"
 from {username='slave'} at 127.0.0.1:55786
I20260709 12:22:14.906234  2203 raft_consensus.cc:606] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Received request to transfer leadership to TS 91e3a0484e714200a61187a599eca1a1
I20260709 12:22:14.907698  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:14.908378  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:15.290025  2432 raft_consensus.cc:993] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb: : Instructing follower 91e3a0484e714200a61187a599eca1a1 to start an election
I20260709 12:22:15.290498  2497 raft_consensus.cc:1081] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Signalling peer 91e3a0484e714200a61187a599eca1a1 to start an election
I20260709 12:22:15.292356  2354 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03"
dest_uuid: "91e3a0484e714200a61187a599eca1a1"
 from {username='slave'} at 127.0.0.1:53336
I20260709 12:22:15.292912  2354 raft_consensus.cc:493] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:15.293251  2354 raft_consensus.cc:3060] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:15.298602  2354 raft_consensus.cc:515] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:15.301203  2203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:15.301872  2203 raft_consensus.cc:3055] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:15.302212  2203 raft_consensus.cc:740] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Running, Role: LEADER
I20260709 12:22:15.302841  2276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:15.302996  2203 consensus_queue.cc:260] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:15.303577  2276 raft_consensus.cc:3060] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:15.301216  2354 leader_election.cc:290] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 3 election: Requested vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805)
I20260709 12:22:15.304136  2203 raft_consensus.cc:3060] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:15.309814  2203 raft_consensus.cc:2468] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 3.
I20260709 12:22:15.309913  2276 raft_consensus.cc:2468] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 3.
I20260709 12:22:15.311198  2317 leader_election.cc:304] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:15.312009  2490 raft_consensus.cc:2804] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:15.313266  2490 raft_consensus.cc:697] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 3 LEADER]: Becoming Leader. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Running, Role: LEADER
I20260709 12:22:15.314179  2490 consensus_queue.cc:237] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:15.321534  2097 catalog_manager.cc:5697] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 reported cstate change: term changed from 2 to 3, leader changed from c2759c2eeae9433d807b067536179bfb (127.2.4.193) to 91e3a0484e714200a61187a599eca1a1 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "91e3a0484e714200a61187a599eca1a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:15.339288  2505 raft_consensus.cc:993] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53: : Instructing follower 91e3a0484e714200a61187a599eca1a1 to start an election
I20260709 12:22:15.339917  2404 raft_consensus.cc:1081] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 LEADER]: Signalling peer 91e3a0484e714200a61187a599eca1a1 to start an election
I20260709 12:22:15.341760  2353 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f984985863e44299701c48dba1f4315"
dest_uuid: "91e3a0484e714200a61187a599eca1a1"
 from {username='slave'} at 127.0.0.1:53328
I20260709 12:22:15.342622  2353 raft_consensus.cc:493] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:15.343088  2353 raft_consensus.cc:3060] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:15.348274  2353 raft_consensus.cc:515] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:15.350123  2353 leader_election.cc:290] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 3 election: Requested vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805)
I20260709 12:22:15.351454  2203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f984985863e44299701c48dba1f4315" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:15.351490  2276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f984985863e44299701c48dba1f4315" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:15.352530  2203 raft_consensus.cc:3060] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:15.352537  2276 raft_consensus.cc:3055] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:15.353107  2276 raft_consensus.cc:740] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:15.353899  2276 consensus_queue.cc:260] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:15.354916  2276 raft_consensus.cc:3060] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:15.358964  2203 raft_consensus.cc:2468] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 3.
I20260709 12:22:15.359988  2276 raft_consensus.cc:2468] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 3.
I20260709 12:22:15.360391  2317 leader_election.cc:304] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:15.361433  2490 raft_consensus.cc:2804] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:15.362028  2490 raft_consensus.cc:697] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 3 LEADER]: Becoming Leader. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Running, Role: LEADER
I20260709 12:22:15.362728  2490 consensus_queue.cc:237] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:15.371765  2097 catalog_manager.cc:5697] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 reported cstate change: term changed from 2 to 3, leader changed from b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194) to 91e3a0484e714200a61187a599eca1a1 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "91e3a0484e714200a61187a599eca1a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:15.762233  2276 raft_consensus.cc:1275] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Refusing update from remote peer 91e3a0484e714200a61187a599eca1a1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:15.763859  2490 consensus_queue.cc:1048] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:15.776093  2203 raft_consensus.cc:1275] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Refusing update from remote peer 91e3a0484e714200a61187a599eca1a1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:15.777717  2490 consensus_queue.cc:1048] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:15.876775  2203 raft_consensus.cc:1275] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Refusing update from remote peer 91e3a0484e714200a61187a599eca1a1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:15.878455  2490 consensus_queue.cc:1048] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:15.890636  2276 raft_consensus.cc:1275] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Refusing update from remote peer 91e3a0484e714200a61187a599eca1a1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:15.892585  2498 consensus_queue.cc:1048] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:15.909655  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:15.915123  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:15.915649  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:16.916826  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:16.924454  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:16.924948  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:17.926635  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:17.933295  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:17.933835  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:18.935124  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:18.940318  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:18.940840  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:19.942015  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:19.947985  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:19.948681  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:20.950003  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:20.955345  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:20.955890  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:21.957083  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:21.962646  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:21.963179  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:22.964325  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:22.969439  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:22.969940  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:23.971153  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:23.976588  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:23.977159  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:25.014535  2067 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:22:25.021256  2203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4"
dest_uuid: "c2759c2eeae9433d807b067536179bfb"
mode: GRACEFUL
new_leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
 from {username='slave'} at 127.0.0.1:55784
I20260709 12:22:25.022281  2203 raft_consensus.cc:606] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Received request to transfer leadership to TS b642f379bf7c4533aba8f266d7d0fe53
I20260709 12:22:25.029295  2203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3f2033c07ca45a183a62bac15cefbdc"
dest_uuid: "c2759c2eeae9433d807b067536179bfb"
mode: GRACEFUL
new_leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
 from {username='slave'} at 127.0.0.1:55784
I20260709 12:22:25.030088  2203 raft_consensus.cc:606] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Received request to transfer leadership to TS b642f379bf7c4533aba8f266d7d0fe53
I20260709 12:22:25.037386  2354 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03"
dest_uuid: "91e3a0484e714200a61187a599eca1a1"
mode: GRACEFUL
new_leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
 from {username='slave'} at 127.0.0.1:53348
I20260709 12:22:25.037962  2354 raft_consensus.cc:606] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 3 LEADER]: Received request to transfer leadership to TS b642f379bf7c4533aba8f266d7d0fe53
I20260709 12:22:25.044190  2203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c011ea4eb8c4263918f63ca65871907"
dest_uuid: "c2759c2eeae9433d807b067536179bfb"
mode: GRACEFUL
new_leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
 from {username='slave'} at 127.0.0.1:55784
I20260709 12:22:25.044845  2203 raft_consensus.cc:606] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 1 LEADER]: Received request to transfer leadership to TS b642f379bf7c4533aba8f266d7d0fe53
I20260709 12:22:25.061781  2354 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f984985863e44299701c48dba1f4315"
dest_uuid: "91e3a0484e714200a61187a599eca1a1"
mode: GRACEFUL
new_leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
 from {username='slave'} at 127.0.0.1:53348
I20260709 12:22:25.062441  2354 raft_consensus.cc:606] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 3 LEADER]: Received request to transfer leadership to TS b642f379bf7c4533aba8f266d7d0fe53
I20260709 12:22:25.330039  2525 raft_consensus.cc:993] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb: : Instructing follower b642f379bf7c4533aba8f266d7d0fe53 to start an election
I20260709 12:22:25.330704  2528 raft_consensus.cc:1081] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Signalling peer b642f379bf7c4533aba8f266d7d0fe53 to start an election
I20260709 12:22:25.332906  2276 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4"
dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
 from {username='slave'} at 127.0.0.1:60426
I20260709 12:22:25.333530  2276 raft_consensus.cc:493] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:25.333927  2276 raft_consensus.cc:3060] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:25.339113  2276 raft_consensus.cc:515] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:25.340896  2276 leader_election.cc:290] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 3 election: Requested vote from peers 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031), c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261)
I20260709 12:22:25.342198  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:25.342665  2203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:25.342941  2354 raft_consensus.cc:3060] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:25.343336  2203 raft_consensus.cc:3055] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:25.343634  2203 raft_consensus.cc:740] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Running, Role: LEADER
I20260709 12:22:25.344476  2203 consensus_queue.cc:260] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:25.345592  2203 raft_consensus.cc:3060] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:25.348248  2354 raft_consensus.cc:2468] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 3.
I20260709 12:22:25.350360  2243 leader_election.cc:304] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, b642f379bf7c4533aba8f266d7d0fe53; no voters: 
I20260709 12:22:25.352694  2516 raft_consensus.cc:2804] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:25.353528  2516 raft_consensus.cc:697] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 LEADER]: Becoming Leader. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:25.354774  2520 raft_consensus.cc:993] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1: : Instructing follower b642f379bf7c4533aba8f266d7d0fe53 to start an election
I20260709 12:22:25.355067  2203 raft_consensus.cc:2468] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 3.
I20260709 12:22:25.354995  2516 consensus_queue.cc:237] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:25.355291  2520 raft_consensus.cc:1081] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 3 LEADER]: Signalling peer b642f379bf7c4533aba8f266d7d0fe53 to start an election
I20260709 12:22:25.358676  2276 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03"
dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
 from {username='slave'} at 127.0.0.1:60420
I20260709 12:22:25.359261  2276 raft_consensus.cc:493] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:25.359583  2276 raft_consensus.cc:3060] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:25.365989  2276 raft_consensus.cc:515] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:25.365996  2096 catalog_manager.cc:5697] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 reported cstate change: term changed from 2 to 3, leader changed from c2759c2eeae9433d807b067536179bfb (127.2.4.193) to b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194). New cstate: current_term: 3 leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:25.368129  2276 leader_election.cc:290] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 4 election: Requested vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:25.369089  2203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:25.369829  2203 raft_consensus.cc:3060] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:25.370050  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:25.370831  2354 raft_consensus.cc:3055] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:25.371299  2354 raft_consensus.cc:740] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Running, Role: LEADER
I20260709 12:22:25.372203  2354 consensus_queue.cc:260] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:25.373261  2354 raft_consensus.cc:3060] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:25.379977  2354 raft_consensus.cc:2468] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 4.
I20260709 12:22:25.381361  2243 leader_election.cc:304] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, b642f379bf7c4533aba8f266d7d0fe53; no voters: 
I20260709 12:22:25.381553  2203 raft_consensus.cc:2468] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 4.
I20260709 12:22:25.382341  2516 raft_consensus.cc:2804] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:25.382859  2516 raft_consensus.cc:697] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 LEADER]: Becoming Leader. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:25.383723  2516 consensus_queue.cc:237] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:25.399257  2096 catalog_manager.cc:5697] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 reported cstate change: term changed from 3 to 4, leader changed from 91e3a0484e714200a61187a599eca1a1 (127.2.4.195) to b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194). New cstate: current_term: 4 leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:25.400153  2520 raft_consensus.cc:993] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1: : Instructing follower b642f379bf7c4533aba8f266d7d0fe53 to start an election
I20260709 12:22:25.400938  2529 raft_consensus.cc:1081] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 3 LEADER]: Signalling peer b642f379bf7c4533aba8f266d7d0fe53 to start an election
I20260709 12:22:25.402798  2276 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f984985863e44299701c48dba1f4315"
dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
 from {username='slave'} at 127.0.0.1:60420
I20260709 12:22:25.403313  2276 raft_consensus.cc:493] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:25.403630  2276 raft_consensus.cc:3060] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:25.410598  2276 raft_consensus.cc:515] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:25.412954  2276 leader_election.cc:290] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 4 election: Requested vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:25.413790  2203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f984985863e44299701c48dba1f4315" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:25.414150  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f984985863e44299701c48dba1f4315" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:25.414557  2203 raft_consensus.cc:3060] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:25.414837  2354 raft_consensus.cc:3055] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:25.415198  2354 raft_consensus.cc:740] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Running, Role: LEADER
I20260709 12:22:25.416007  2354 consensus_queue.cc:260] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:25.416958  2354 raft_consensus.cc:3060] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:25.420029  2203 raft_consensus.cc:2468] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 4.
I20260709 12:22:25.421308  2243 leader_election.cc:304] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:25.421737  2354 raft_consensus.cc:2468] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 4.
I20260709 12:22:25.422230  2516 raft_consensus.cc:2804] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:25.422689  2516 raft_consensus.cc:697] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 LEADER]: Becoming Leader. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:25.423591  2516 consensus_queue.cc:237] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:25.425514  2525 raft_consensus.cc:993] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb: : Instructing follower b642f379bf7c4533aba8f266d7d0fe53 to start an election
I20260709 12:22:25.426112  2528 raft_consensus.cc:1081] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Signalling peer b642f379bf7c4533aba8f266d7d0fe53 to start an election
I20260709 12:22:25.428014  2276 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d3f2033c07ca45a183a62bac15cefbdc"
dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
 from {username='slave'} at 127.0.0.1:60426
I20260709 12:22:25.428726  2276 raft_consensus.cc:493] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:25.429147  2276 raft_consensus.cc:3060] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:25.432297  2096 catalog_manager.cc:5697] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 reported cstate change: term changed from 3 to 4, leader changed from 91e3a0484e714200a61187a599eca1a1 (127.2.4.195) to b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194). New cstate: current_term: 4 leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:25.433872  2276 raft_consensus.cc:515] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:25.436537  2276 leader_election.cc:290] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 3 election: Requested vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:25.437355  2203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f2033c07ca45a183a62bac15cefbdc" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:25.437696  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f2033c07ca45a183a62bac15cefbdc" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:25.438067  2203 raft_consensus.cc:3055] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:25.438185  2354 raft_consensus.cc:3060] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:25.438410  2203 raft_consensus.cc:740] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Running, Role: LEADER
I20260709 12:22:25.439276  2203 consensus_queue.cc:260] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:25.440516  2203 raft_consensus.cc:3060] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:25.445433  2354 raft_consensus.cc:2468] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 3.
I20260709 12:22:25.445969  2203 raft_consensus.cc:2468] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 3.
I20260709 12:22:25.446730  2243 leader_election.cc:304] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, b642f379bf7c4533aba8f266d7d0fe53; no voters: 
I20260709 12:22:25.447556  2516 raft_consensus.cc:2804] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:25.448041  2516 raft_consensus.cc:697] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 3 LEADER]: Becoming Leader. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:25.448848  2516 consensus_queue.cc:237] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:25.456836  2096 catalog_manager.cc:5697] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 reported cstate change: term changed from 2 to 3, leader changed from c2759c2eeae9433d807b067536179bfb (127.2.4.193) to b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194). New cstate: current_term: 3 leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:25.521111  2525 raft_consensus.cc:993] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb: : Instructing follower b642f379bf7c4533aba8f266d7d0fe53 to start an election
I20260709 12:22:25.521569  2528 raft_consensus.cc:1081] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 1 LEADER]: Signalling peer b642f379bf7c4533aba8f266d7d0fe53 to start an election
I20260709 12:22:25.523304  2276 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c011ea4eb8c4263918f63ca65871907"
dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
 from {username='slave'} at 127.0.0.1:60426
I20260709 12:22:25.524037  2276 raft_consensus.cc:493] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:25.524367  2276 raft_consensus.cc:3060] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:25.529745  2276 raft_consensus.cc:515] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:25.532116  2276 leader_election.cc:290] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 2 election: Requested vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:25.533452  2203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c011ea4eb8c4263918f63ca65871907" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:25.533782  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c011ea4eb8c4263918f63ca65871907" candidate_uuid: "b642f379bf7c4533aba8f266d7d0fe53" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:25.534217  2203 raft_consensus.cc:3055] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:25.534595  2203 raft_consensus.cc:740] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Running, Role: LEADER
I20260709 12:22:25.534526  2354 raft_consensus.cc:3060] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:25.535537  2203 consensus_queue.cc:260] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:25.536814  2203 raft_consensus.cc:3060] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:25.540115  2354 raft_consensus.cc:2468] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 2.
I20260709 12:22:25.541468  2243 leader_election.cc:304] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, b642f379bf7c4533aba8f266d7d0fe53; no voters: 
I20260709 12:22:25.542537  2516 raft_consensus.cc:2804] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:25.543061  2516 raft_consensus.cc:697] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 LEADER]: Becoming Leader. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:25.543094  2203 raft_consensus.cc:2468] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b642f379bf7c4533aba8f266d7d0fe53 in term 2.
I20260709 12:22:25.544047  2516 consensus_queue.cc:237] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:25.553100  2096 catalog_manager.cc:5697] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 reported cstate change: term changed from 1 to 2, leader changed from c2759c2eeae9433d807b067536179bfb (127.2.4.193) to b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "b642f379bf7c4533aba8f266d7d0fe53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:25.765039  2203 raft_consensus.cc:1275] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 4 FOLLOWER]: Refusing update from remote peer b642f379bf7c4533aba8f266d7d0fe53: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:25.767045  2516 consensus_queue.cc:1048] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:25.781594  2354 raft_consensus.cc:1275] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 4 FOLLOWER]: Refusing update from remote peer b642f379bf7c4533aba8f266d7d0fe53: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:25.783527  2516 consensus_queue.cc:1048] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:25.804117  2203 raft_consensus.cc:1275] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Refusing update from remote peer b642f379bf7c4533aba8f266d7d0fe53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:25.805866  2527 consensus_queue.cc:1048] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:25.814883  2354 raft_consensus.cc:1275] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 4 FOLLOWER]: Refusing update from remote peer b642f379bf7c4533aba8f266d7d0fe53: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:25.816521  2541 consensus_queue.cc:1048] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:25.817097  2354 raft_consensus.cc:1275] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Refusing update from remote peer b642f379bf7c4533aba8f266d7d0fe53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:25.819571  2541 consensus_queue.cc:1048] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:25.832700  2201 raft_consensus.cc:1275] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 4 FOLLOWER]: Refusing update from remote peer b642f379bf7c4533aba8f266d7d0fe53: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:25.835307  2527 consensus_queue.cc:1048] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:25.946316  2353 raft_consensus.cc:1275] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Refusing update from remote peer b642f379bf7c4533aba8f266d7d0fe53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:25.947983  2527 consensus_queue.cc:1048] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:25.957986  2203 raft_consensus.cc:1275] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Refusing update from remote peer b642f379bf7c4533aba8f266d7d0fe53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:25.959707  2203 raft_consensus.cc:1275] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Refusing update from remote peer b642f379bf7c4533aba8f266d7d0fe53: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:25.960304  2516 consensus_queue.cc:1048] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:25.961493  2541 consensus_queue.cc:1048] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:25.975502  2353 raft_consensus.cc:1275] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Refusing update from remote peer b642f379bf7c4533aba8f266d7d0fe53: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:25.977872  2527 consensus_queue.cc:1048] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:28.108209  2067 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:22:28.109241  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:28.117262  2276 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4"
dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
mode: GRACEFUL
new_leader_uuid: "91e3a0484e714200a61187a599eca1a1"
 from {username='slave'} at 127.0.0.1:60438
I20260709 12:22:28.118125  2276 raft_consensus.cc:606] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 LEADER]: Received request to transfer leadership to TS 91e3a0484e714200a61187a599eca1a1
I20260709 12:22:28.121788  2276 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c011ea4eb8c4263918f63ca65871907"
dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
mode: GRACEFUL
new_leader_uuid: "c2759c2eeae9433d807b067536179bfb"
 from {username='slave'} at 127.0.0.1:60438
I20260709 12:22:28.122314  2276 raft_consensus.cc:606] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 LEADER]: Received request to transfer leadership to TS c2759c2eeae9433d807b067536179bfb
I20260709 12:22:28.125085  2276 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03"
dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
mode: GRACEFUL
new_leader_uuid: "c2759c2eeae9433d807b067536179bfb"
 from {username='slave'} at 127.0.0.1:60438
I20260709 12:22:28.125589  2276 raft_consensus.cc:606] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 LEADER]: Received request to transfer leadership to TS c2759c2eeae9433d807b067536179bfb
I20260709 12:22:28.128494  2276 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91323316a13b463db86243d302b1fbe5"
dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
mode: GRACEFUL
new_leader_uuid: "91e3a0484e714200a61187a599eca1a1"
 from {username='slave'} at 127.0.0.1:60438
I20260709 12:22:28.128993  2276 raft_consensus.cc:606] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 LEADER]: Received request to transfer leadership to TS 91e3a0484e714200a61187a599eca1a1
I20260709 12:22:28.132500  2276 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7b8bf01edfc401c8c9269e27f1f48e1"
dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
mode: GRACEFUL
new_leader_uuid: "c2759c2eeae9433d807b067536179bfb"
 from {username='slave'} at 127.0.0.1:60438
I20260709 12:22:28.133142  2276 raft_consensus.cc:606] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 LEADER]: Received request to transfer leadership to TS c2759c2eeae9433d807b067536179bfb
I20260709 12:22:28.134969  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:28.135699  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:28.354658  2555 raft_consensus.cc:993] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53: : Instructing follower c2759c2eeae9433d807b067536179bfb to start an election
I20260709 12:22:28.355584  2556 raft_consensus.cc:993] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53: : Instructing follower 91e3a0484e714200a61187a599eca1a1 to start an election
I20260709 12:22:28.355525  2527 raft_consensus.cc:1081] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 LEADER]: Signalling peer c2759c2eeae9433d807b067536179bfb to start an election
I20260709 12:22:28.356878  2557 raft_consensus.cc:1081] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 LEADER]: Signalling peer 91e3a0484e714200a61187a599eca1a1 to start an election
I20260709 12:22:28.358435  2201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f7b8bf01edfc401c8c9269e27f1f48e1"
dest_uuid: "c2759c2eeae9433d807b067536179bfb"
 from {username='slave'} at 127.0.0.1:55768
I20260709 12:22:28.359014  2353 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4"
dest_uuid: "91e3a0484e714200a61187a599eca1a1"
 from {username='slave'} at 127.0.0.1:53328
I20260709 12:22:28.359333  2201 raft_consensus.cc:493] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:28.359519  2353 raft_consensus.cc:493] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:28.359834  2201 raft_consensus.cc:3060] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:28.359896  2353 raft_consensus.cc:3060] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:28.368321  2353 raft_consensus.cc:515] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:28.368319  2201 raft_consensus.cc:515] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:28.371615  2201 leader_election.cc:290] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 2 election: Requested vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:28.371615  2353 leader_election.cc:290] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 4 election: Requested vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261)
I20260709 12:22:28.373575  2276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b8bf01edfc401c8c9269e27f1f48e1" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:28.374316  2278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:28.374570  2276 raft_consensus.cc:3055] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:28.373575  2201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b92f57dc0754ad8b04b6fe89d7a9fe4" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:28.374717  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b8bf01edfc401c8c9269e27f1f48e1" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:28.374882  2278 raft_consensus.cc:3055] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:28.375232  2276 raft_consensus.cc:740] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:28.375569  2278 raft_consensus.cc:740] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:28.375583  2201 raft_consensus.cc:3060] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:28.376143  2276 consensus_queue.cc:260] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:28.376521  2278 consensus_queue.cc:260] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:28.377453  2353 raft_consensus.cc:3060] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:28.377897  2276 raft_consensus.cc:3060] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:28.379549  2278 raft_consensus.cc:3060] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:28.384758  2201 raft_consensus.cc:2468] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 4.
I20260709 12:22:28.385378  2353 raft_consensus.cc:2468] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 2.
I20260709 12:22:28.386431  2276 raft_consensus.cc:2468] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 2.
I20260709 12:22:28.387019  2317 leader_election.cc:304] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:28.387071  2167 leader_election.cc:304] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:28.388320  2529 raft_consensus.cc:2804] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:28.388770  2278 raft_consensus.cc:2468] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 4.
I20260709 12:22:28.390002  2558 raft_consensus.cc:2804] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:28.391048  2529 raft_consensus.cc:697] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 4 LEADER]: Becoming Leader. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Running, Role: LEADER
I20260709 12:22:28.391722  2558 raft_consensus.cc:697] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Becoming Leader. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Running, Role: LEADER
I20260709 12:22:28.392400  2529 consensus_queue.cc:237] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } }
I20260709 12:22:28.392791  2558 consensus_queue.cc:237] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:28.403995  2097 catalog_manager.cc:5697] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb reported cstate change: term changed from 1 to 2, leader changed from b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194) to c2759c2eeae9433d807b067536179bfb (127.2.4.193). New cstate: current_term: 2 leader_uuid: "c2759c2eeae9433d807b067536179bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:28.404340  2096 catalog_manager.cc:5697] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 reported cstate change: term changed from 3 to 4, leader changed from b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194) to 91e3a0484e714200a61187a599eca1a1 (127.2.4.195). New cstate: current_term: 4 leader_uuid: "91e3a0484e714200a61187a599eca1a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:28.455515  2556 raft_consensus.cc:993] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53: : Instructing follower c2759c2eeae9433d807b067536179bfb to start an election
I20260709 12:22:28.456099  2556 raft_consensus.cc:1081] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 LEADER]: Signalling peer c2759c2eeae9433d807b067536179bfb to start an election
I20260709 12:22:28.458535  2201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c011ea4eb8c4263918f63ca65871907"
dest_uuid: "c2759c2eeae9433d807b067536179bfb"
 from {username='slave'} at 127.0.0.1:55768
I20260709 12:22:28.458997  2557 raft_consensus.cc:993] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53: : Instructing follower c2759c2eeae9433d807b067536179bfb to start an election
I20260709 12:22:28.459647  2201 raft_consensus.cc:493] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:28.459975  2556 raft_consensus.cc:1081] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 LEADER]: Signalling peer c2759c2eeae9433d807b067536179bfb to start an election
I20260709 12:22:28.460165  2201 raft_consensus.cc:3060] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:28.462286  2203 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03"
dest_uuid: "c2759c2eeae9433d807b067536179bfb"
 from {username='slave'} at 127.0.0.1:55768
I20260709 12:22:28.462854  2203 raft_consensus.cc:493] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:28.463158  2203 raft_consensus.cc:3060] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:28.467743  2201 raft_consensus.cc:515] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:28.471150  2201 leader_election.cc:290] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 3 election: Requested vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:28.470784  2203 raft_consensus.cc:515] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:28.472607  2353 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c011ea4eb8c4263918f63ca65871907" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:28.472694  2278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c011ea4eb8c4263918f63ca65871907" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:28.473383  2353 raft_consensus.cc:3060] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:28.474650  2203 leader_election.cc:290] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 5 election: Requested vote from peers b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805), 91e3a0484e714200a61187a599eca1a1 (127.2.4.195:40031)
I20260709 12:22:28.473949  2278 raft_consensus.cc:3055] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:28.476161  2354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "91e3a0484e714200a61187a599eca1a1"
I20260709 12:22:28.476899  2354 raft_consensus.cc:3060] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:28.476495  2278 raft_consensus.cc:740] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:28.477213  2276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7effe2c171054fd0a4fdbd5216c88e03" candidate_uuid: "c2759c2eeae9433d807b067536179bfb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:28.478139  2276 raft_consensus.cc:3055] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:28.478552  2276 raft_consensus.cc:740] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:28.479147  2278 consensus_queue.cc:260] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:28.479557  2276 consensus_queue.cc:260] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:28.481271  2278 raft_consensus.cc:3060] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:28.481237  2276 raft_consensus.cc:3060] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:28.483731  2353 raft_consensus.cc:2468] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 3.
I20260709 12:22:28.485054  2167 leader_election.cc:304] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:28.486124  2558 raft_consensus.cc:2804] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:28.487566  2558 raft_consensus.cc:697] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 3 LEADER]: Becoming Leader. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Running, Role: LEADER
I20260709 12:22:28.488644  2558 consensus_queue.cc:237] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:28.489243  2276 raft_consensus.cc:2468] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 5.
I20260709 12:22:28.489243  2354 raft_consensus.cc:2468] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 5.
I20260709 12:22:28.490865  2166 leader_election.cc:304] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b642f379bf7c4533aba8f266d7d0fe53, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:28.491950  2560 raft_consensus.cc:2804] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:28.492998  2278 raft_consensus.cc:2468] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2759c2eeae9433d807b067536179bfb in term 3.
I20260709 12:22:28.493595  2560 raft_consensus.cc:697] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 5 LEADER]: Becoming Leader. State: Replica: c2759c2eeae9433d807b067536179bfb, State: Running, Role: LEADER
I20260709 12:22:28.494738  2560 consensus_queue.cc:237] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:28.500072  2097 catalog_manager.cc:5697] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb reported cstate change: term changed from 2 to 3, leader changed from b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194) to c2759c2eeae9433d807b067536179bfb (127.2.4.193). New cstate: current_term: 3 leader_uuid: "c2759c2eeae9433d807b067536179bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:28.513593  2097 catalog_manager.cc:5697] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb reported cstate change: term changed from 4 to 5, leader changed from b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194) to c2759c2eeae9433d807b067536179bfb (127.2.4.193). New cstate: current_term: 5 leader_uuid: "c2759c2eeae9433d807b067536179bfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:28.685097  2556 raft_consensus.cc:993] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53: : Instructing follower 91e3a0484e714200a61187a599eca1a1 to start an election
I20260709 12:22:28.685902  2556 raft_consensus.cc:1081] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 LEADER]: Signalling peer 91e3a0484e714200a61187a599eca1a1 to start an election
I20260709 12:22:28.687788  2354 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91323316a13b463db86243d302b1fbe5"
dest_uuid: "91e3a0484e714200a61187a599eca1a1"
 from {username='slave'} at 127.0.0.1:53328
I20260709 12:22:28.688315  2354 raft_consensus.cc:493] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:28.688592  2354 raft_consensus.cc:3060] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:28.693753  2354 raft_consensus.cc:515] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:28.695730  2354 leader_election.cc:290] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 2 election: Requested vote from peers c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261), b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805)
I20260709 12:22:28.697082  2278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91323316a13b463db86243d302b1fbe5" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b642f379bf7c4533aba8f266d7d0fe53"
I20260709 12:22:28.697175  2201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91323316a13b463db86243d302b1fbe5" candidate_uuid: "91e3a0484e714200a61187a599eca1a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c2759c2eeae9433d807b067536179bfb"
I20260709 12:22:28.697878  2278 raft_consensus.cc:3055] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:28.697968  2201 raft_consensus.cc:3060] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:28.698308  2278 raft_consensus.cc:740] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b642f379bf7c4533aba8f266d7d0fe53, State: Running, Role: LEADER
I20260709 12:22:28.699314  2278 consensus_queue.cc:260] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:28.700383  2278 raft_consensus.cc:3060] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:28.704865  2201 raft_consensus.cc:2468] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 2.
I20260709 12:22:28.706439  2317 leader_election.cc:304] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e3a0484e714200a61187a599eca1a1, c2759c2eeae9433d807b067536179bfb; no voters: 
I20260709 12:22:28.706893  2278 raft_consensus.cc:2468] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91e3a0484e714200a61187a599eca1a1 in term 2.
I20260709 12:22:28.707352  2529 raft_consensus.cc:2804] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:28.707892  2529 raft_consensus.cc:697] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [term 2 LEADER]: Becoming Leader. State: Replica: 91e3a0484e714200a61187a599eca1a1, State: Running, Role: LEADER
I20260709 12:22:28.708884  2529 consensus_queue.cc:237] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } }
I20260709 12:22:28.717335  2097 catalog_manager.cc:5697] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 reported cstate change: term changed from 1 to 2, leader changed from b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194) to 91e3a0484e714200a61187a599eca1a1 (127.2.4.195). New cstate: current_term: 2 leader_uuid: "91e3a0484e714200a61187a599eca1a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:28.926553  2278 raft_consensus.cc:1275] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Refusing update from remote peer c2759c2eeae9433d807b067536179bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:28.928566  2560 consensus_queue.cc:1048] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:28.937302  2354 raft_consensus.cc:1275] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Refusing update from remote peer c2759c2eeae9433d807b067536179bfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:28.939666  2560 consensus_queue.cc:1048] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:28.965782  2201 raft_consensus.cc:1275] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 4 FOLLOWER]: Refusing update from remote peer 91e3a0484e714200a61187a599eca1a1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:28.966389  2278 raft_consensus.cc:1275] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 FOLLOWER]: Refusing update from remote peer 91e3a0484e714200a61187a599eca1a1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:28.967814  2529 consensus_queue.cc:1048] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:28.968676  2559 consensus_queue.cc:1048] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:28.977537  2276 raft_consensus.cc:1275] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 5 FOLLOWER]: Refusing update from remote peer c2759c2eeae9433d807b067536179bfb: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:28.979379  2560 consensus_queue.cc:1048] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:28.986052  2354 raft_consensus.cc:1275] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Refusing update from remote peer c2759c2eeae9433d807b067536179bfb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:28.988577  2560 consensus_queue.cc:1048] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:28.993453  2277 raft_consensus.cc:1275] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Refusing update from remote peer c2759c2eeae9433d807b067536179bfb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:28.996140  2558 consensus_queue.cc:1048] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:29.002578  2353 raft_consensus.cc:1275] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 5 FOLLOWER]: Refusing update from remote peer c2759c2eeae9433d807b067536179bfb: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:29.004702  2558 consensus_queue.cc:1048] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e3a0484e714200a61187a599eca1a1" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:22:29.137379  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:29.142308  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:29.142697  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:29.155821  2277 raft_consensus.cc:1275] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Refusing update from remote peer 91e3a0484e714200a61187a599eca1a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:29.158046  2559 consensus_queue.cc:1048] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b642f379bf7c4533aba8f266d7d0fe53" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:29.171155  2201 raft_consensus.cc:1275] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Refusing update from remote peer 91e3a0484e714200a61187a599eca1a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:29.173319  2585 consensus_queue.cc:1048] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2759c2eeae9433d807b067536179bfb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:30.144075  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:30.149767  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:30.150372  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:31.151628  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:31.156981  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:31.157379  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:32.158524  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:32.164463  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:32.164999  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:33.166296  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:33.172448  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:33.173107  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:34.174400  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:34.181277  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:34.181824  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:35.183657  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:35.189208  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:35.189692  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:36.190827  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:36.196718  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:36.197252  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:37.198664  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:37.206090  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:37.206643  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:38.246022  2067 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:22:38.246778  2067 tablet_server.cc:179] TabletServer@127.2.4.193:0 shutting down...
I20260709 12:22:38.286410  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:38.287488  2067 tablet_replica.cc:333] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb: stopping tablet replica
I20260709 12:22:38.288372  2067 raft_consensus.cc:2243] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.289294  2067 raft_consensus.cc:2272] T 0b055ff9d3d7458faaaef52610877acf P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.293051  2067 tablet_replica.cc:333] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb: stopping tablet replica
I20260709 12:22:38.294059  2067 raft_consensus.cc:2243] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.294809  2067 raft_consensus.cc:2272] T 91323316a13b463db86243d302b1fbe5 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.297183  2067 tablet_replica.cc:333] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb: stopping tablet replica
I20260709 12:22:38.297869  2067 raft_consensus.cc:2243] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.298413  2067 raft_consensus.cc:2272] T 1f984985863e44299701c48dba1f4315 P c2759c2eeae9433d807b067536179bfb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.300349  2067 tablet_replica.cc:333] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb: stopping tablet replica
I20260709 12:22:38.300925  2067 raft_consensus.cc:2243] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:38.301967  2067 raft_consensus.cc:2272] T 3c011ea4eb8c4263918f63ca65871907 P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.304083  2067 tablet_replica.cc:333] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb: stopping tablet replica
I20260709 12:22:38.304701  2067 raft_consensus.cc:2243] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.305299  2067 raft_consensus.cc:2272] T d159b73e1fed4f1eab8d8caf495fc751 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.308113  2067 tablet_replica.cc:333] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb: stopping tablet replica
I20260709 12:22:38.308723  2067 raft_consensus.cc:2243] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:22:38.309707  2067 raft_consensus.cc:2272] T 7effe2c171054fd0a4fdbd5216c88e03 P c2759c2eeae9433d807b067536179bfb [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.312422  2067 tablet_replica.cc:333] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb: stopping tablet replica
I20260709 12:22:38.313058  2067 raft_consensus.cc:2243] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.313527  2067 raft_consensus.cc:2272] T d3f2033c07ca45a183a62bac15cefbdc P c2759c2eeae9433d807b067536179bfb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.316229  2067 tablet_replica.cc:333] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb: stopping tablet replica
I20260709 12:22:38.316974  2067 raft_consensus.cc:2243] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:38.318384  2067 raft_consensus.cc:2272] T f7b8bf01edfc401c8c9269e27f1f48e1 P c2759c2eeae9433d807b067536179bfb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.320928  2067 tablet_replica.cc:333] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb: stopping tablet replica
I20260709 12:22:38.321455  2067 raft_consensus.cc:2243] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.322070  2067 raft_consensus.cc:2272] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P c2759c2eeae9433d807b067536179bfb [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:38.456207  2317 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37261: connect: Connection refused (error 111)
I20260709 12:22:38.458886  2067 tablet_server.cc:196] TabletServer@127.2.4.193:0 shutdown complete.
W20260709 12:22:38.461932  2317 consensus_peers.cc:597] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 -> Peer c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261): Couldn't send request to peer c2759c2eeae9433d807b067536179bfb. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37261: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:38.487401  2067 tablet_server.cc:179] TabletServer@127.2.4.194:0 shutting down...
W20260709 12:22:38.522013  2316 consensus_peers.cc:597] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 -> Peer b642f379bf7c4533aba8f266d7d0fe53 (127.2.4.194:44805): Couldn't send request to peer b642f379bf7c4533aba8f266d7d0fe53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:38.529811  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:38.530915  2067 tablet_replica.cc:333] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53: stopping tablet replica
I20260709 12:22:38.531778  2067 raft_consensus.cc:2243] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.532341  2067 raft_consensus.cc:2272] T 0b055ff9d3d7458faaaef52610877acf P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.534431  2067 tablet_replica.cc:333] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53: stopping tablet replica
I20260709 12:22:38.535182  2067 raft_consensus.cc:2243] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:22:38.536149  2067 raft_consensus.cc:2272] T 1f984985863e44299701c48dba1f4315 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.538137  2067 tablet_replica.cc:333] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53: stopping tablet replica
I20260709 12:22:38.538625  2067 raft_consensus.cc:2243] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:38.539386  2067 raft_consensus.cc:2272] T d159b73e1fed4f1eab8d8caf495fc751 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.541033  2067 tablet_replica.cc:333] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53: stopping tablet replica
I20260709 12:22:38.541491  2067 raft_consensus.cc:2243] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.541954  2067 raft_consensus.cc:2272] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P b642f379bf7c4533aba8f266d7d0fe53 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.543741  2067 tablet_replica.cc:333] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53: stopping tablet replica
I20260709 12:22:38.544236  2067 raft_consensus.cc:2243] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.544651  2067 raft_consensus.cc:2272] T 7effe2c171054fd0a4fdbd5216c88e03 P b642f379bf7c4533aba8f266d7d0fe53 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.546638  2067 tablet_replica.cc:333] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53: stopping tablet replica
I20260709 12:22:38.547122  2067 raft_consensus.cc:2243] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:38.547910  2067 raft_consensus.cc:2272] T d3f2033c07ca45a183a62bac15cefbdc P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.550127  2067 tablet_replica.cc:333] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53: stopping tablet replica
I20260709 12:22:38.550655  2067 raft_consensus.cc:2243] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.551077  2067 raft_consensus.cc:2272] T 91323316a13b463db86243d302b1fbe5 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.553433  2067 tablet_replica.cc:333] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53: stopping tablet replica
I20260709 12:22:38.554013  2067 raft_consensus.cc:2243] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.554577  2067 raft_consensus.cc:2272] T f7b8bf01edfc401c8c9269e27f1f48e1 P b642f379bf7c4533aba8f266d7d0fe53 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.556977  2067 tablet_replica.cc:333] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53: stopping tablet replica
I20260709 12:22:38.557662  2067 raft_consensus.cc:2243] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:38.558230  2317 consensus_peers.cc:597] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 -> Peer c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261): Couldn't send request to peer c2759c2eeae9433d807b067536179bfb. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37261: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:38.558279  2067 raft_consensus.cc:2272] T 3c011ea4eb8c4263918f63ca65871907 P b642f379bf7c4533aba8f266d7d0fe53 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.592247  2067 tablet_server.cc:196] TabletServer@127.2.4.194:0 shutdown complete.
I20260709 12:22:38.612936  2067 tablet_server.cc:179] TabletServer@127.2.4.195:0 shutting down...
W20260709 12:22:38.643278  2317 consensus_peers.cc:597] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 -> Peer c2759c2eeae9433d807b067536179bfb (127.2.4.193:37261): Couldn't send request to peer c2759c2eeae9433d807b067536179bfb. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37261: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:38.654379  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:38.655380  2067 tablet_replica.cc:333] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1: stopping tablet replica
I20260709 12:22:38.656144  2067 raft_consensus.cc:2243] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:38.657200  2067 raft_consensus.cc:2272] T 0b055ff9d3d7458faaaef52610877acf P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.659364  2067 tablet_replica.cc:333] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1: stopping tablet replica
I20260709 12:22:38.659961  2067 raft_consensus.cc:2243] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:22:38.660881  2067 raft_consensus.cc:2272] T 1b92f57dc0754ad8b04b6fe89d7a9fe4 P 91e3a0484e714200a61187a599eca1a1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.663055  2067 tablet_replica.cc:333] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1: stopping tablet replica
I20260709 12:22:38.663650  2067 raft_consensus.cc:2243] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.664135  2067 raft_consensus.cc:2272] T f7b8bf01edfc401c8c9269e27f1f48e1 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.666177  2067 tablet_replica.cc:333] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1: stopping tablet replica
I20260709 12:22:38.666802  2067 raft_consensus.cc:2243] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.667272  2067 raft_consensus.cc:2272] T 3c011ea4eb8c4263918f63ca65871907 P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.669154  2067 tablet_replica.cc:333] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1: stopping tablet replica
I20260709 12:22:38.669853  2067 raft_consensus.cc:2243] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.670365  2067 raft_consensus.cc:2272] T d3f2033c07ca45a183a62bac15cefbdc P 91e3a0484e714200a61187a599eca1a1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.672308  2067 tablet_replica.cc:333] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1: stopping tablet replica
I20260709 12:22:38.673015  2067 raft_consensus.cc:2243] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.673478  2067 raft_consensus.cc:2272] T 7effe2c171054fd0a4fdbd5216c88e03 P 91e3a0484e714200a61187a599eca1a1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.675371  2067 tablet_replica.cc:333] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1: stopping tablet replica
I20260709 12:22:38.675840  2067 raft_consensus.cc:2243] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.676224  2067 raft_consensus.cc:2272] T 1f984985863e44299701c48dba1f4315 P 91e3a0484e714200a61187a599eca1a1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.678059  2067 tablet_replica.cc:333] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1: stopping tablet replica
I20260709 12:22:38.678500  2067 raft_consensus.cc:2243] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:38.679309  2067 raft_consensus.cc:2272] T 91323316a13b463db86243d302b1fbe5 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.681027  2067 tablet_replica.cc:333] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1: stopping tablet replica
I20260709 12:22:38.681448  2067 raft_consensus.cc:2243] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:38.681908  2067 raft_consensus.cc:2272] T d159b73e1fed4f1eab8d8caf495fc751 P 91e3a0484e714200a61187a599eca1a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.714330  2067 tablet_server.cc:196] TabletServer@127.2.4.195:0 shutdown complete.
I20260709 12:22:38.738665  2067 master.cc:562] Master@127.2.4.254:32837 shutting down...
I20260709 12:22:38.970160  2067 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:38.970934  2067 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:38.971251  2067 tablet_replica.cc:333] T 00000000000000000000000000000000 P 371363936f9f4bb88d016afa314471ca: stopping tablet replica
I20260709 12:22:38.995280  2067 master.cc:584] Master@127.2.4.254:32837 shutdown complete.
91e3a0484e714200a61187a599eca1a1  1
b642f379bf7c4533aba8f266d7d0fe53  4
c2759c2eeae9433d807b067536179bfb  4
91e3a0484e714200a61187a599eca1a1  3
b642f379bf7c4533aba8f266d7d0fe53  3
c2759c2eeae9433d807b067536179bfb  3
91e3a0484e714200a61187a599eca1a1  1
b642f379bf7c4533aba8f266d7d0fe53  8
c2759c2eeae9433d807b067536179bfb  0
91e3a0484e714200a61187a599eca1a1  3
b642f379bf7c4533aba8f266d7d0fe53  3
c2759c2eeae9433d807b067536179bfb  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (31751 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:22:39.051055  2067 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.4.254:38309
I20260709 12:22:39.052488  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:39.059429  2598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:39.061098  2599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:39.061853  2601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:39.063463  2067 server_base.cc:1061] running on GCE node
I20260709 12:22:39.064842  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:39.065057  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:39.065188  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599759065175 us; error 0 us; skew 500 ppm
I20260709 12:22:39.065881  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:39.069049  2067 webserver.cc:533] Webserver started at http://127.2.4.254:39891/ using document root <none> and password file <none>
I20260709 12:22:39.069558  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:39.069851  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:39.070205  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:39.071583  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/master-0-root/instance:
uuid: "7cb8e5219c7f47ad849710af4312160e"
format_stamp: "Formatted at 2026-07-09 12:22:39 on dist-test-slave-4qvp"
I20260709 12:22:39.076946  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 12:22:39.081534  2606 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:39.082849  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:22:39.083189  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/master-0-root
uuid: "7cb8e5219c7f47ad849710af4312160e"
format_stamp: "Formatted at 2026-07-09 12:22:39 on dist-test-slave-4qvp"
I20260709 12:22:39.083478  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:39.113583  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:39.115001  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:39.161535  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.254:38309
I20260709 12:22:39.161813  2658 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.254:38309 every 8 connection(s)
I20260709 12:22:39.166326  2659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.177762  2659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e: Bootstrap starting.
I20260709 12:22:39.183288  2659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.188778  2659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e: No bootstrap required, opened a new log
I20260709 12:22:39.191533  2659 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb8e5219c7f47ad849710af4312160e" member_type: VOTER }
I20260709 12:22:39.192056  2659 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.192252  2659 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cb8e5219c7f47ad849710af4312160e, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.192884  2659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb8e5219c7f47ad849710af4312160e" member_type: VOTER }
I20260709 12:22:39.193356  2659 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:39.193554  2659 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:39.193832  2659 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.198555  2659 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb8e5219c7f47ad849710af4312160e" member_type: VOTER }
I20260709 12:22:39.199118  2659 leader_election.cc:304] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cb8e5219c7f47ad849710af4312160e; no voters: 
I20260709 12:22:39.200482  2659 leader_election.cc:290] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:39.201033  2662 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:39.202670  2662 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [term 1 LEADER]: Becoming Leader. State: Replica: 7cb8e5219c7f47ad849710af4312160e, State: Running, Role: LEADER
I20260709 12:22:39.203467  2662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb8e5219c7f47ad849710af4312160e" member_type: VOTER }
I20260709 12:22:39.203827  2659 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:39.209846  2664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cb8e5219c7f47ad849710af4312160e. Latest consensus state: current_term: 1 leader_uuid: "7cb8e5219c7f47ad849710af4312160e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb8e5219c7f47ad849710af4312160e" member_type: VOTER } }
I20260709 12:22:39.210494  2664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:39.210997  2663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cb8e5219c7f47ad849710af4312160e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb8e5219c7f47ad849710af4312160e" member_type: VOTER } }
I20260709 12:22:39.211728  2663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:39.213169  2670 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:39.222412  2670 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:39.223611  2067 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:39.233518  2670 catalog_manager.cc:1383] Generated new cluster ID: d26bbe327c5e4143a7de6b369a89ea86
I20260709 12:22:39.233930  2670 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:39.252718  2670 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:39.254336  2670 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:39.266835  2670 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e: Generated new TSK 0
I20260709 12:22:39.267640  2670 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:39.290941  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:39.298346  2680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:39.299805  2681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:39.303975  2067 server_base.cc:1061] running on GCE node
W20260709 12:22:39.304047  2683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:39.305454  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:39.305756  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:39.305979  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599759305953 us; error 0 us; skew 500 ppm
I20260709 12:22:39.306876  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:39.310286  2067 webserver.cc:533] Webserver started at http://127.2.4.193:40255/ using document root <none> and password file <none>
I20260709 12:22:39.310895  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:39.311066  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:39.311407  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:39.312602  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/ts-0-root/instance:
uuid: "51c0e412fd9c4909bb014ae370587bd2"
format_stamp: "Formatted at 2026-07-09 12:22:39 on dist-test-slave-4qvp"
I20260709 12:22:39.318172  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:22:39.322333  2688 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:39.323460  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:22:39.323765  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/ts-0-root
uuid: "51c0e412fd9c4909bb014ae370587bd2"
format_stamp: "Formatted at 2026-07-09 12:22:39 on dist-test-slave-4qvp"
I20260709 12:22:39.324087  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:39.347604  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:39.349069  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:39.350852  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:39.354355  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:39.354575  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:39.354940  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:39.355115  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:39.407560  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.193:35425
I20260709 12:22:39.407704  2750 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.193:35425 every 8 connection(s)
I20260709 12:22:39.413758  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:39.422904  2755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:39.424551  2756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:39.427098  2758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:39.428843  2067 server_base.cc:1061] running on GCE node
I20260709 12:22:39.430855  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:39.431119  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:39.431259  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599759431246 us; error 0 us; skew 500 ppm
I20260709 12:22:39.431769  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:39.435330  2067 webserver.cc:533] Webserver started at http://127.2.4.194:38193/ using document root <none> and password file <none>
I20260709 12:22:39.436100  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:39.436384  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:39.436741  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:39.437862  2751 heartbeater.cc:344] Connected to a master server at 127.2.4.254:38309
I20260709 12:22:39.438333  2751 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:39.438400  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/ts-1-root/instance:
uuid: "697f4bdcd0944ad58ce468159b075fe8"
format_stamp: "Formatted at 2026-07-09 12:22:39 on dist-test-slave-4qvp"
I20260709 12:22:39.439208  2751 heartbeater.cc:507] Master 127.2.4.254:38309 requested a full tablet report, sending...
I20260709 12:22:39.441910  2624 ts_manager.cc:194] Registered new tserver with Master: 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425)
I20260709 12:22:39.444096  2624 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49642
I20260709 12:22:39.444864  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:22:39.449236  2763 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:39.450351  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:22:39.450770  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/ts-1-root
uuid: "697f4bdcd0944ad58ce468159b075fe8"
format_stamp: "Formatted at 2026-07-09 12:22:39 on dist-test-slave-4qvp"
I20260709 12:22:39.451122  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:39.463158  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:39.464805  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:39.466849  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:39.469821  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:39.470090  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:39.470364  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:39.470551  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:39.521947  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.194:46485
I20260709 12:22:39.522118  2825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.194:46485 every 8 connection(s)
I20260709 12:22:39.528416  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:39.537545  2829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:39.538676  2830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:39.544797  2826 heartbeater.cc:344] Connected to a master server at 127.2.4.254:38309
W20260709 12:22:39.545223  2832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:39.545430  2826 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:39.545451  2067 server_base.cc:1061] running on GCE node
I20260709 12:22:39.546940  2826 heartbeater.cc:507] Master 127.2.4.254:38309 requested a full tablet report, sending...
I20260709 12:22:39.547019  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:39.547431  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:39.547680  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599759547658 us; error 0 us; skew 500 ppm
I20260709 12:22:39.548538  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:39.549984  2624 ts_manager.cc:194] Registered new tserver with Master: 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485)
I20260709 12:22:39.551746  2624 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49648
I20260709 12:22:39.552013  2067 webserver.cc:533] Webserver started at http://127.2.4.195:45549/ using document root <none> and password file <none>
I20260709 12:22:39.552990  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:39.553288  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:39.553784  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:39.555310  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/ts-2-root/instance:
uuid: "9bff7abdc5314c13a9534adf58b5002e"
format_stamp: "Formatted at 2026-07-09 12:22:39 on dist-test-slave-4qvp"
I20260709 12:22:39.561007  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.003s
I20260709 12:22:39.565433  2837 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:39.566529  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 12:22:39.566979  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/ts-2-root
uuid: "9bff7abdc5314c13a9534adf58b5002e"
format_stamp: "Formatted at 2026-07-09 12:22:39 on dist-test-slave-4qvp"
I20260709 12:22:39.567416  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599727094521-2067-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:39.587137  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:39.588613  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:39.590126  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:39.592760  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:39.593010  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:39.593284  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:39.593469  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:39.644675  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.195:46271
I20260709 12:22:39.644794  2900 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.195:46271 every 8 connection(s)
I20260709 12:22:39.662140  2901 heartbeater.cc:344] Connected to a master server at 127.2.4.254:38309
I20260709 12:22:39.662596  2901 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:39.663421  2901 heartbeater.cc:507] Master 127.2.4.254:38309 requested a full tablet report, sending...
I20260709 12:22:39.666188  2624 ts_manager.cc:194] Registered new tserver with Master: 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:39.666863  2067 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018078936s
I20260709 12:22:39.668083  2624 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51568
I20260709 12:22:39.698060  2624 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51582:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:39.701125  2624 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:39.775102  2865 tablet_service.cc:1511] Processing CreateTablet for tablet c45c2ae497034799b8ddab73de61c8dc (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:39.775928  2864 tablet_service.cc:1511] Processing CreateTablet for tablet 4473c73827fc420a9222f7b3947ba58b (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:39.776706  2865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c45c2ae497034799b8ddab73de61c8dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.780102  2863 tablet_service.cc:1511] Processing CreateTablet for tablet 4ca5ce11aa9a430a8a06e20205ba2b1b (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:39.780277  2866 tablet_service.cc:1511] Processing CreateTablet for tablet 0bb8c0788a7d4d0c8e19fbd44da9fbbc (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:39.777529  2864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4473c73827fc420a9222f7b3947ba58b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.782282  2866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bb8c0788a7d4d0c8e19fbd44da9fbbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.787572  2862 tablet_service.cc:1511] Processing CreateTablet for tablet aae17ace8aad40fe8ec2b304a4a97cb5 (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:39.789142  2862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aae17ace8aad40fe8ec2b304a4a97cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.790014  2863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ca5ce11aa9a430a8a06e20205ba2b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.794250  2861 tablet_service.cc:1511] Processing CreateTablet for tablet 553040872de34092969954cb33c7de3b (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:39.795782  2861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 553040872de34092969954cb33c7de3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.798089  2860 tablet_service.cc:1511] Processing CreateTablet for tablet 53e4b64c0177467887dee4d5aa6e3a44 (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:39.799837  2860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53e4b64c0177467887dee4d5aa6e3a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.810850  2716 tablet_service.cc:1511] Processing CreateTablet for tablet 0bb8c0788a7d4d0c8e19fbd44da9fbbc (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:39.812479  2716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bb8c0788a7d4d0c8e19fbd44da9fbbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.813868  2713 tablet_service.cc:1511] Processing CreateTablet for tablet 4ca5ce11aa9a430a8a06e20205ba2b1b (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:39.815142  2715 tablet_service.cc:1511] Processing CreateTablet for tablet c45c2ae497034799b8ddab73de61c8dc (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:39.817415  2712 tablet_service.cc:1511] Processing CreateTablet for tablet aae17ace8aad40fe8ec2b304a4a97cb5 (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:39.817674  2714 tablet_service.cc:1511] Processing CreateTablet for tablet 4473c73827fc420a9222f7b3947ba58b (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:39.813868  2859 tablet_service.cc:1511] Processing CreateTablet for tablet 64d4799f52d9401c8c86306eeb1a3623 (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:39.819151  2713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ca5ce11aa9a430a8a06e20205ba2b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.820361  2711 tablet_service.cc:1511] Processing CreateTablet for tablet 553040872de34092969954cb33c7de3b (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:39.821859  2711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 553040872de34092969954cb33c7de3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.822559  2715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c45c2ae497034799b8ddab73de61c8dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.822151  2858 tablet_service.cc:1511] Processing CreateTablet for tablet ad190a87809b4ea7ac8d72146be24a61 (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:39.822870  2708 tablet_service.cc:1511] Processing CreateTablet for tablet ad190a87809b4ea7ac8d72146be24a61 (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:39.824352  2708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad190a87809b4ea7ac8d72146be24a61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.825841  2714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4473c73827fc420a9222f7b3947ba58b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.827148  2712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aae17ace8aad40fe8ec2b304a4a97cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.826776  2710 tablet_service.cc:1511] Processing CreateTablet for tablet 53e4b64c0177467887dee4d5aa6e3a44 (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:39.828244  2710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53e4b64c0177467887dee4d5aa6e3a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.828146  2707 tablet_service.cc:1511] Processing CreateTablet for tablet b7b0e8a1a9224999bd5bcfc28b4c4bb8 (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:39.830667  2707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7b0e8a1a9224999bd5bcfc28b4c4bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.831266  2709 tablet_service.cc:1511] Processing CreateTablet for tablet 64d4799f52d9401c8c86306eeb1a3623 (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:39.833076  2709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64d4799f52d9401c8c86306eeb1a3623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.840634  2857 tablet_service.cc:1511] Processing CreateTablet for tablet b7b0e8a1a9224999bd5bcfc28b4c4bb8 (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:39.866533  2858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad190a87809b4ea7ac8d72146be24a61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.868453  2857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7b0e8a1a9224999bd5bcfc28b4c4bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.869920  2859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64d4799f52d9401c8c86306eeb1a3623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.895255  2791 tablet_service.cc:1511] Processing CreateTablet for tablet 0bb8c0788a7d4d0c8e19fbd44da9fbbc (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:39.896903  2791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bb8c0788a7d4d0c8e19fbd44da9fbbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.906962  2790 tablet_service.cc:1511] Processing CreateTablet for tablet c45c2ae497034799b8ddab73de61c8dc (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:39.908722  2790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c45c2ae497034799b8ddab73de61c8dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.915518  2789 tablet_service.cc:1511] Processing CreateTablet for tablet 4473c73827fc420a9222f7b3947ba58b (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:39.917692  2789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4473c73827fc420a9222f7b3947ba58b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.924402  2788 tablet_service.cc:1511] Processing CreateTablet for tablet 4ca5ce11aa9a430a8a06e20205ba2b1b (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:39.927224  2788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ca5ce11aa9a430a8a06e20205ba2b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.943836  2787 tablet_service.cc:1511] Processing CreateTablet for tablet aae17ace8aad40fe8ec2b304a4a97cb5 (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:39.944433  2785 tablet_service.cc:1511] Processing CreateTablet for tablet 53e4b64c0177467887dee4d5aa6e3a44 (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:39.947431  2786 tablet_service.cc:1511] Processing CreateTablet for tablet 553040872de34092969954cb33c7de3b (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:39.956351  2787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aae17ace8aad40fe8ec2b304a4a97cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.959030  2783 tablet_service.cc:1511] Processing CreateTablet for tablet ad190a87809b4ea7ac8d72146be24a61 (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:39.961077  2783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad190a87809b4ea7ac8d72146be24a61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.957074  2784 tablet_service.cc:1511] Processing CreateTablet for tablet 64d4799f52d9401c8c86306eeb1a3623 (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:39.966280  2782 tablet_service.cc:1511] Processing CreateTablet for tablet b7b0e8a1a9224999bd5bcfc28b4c4bb8 (DEFAULT_TABLE table=test-workload [id=80727df187534f88acab9242eb5b4a45]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:39.967442  2785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53e4b64c0177467887dee4d5aa6e3a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.968477  2782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7b0e8a1a9224999bd5bcfc28b4c4bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.969750  2784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64d4799f52d9401c8c86306eeb1a3623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:39.976758  2786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 553040872de34092969954cb33c7de3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:40.027933  2915 tablet_bootstrap.cc:492] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e: Bootstrap starting.
I20260709 12:22:40.092360  2916 tablet_bootstrap.cc:492] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2: Bootstrap starting.
I20260709 12:22:40.094486  2917 tablet_bootstrap.cc:492] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8: Bootstrap starting.
I20260709 12:22:40.099184  2915 tablet_bootstrap.cc:654] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.103688  2917 tablet_bootstrap.cc:654] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.120549  2916 tablet_bootstrap.cc:654] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.120558  2917 tablet_bootstrap.cc:492] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8: No bootstrap required, opened a new log
I20260709 12:22:40.121798  2917 ts_tablet_manager.cc:1403] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.005s
I20260709 12:22:40.122998  2915 tablet_bootstrap.cc:492] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e: No bootstrap required, opened a new log
I20260709 12:22:40.123554  2915 ts_tablet_manager.cc:1403] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e: Time spent bootstrapping tablet: real 0.096s	user 0.019s	sys 0.009s
I20260709 12:22:40.126533  2917 raft_consensus.cc:359] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.126739  2915 raft_consensus.cc:359] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:40.127319  2916 tablet_bootstrap.cc:492] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2: No bootstrap required, opened a new log
I20260709 12:22:40.127729  2915 raft_consensus.cc:385] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.127465  2917 raft_consensus.cc:385] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.128082  2916 ts_tablet_manager.cc:1403] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.000s
I20260709 12:22:40.128089  2915 raft_consensus.cc:740] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.128705  2917 raft_consensus.cc:740] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.129503  2915 consensus_queue.cc:260] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:40.129498  2917 consensus_queue.cc:260] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.131287  2916 raft_consensus.cc:359] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.132167  2916 raft_consensus.cc:385] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.132481  2916 raft_consensus.cc:740] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.133294  2915 ts_tablet_manager.cc:1434] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.003s
I20260709 12:22:40.133508  2826 heartbeater.cc:499] Master 127.2.4.254:38309 was elected leader, sending a full tablet report...
I20260709 12:22:40.133742  2917 ts_tablet_manager.cc:1434] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 12:22:40.133512  2916 consensus_queue.cc:260] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.134810  2917 tablet_bootstrap.cc:492] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8: Bootstrap starting.
I20260709 12:22:40.136591  2751 heartbeater.cc:499] Master 127.2.4.254:38309 was elected leader, sending a full tablet report...
I20260709 12:22:40.136997  2916 ts_tablet_manager.cc:1434] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:22:40.138195  2916 tablet_bootstrap.cc:492] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2: Bootstrap starting.
I20260709 12:22:40.142167  2917 tablet_bootstrap.cc:654] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.146335  2916 tablet_bootstrap.cc:654] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.149415  2917 tablet_bootstrap.cc:492] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8: No bootstrap required, opened a new log
I20260709 12:22:40.150856  2917 ts_tablet_manager.cc:1403] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.004s
I20260709 12:22:40.151876  2901 heartbeater.cc:499] Master 127.2.4.254:38309 was elected leader, sending a full tablet report...
I20260709 12:22:40.153012  2916 tablet_bootstrap.cc:492] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2: No bootstrap required, opened a new log
I20260709 12:22:40.153702  2916 ts_tablet_manager.cc:1403] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.009s
I20260709 12:22:40.154578  2915 tablet_bootstrap.cc:492] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e: Bootstrap starting.
I20260709 12:22:40.158120  2917 raft_consensus.cc:359] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.158962  2917 raft_consensus.cc:385] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.159291  2917 raft_consensus.cc:740] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.160293  2917 consensus_queue.cc:260] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.160802  2916 raft_consensus.cc:359] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.161850  2916 raft_consensus.cc:385] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.162199  2916 raft_consensus.cc:740] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.162773  2917 ts_tablet_manager.cc:1434] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 12:22:40.163115  2916 consensus_queue.cc:260] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.164234  2917 tablet_bootstrap.cc:492] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8: Bootstrap starting.
I20260709 12:22:40.165656  2916 ts_tablet_manager.cc:1434] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 12:22:40.166834  2916 tablet_bootstrap.cc:492] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2: Bootstrap starting.
I20260709 12:22:40.167675  2915 tablet_bootstrap.cc:654] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.172950  2917 tablet_bootstrap.cc:654] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.174789  2916 tablet_bootstrap.cc:654] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.181173  2915 tablet_bootstrap.cc:492] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e: No bootstrap required, opened a new log
I20260709 12:22:40.186530  2915 ts_tablet_manager.cc:1403] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e: Time spent bootstrapping tablet: real 0.032s	user 0.018s	sys 0.003s
I20260709 12:22:40.186605  2917 tablet_bootstrap.cc:492] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8: No bootstrap required, opened a new log
I20260709 12:22:40.187222  2917 ts_tablet_manager.cc:1403] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.008s
I20260709 12:22:40.187297  2916 tablet_bootstrap.cc:492] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2: No bootstrap required, opened a new log
I20260709 12:22:40.188506  2916 ts_tablet_manager.cc:1403] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.013s
I20260709 12:22:40.190750  2917 raft_consensus.cc:359] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.191663  2917 raft_consensus.cc:385] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.191360  2915 raft_consensus.cc:359] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.191979  2917 raft_consensus.cc:740] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.192065  2915 raft_consensus.cc:385] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.192454  2915 raft_consensus.cc:740] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.192566  2916 raft_consensus.cc:359] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.193097  2917 consensus_queue.cc:260] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.193523  2916 raft_consensus.cc:385] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.194036  2916 raft_consensus.cc:740] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.194411  2915 consensus_queue.cc:260] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.194937  2916 consensus_queue.cc:260] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.195914  2917 ts_tablet_manager.cc:1434] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:22:40.196928  2917 tablet_bootstrap.cc:492] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8: Bootstrap starting.
I20260709 12:22:40.198180  2916 ts_tablet_manager.cc:1434] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:22:40.199530  2916 tablet_bootstrap.cc:492] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2: Bootstrap starting.
I20260709 12:22:40.200417  2915 ts_tablet_manager.cc:1434] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 12:22:40.201589  2915 tablet_bootstrap.cc:492] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e: Bootstrap starting.
I20260709 12:22:40.203811  2917 tablet_bootstrap.cc:654] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.207435  2916 tablet_bootstrap.cc:654] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.209496  2915 tablet_bootstrap.cc:654] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:40.216089  2672 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:40.216430  2917 tablet_bootstrap.cc:492] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8: No bootstrap required, opened a new log
I20260709 12:22:40.217036  2917 ts_tablet_manager.cc:1403] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 12:22:40.221388  2917 raft_consensus.cc:359] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.222479  2917 raft_consensus.cc:385] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.223167  2917 raft_consensus.cc:740] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.224030  2917 consensus_queue.cc:260] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.226892  2917 ts_tablet_manager.cc:1434] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.004s
I20260709 12:22:40.230511  2915 tablet_bootstrap.cc:492] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e: No bootstrap required, opened a new log
I20260709 12:22:40.231062  2915 ts_tablet_manager.cc:1403] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.008s
I20260709 12:22:40.232663  2917 tablet_bootstrap.cc:492] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8: Bootstrap starting.
I20260709 12:22:40.234788  2915 raft_consensus.cc:359] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.235577  2915 raft_consensus.cc:385] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.235909  2915 raft_consensus.cc:740] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.236013  2923 raft_consensus.cc:493] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.236665  2923 raft_consensus.cc:515] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.237193  2915 consensus_queue.cc:260] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.240471  2915 ts_tablet_manager.cc:1434] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:22:40.241257  2917 tablet_bootstrap.cc:654] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.241742  2915 tablet_bootstrap.cc:492] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e: Bootstrap starting.
I20260709 12:22:40.243170  2923 leader_election.cc:290] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.251209  2915 tablet_bootstrap.cc:654] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.254679  2917 tablet_bootstrap.cc:492] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8: No bootstrap required, opened a new log
I20260709 12:22:40.255975  2917 ts_tablet_manager.cc:1403] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.011s
I20260709 12:22:40.260231  2917 raft_consensus.cc:359] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.261279  2917 raft_consensus.cc:385] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.261843  2917 raft_consensus.cc:740] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.263114  2917 consensus_queue.cc:260] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.266887  2917 ts_tablet_manager.cc:1434] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:22:40.273020  2917 tablet_bootstrap.cc:492] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8: Bootstrap starting.
I20260709 12:22:40.274794  2915 tablet_bootstrap.cc:492] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e: No bootstrap required, opened a new log
I20260709 12:22:40.275362  2915 ts_tablet_manager.cc:1403] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e: Time spent bootstrapping tablet: real 0.034s	user 0.023s	sys 0.008s
I20260709 12:22:40.278681  2915 raft_consensus.cc:359] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.279443  2915 raft_consensus.cc:385] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.279757  2915 raft_consensus.cc:740] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.280220  2917 tablet_bootstrap.cc:654] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.280767  2915 consensus_queue.cc:260] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.283843  2915 ts_tablet_manager.cc:1434] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.003s
I20260709 12:22:40.285282  2915 tablet_bootstrap.cc:492] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e: Bootstrap starting.
I20260709 12:22:40.296550  2915 tablet_bootstrap.cc:654] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.297554  2917 tablet_bootstrap.cc:492] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8: No bootstrap required, opened a new log
I20260709 12:22:40.298184  2917 ts_tablet_manager.cc:1403] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.011s
I20260709 12:22:40.302310  2917 raft_consensus.cc:359] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.303810  2917 raft_consensus.cc:385] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.304332  2917 raft_consensus.cc:740] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.305661  2917 consensus_queue.cc:260] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.308629  2915 tablet_bootstrap.cc:492] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e: No bootstrap required, opened a new log
I20260709 12:22:40.309226  2915 ts_tablet_manager.cc:1403] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.008s
I20260709 12:22:40.311131  2917 ts_tablet_manager.cc:1434] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:22:40.311796  2923 raft_consensus.cc:493] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.312566  2917 tablet_bootstrap.cc:492] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8: Bootstrap starting.
I20260709 12:22:40.312458  2923 raft_consensus.cc:515] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.312493  2915 raft_consensus.cc:359] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.313576  2915 raft_consensus.cc:385] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.313962  2915 raft_consensus.cc:740] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.314981  2915 consensus_queue.cc:260] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.317950  2915 ts_tablet_manager.cc:1434] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
I20260709 12:22:40.318670  2923 leader_election.cc:290] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.319022  2915 tablet_bootstrap.cc:492] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e: Bootstrap starting.
I20260709 12:22:40.320479  2917 tablet_bootstrap.cc:654] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.322744  2916 tablet_bootstrap.cc:492] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2: No bootstrap required, opened a new log
I20260709 12:22:40.323393  2916 ts_tablet_manager.cc:1403] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2: Time spent bootstrapping tablet: real 0.124s	user 0.046s	sys 0.037s
I20260709 12:22:40.326854  2916 raft_consensus.cc:359] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.327673  2916 raft_consensus.cc:385] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.327859  2915 tablet_bootstrap.cc:654] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.328034  2916 raft_consensus.cc:740] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.329325  2916 consensus_queue.cc:260] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.330308  2923 raft_consensus.cc:493] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.330814  2923 raft_consensus.cc:515] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.332917  2922 raft_consensus.cc:493] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.333521  2922 raft_consensus.cc:515] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.336136  2923 leader_election.cc:290] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.338626  2916 ts_tablet_manager.cc:1434] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:22:40.339849  2916 tablet_bootstrap.cc:492] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2: Bootstrap starting.
I20260709 12:22:40.344355  2915 tablet_bootstrap.cc:492] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e: No bootstrap required, opened a new log
I20260709 12:22:40.345022  2915 ts_tablet_manager.cc:1403] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.013s
I20260709 12:22:40.348055  2916 tablet_bootstrap.cc:654] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.348928  2922 leader_election.cc:290] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.348973  2915 raft_consensus.cc:359] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.350078  2915 raft_consensus.cc:385] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.350473  2915 raft_consensus.cc:740] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.351753  2915 consensus_queue.cc:260] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.355279  2915 ts_tablet_manager.cc:1434] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.005s
I20260709 12:22:40.356745  2915 tablet_bootstrap.cc:492] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e: Bootstrap starting.
I20260709 12:22:40.366010  2915 tablet_bootstrap.cc:654] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.333524  2917 tablet_bootstrap.cc:492] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8: No bootstrap required, opened a new log
I20260709 12:22:40.368465  2917 ts_tablet_manager.cc:1403] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8: Time spent bootstrapping tablet: real 0.056s	user 0.013s	sys 0.007s
I20260709 12:22:40.370893  2922 raft_consensus.cc:493] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.371630  2922 raft_consensus.cc:515] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.375830  2922 leader_election.cc:290] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.379766  2915 tablet_bootstrap.cc:492] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e: No bootstrap required, opened a new log
I20260709 12:22:40.380470  2915 ts_tablet_manager.cc:1403] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e: Time spent bootstrapping tablet: real 0.024s	user 0.004s	sys 0.016s
I20260709 12:22:40.381973  2916 tablet_bootstrap.cc:492] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2: No bootstrap required, opened a new log
I20260709 12:22:40.382503  2916 ts_tablet_manager.cc:1403] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2: Time spent bootstrapping tablet: real 0.043s	user 0.019s	sys 0.022s
I20260709 12:22:40.384373  2915 raft_consensus.cc:359] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.385387  2915 raft_consensus.cc:385] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.385195  2916 raft_consensus.cc:359] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.386096  2915 raft_consensus.cc:740] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.386564  2916 raft_consensus.cc:385] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.386945  2916 raft_consensus.cc:740] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.387158  2915 consensus_queue.cc:260] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.387964  2916 consensus_queue.cc:260] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.390894  2916 ts_tablet_manager.cc:1434] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:22:40.392113  2916 tablet_bootstrap.cc:492] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2: Bootstrap starting.
I20260709 12:22:40.388182  2917 raft_consensus.cc:359] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.392664  2917 raft_consensus.cc:385] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.393030  2917 raft_consensus.cc:740] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.394603  2915 ts_tablet_manager.cc:1434] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.003s
I20260709 12:22:40.396054  2915 tablet_bootstrap.cc:492] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e: Bootstrap starting.
I20260709 12:22:40.398926  2916 tablet_bootstrap.cc:654] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.394688  2917 consensus_queue.cc:260] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.403219  2915 tablet_bootstrap.cc:654] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.410817  2916 tablet_bootstrap.cc:492] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2: No bootstrap required, opened a new log
I20260709 12:22:40.411374  2916 ts_tablet_manager.cc:1403] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 12:22:40.411541  2917 ts_tablet_manager.cc:1434] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.000s
I20260709 12:22:40.412647  2917 tablet_bootstrap.cc:492] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8: Bootstrap starting.
I20260709 12:22:40.414603  2916 raft_consensus.cc:359] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.415594  2916 raft_consensus.cc:385] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.415912  2916 raft_consensus.cc:740] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.416854  2916 consensus_queue.cc:260] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.418569  2915 tablet_bootstrap.cc:492] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e: No bootstrap required, opened a new log
I20260709 12:22:40.418917  2916 ts_tablet_manager.cc:1434] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:22:40.419126  2915 ts_tablet_manager.cc:1403] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.001s
I20260709 12:22:40.420009  2916 tablet_bootstrap.cc:492] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2: Bootstrap starting.
I20260709 12:22:40.422228  2915 raft_consensus.cc:359] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.422996  2915 raft_consensus.cc:385] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.423283  2915 raft_consensus.cc:740] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.424260  2915 consensus_queue.cc:260] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.427098  2917 tablet_bootstrap.cc:654] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.427098  2916 tablet_bootstrap.cc:654] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.428781  2915 ts_tablet_manager.cc:1434] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:22:40.429888  2915 tablet_bootstrap.cc:492] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e: Bootstrap starting.
I20260709 12:22:40.436358  2915 tablet_bootstrap.cc:654] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.439479  2916 tablet_bootstrap.cc:492] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2: No bootstrap required, opened a new log
I20260709 12:22:40.440004  2916 ts_tablet_manager.cc:1403] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 12:22:40.440055  2876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0e8a1a9224999bd5bcfc28b4c4bb8" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e" is_pre_election: true
I20260709 12:22:40.443153  2916 raft_consensus.cc:359] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.443766  2917 tablet_bootstrap.cc:492] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8: No bootstrap required, opened a new log
I20260709 12:22:40.443930  2916 raft_consensus.cc:385] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.444334  2916 raft_consensus.cc:740] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.444509  2917 ts_tablet_manager.cc:1403] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.004s
I20260709 12:22:40.445444  2916 consensus_queue.cc:260] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.447410  2917 raft_consensus.cc:359] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.447933  2916 ts_tablet_manager.cc:1434] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:22:40.448378  2917 raft_consensus.cc:385] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.448729  2917 raft_consensus.cc:740] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.449066  2916 tablet_bootstrap.cc:492] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2: Bootstrap starting.
I20260709 12:22:40.449707  2917 consensus_queue.cc:260] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.451804  2917 ts_tablet_manager.cc:1434] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:22:40.453761  2876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad190a87809b4ea7ac8d72146be24a61" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e" is_pre_election: true
I20260709 12:22:40.454833  2876 raft_consensus.cc:2468] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 0.
I20260709 12:22:40.455272  2917 tablet_bootstrap.cc:492] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8: Bootstrap starting.
I20260709 12:22:40.455992  2916 tablet_bootstrap.cc:654] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.461840  2876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e4b64c0177467887dee4d5aa6e3a44" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e" is_pre_election: true
I20260709 12:22:40.463056  2917 tablet_bootstrap.cc:654] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.462914  2876 raft_consensus.cc:2468] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 0.
W20260709 12:22:40.464435  2690 leader_election.cc:343] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:40.466086  2801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0e8a1a9224999bd5bcfc28b4c4bb8" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8" is_pre_election: true
I20260709 12:22:40.467110  2801 raft_consensus.cc:2468] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 0.
I20260709 12:22:40.468096  2916 tablet_bootstrap.cc:492] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2: No bootstrap required, opened a new log
I20260709 12:22:40.469239  2923 raft_consensus.cc:493] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.469765  2726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad190a87809b4ea7ac8d72146be24a61" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2" is_pre_election: true
I20260709 12:22:40.470813  2726 raft_consensus.cc:2468] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 0.
I20260709 12:22:40.473533  2690 leader_election.cc:304] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:40.470098  2923 raft_consensus.cc:515] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.480705  2921 raft_consensus.cc:493] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.481902  2726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2" is_pre_election: true
I20260709 12:22:40.482597  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e" is_pre_election: true
I20260709 12:22:40.484561  2801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad190a87809b4ea7ac8d72146be24a61" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8" is_pre_election: true
I20260709 12:22:40.485738  2923 leader_election.cc:290] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.485905  2801 raft_consensus.cc:2468] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 0.
I20260709 12:22:40.486364  2923 raft_consensus.cc:2804] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:40.486907  2923 raft_consensus.cc:493] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.487239  2923 raft_consensus.cc:3060] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.481384  2921 raft_consensus.cc:515] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.489271  2801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e4b64c0177467887dee4d5aa6e3a44" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8" is_pre_election: true
I20260709 12:22:40.490267  2801 raft_consensus.cc:2468] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 0.
I20260709 12:22:40.492166  2921 leader_election.cc:290] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485)
I20260709 12:22:40.492756  2921 raft_consensus.cc:493] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.492712  2801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ca5ce11aa9a430a8a06e20205ba2b1b" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8" is_pre_election: true
I20260709 12:22:40.468696  2916 ts_tablet_manager.cc:1403] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 12:22:40.481462  2876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad190a87809b4ea7ac8d72146be24a61" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e" is_pre_election: true
I20260709 12:22:40.494323  2876 raft_consensus.cc:2468] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 0.
I20260709 12:22:40.493665  2801 raft_consensus.cc:2468] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 0.
I20260709 12:22:40.495385  2875 raft_consensus.cc:2468] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 0.
I20260709 12:22:40.493438  2921 raft_consensus.cc:515] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:40.496863  2916 raft_consensus.cc:359] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.497897  2916 raft_consensus.cc:385] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.498785  2921 leader_election.cc:290] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485)
I20260709 12:22:40.498219  2916 raft_consensus.cc:740] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.499339  2692 leader_election.cc:304] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 697f4bdcd0944ad58ce468159b075fe8; no voters: 9bff7abdc5314c13a9534adf58b5002e
I20260709 12:22:40.502573  2953 raft_consensus.cc:2804] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:40.503120  2953 raft_consensus.cc:493] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.503525  2953 raft_consensus.cc:3060] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.500387  2916 consensus_queue.cc:260] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.503337  2923 raft_consensus.cc:515] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.506357  2690 leader_election.cc:304] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:40.509181  2767 leader_election.cc:304] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 697f4bdcd0944ad58ce468159b075fe8; no voters: 
I20260709 12:22:40.510200  2692 leader_election.cc:304] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 697f4bdcd0944ad58ce468159b075fe8; no voters: 
I20260709 12:22:40.510247  2922 raft_consensus.cc:2804] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:40.512856  2922 raft_consensus.cc:493] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.513365  2922 raft_consensus.cc:3060] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.513984  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ca5ce11aa9a430a8a06e20205ba2b1b" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e" is_pre_election: true
I20260709 12:22:40.514951  2875 raft_consensus.cc:2468] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 0.
I20260709 12:22:40.529054  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad190a87809b4ea7ac8d72146be24a61" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:40.529958  2875 raft_consensus.cc:3060] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:40.510603  2767 leader_election.cc:343] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:40.523933  2958 raft_consensus.cc:2804] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:40.543948  2767 leader_election.cc:304] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 51c0e412fd9c4909bb014ae370587bd2
I20260709 12:22:40.546175  2957 raft_consensus.cc:493] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.546940  2957 raft_consensus.cc:515] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.550398  2959 raft_consensus.cc:2804] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:40.541344  2923 leader_election.cc:290] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 election: Requested vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.550956  2959 raft_consensus.cc:493] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.551395  2959 raft_consensus.cc:3060] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.551826  2875 raft_consensus.cc:2468] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 1.
I20260709 12:22:40.549561  2958 raft_consensus.cc:493] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.554239  2958 raft_consensus.cc:3060] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.555200  2916 ts_tablet_manager.cc:1434] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2: Time spent starting tablet: real 0.061s	user 0.009s	sys 0.006s
I20260709 12:22:40.556288  2916 tablet_bootstrap.cc:492] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2: Bootstrap starting.
I20260709 12:22:40.556692  2690 leader_election.cc:304] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:40.557829  2923 raft_consensus.cc:2804] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:40.559237  2915 tablet_bootstrap.cc:492] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e: No bootstrap required, opened a new log
I20260709 12:22:40.559758  2915 ts_tablet_manager.cc:1403] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e: Time spent bootstrapping tablet: real 0.130s	user 0.021s	sys 0.034s
I20260709 12:22:40.565228  2917 tablet_bootstrap.cc:492] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8: No bootstrap required, opened a new log
I20260709 12:22:40.565979  2917 ts_tablet_manager.cc:1403] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8: Time spent bootstrapping tablet: real 0.111s	user 0.028s	sys 0.022s
W20260709 12:22:40.568081  2827 tablet.cc:2406] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:22:40.573167  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e" is_pre_election: true
I20260709 12:22:40.573057  2915 raft_consensus.cc:359] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.574035  2915 raft_consensus.cc:385] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.574070  2875 raft_consensus.cc:2468] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 0.
I20260709 12:22:40.574561  2915 raft_consensus.cc:740] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.575691  2690 leader_election.cc:304] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:40.575551  2915 consensus_queue.cc:260] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
W20260709 12:22:40.578008  2767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:35425, user_credentials={real_user=slave}} blocked reactor thread for 67397.8us
I20260709 12:22:40.578107  2917 raft_consensus.cc:359] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.579147  2917 raft_consensus.cc:385] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.579489  2917 raft_consensus.cc:740] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.584496  2915 ts_tablet_manager.cc:1434] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.002s
I20260709 12:22:40.585765  2915 tablet_bootstrap.cc:492] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e: Bootstrap starting.
I20260709 12:22:40.569401  2957 leader_election.cc:290] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.582788  2917 consensus_queue.cc:260] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.576689  2923 raft_consensus.cc:697] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Becoming Leader. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:40.589033  2801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad190a87809b4ea7ac8d72146be24a61" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:40.591496  2923 consensus_queue.cc:237] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.599051  2959 raft_consensus.cc:515] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.601245  2958 raft_consensus.cc:515] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.607537  2726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2" is_pre_election: true
I20260709 12:22:40.607867  2922 raft_consensus.cc:515] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.608536  2726 raft_consensus.cc:2468] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bff7abdc5314c13a9534adf58b5002e in term 0.
I20260709 12:22:40.609207  2801 raft_consensus.cc:2393] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 51c0e412fd9c4909bb014ae370587bd2 in current term 1: Already voted for candidate 697f4bdcd0944ad58ce468159b075fe8 in this term.
I20260709 12:22:40.609962  2957 raft_consensus.cc:2804] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:40.610392  2957 raft_consensus.cc:493] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.610818  2957 raft_consensus.cc:3060] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.610433  2725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2" is_pre_election: true
I20260709 12:22:40.611951  2916 tablet_bootstrap.cc:654] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.615514  2801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8" is_pre_election: true
I20260709 12:22:40.615620  2800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8" is_pre_election: true
I20260709 12:22:40.615293  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8" is_pre_election: true
I20260709 12:22:40.616540  2842 leader_election.cc:304] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:40.618142  2961 raft_consensus.cc:2804] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:40.619145  2961 raft_consensus.cc:493] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:40.619364  2842 leader_election.cc:343] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:40.620275  2961 raft_consensus.cc:3060] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.620812  2955 raft_consensus.cc:2804] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:40.621217  2955 raft_consensus.cc:493] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:40.625190  2842 leader_election.cc:343] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:40.623919  2955 raft_consensus.cc:3060] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.627736  2842 leader_election.cc:304] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9bff7abdc5314c13a9534adf58b5002e; no voters: 51c0e412fd9c4909bb014ae370587bd2, 697f4bdcd0944ad58ce468159b075fe8
I20260709 12:22:40.616585  2800 raft_consensus.cc:2468] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bff7abdc5314c13a9534adf58b5002e in term 0.
I20260709 12:22:40.624881  2959 leader_election.cc:290] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 election: Requested vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.629926  2921 raft_consensus.cc:2749] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:40.630457  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad190a87809b4ea7ac8d72146be24a61" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:40.631321  2875 raft_consensus.cc:2393] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 697f4bdcd0944ad58ce468159b075fe8 in current term 1: Already voted for candidate 51c0e412fd9c4909bb014ae370587bd2 in this term.
I20260709 12:22:40.629240  2922 leader_election.cc:290] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.635602  2958 leader_election.cc:290] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 election: Requested vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.635582  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ca5ce11aa9a430a8a06e20205ba2b1b" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:40.636469  2875 raft_consensus.cc:3060] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.637549  2801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ca5ce11aa9a430a8a06e20205ba2b1b" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:40.637560  2876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e4b64c0177467887dee4d5aa6e3a44" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:40.638468  2876 raft_consensus.cc:3060] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.643334  2800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e4b64c0177467887dee4d5aa6e3a44" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:40.643637  2725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad190a87809b4ea7ac8d72146be24a61" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:40.643952  2875 raft_consensus.cc:2468] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 1.
I20260709 12:22:40.644085  2800 raft_consensus.cc:3060] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.645255  2690 leader_election.cc:304] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:40.646551  2876 raft_consensus.cc:2468] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 1.
I20260709 12:22:40.638468  2801 raft_consensus.cc:3060] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.627668  2953 raft_consensus.cc:515] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.651343  2958 raft_consensus.cc:2804] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:40.656656  2800 raft_consensus.cc:2468] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 1.
I20260709 12:22:40.657858  2767 leader_election.cc:304] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8; no voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e
I20260709 12:22:40.658113  2801 raft_consensus.cc:2468] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 1.
I20260709 12:22:40.662935  2690 leader_election.cc:304] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:40.659526  2958 raft_consensus.cc:697] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Becoming Leader. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:40.665176  2923 raft_consensus.cc:2804] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:40.666000  2923 raft_consensus.cc:697] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Becoming Leader. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:40.667387  2922 raft_consensus.cc:2749] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:22:40.667179  2923 consensus_queue.cc:237] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.658344  2961 raft_consensus.cc:515] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.671011  2953 leader_election.cc:290] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 election: Requested vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.674688  2921 raft_consensus.cc:493] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.675284  2921 raft_consensus.cc:515] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.677515  2921 leader_election.cc:290] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485)
I20260709 12:22:40.679286  2725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae17ace8aad40fe8ec2b304a4a97cb5" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2" is_pre_election: true
I20260709 12:22:40.680523  2922 raft_consensus.cc:493] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.681031  2922 raft_consensus.cc:515] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.681525  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae17ace8aad40fe8ec2b304a4a97cb5" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8" is_pre_election: true
I20260709 12:22:40.682523  2799 raft_consensus.cc:2468] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bff7abdc5314c13a9534adf58b5002e in term 0.
I20260709 12:22:40.681926  2623 catalog_manager.cc:5697] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "51c0e412fd9c4909bb014ae370587bd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.685415  2917 ts_tablet_manager.cc:1434] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8: Time spent starting tablet: real 0.119s	user 0.016s	sys 0.013s
W20260709 12:22:40.685956  2842 leader_election.cc:343] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:40.687844  2917 tablet_bootstrap.cc:492] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8: Bootstrap starting.
I20260709 12:22:40.686754  2842 leader_election.cc:304] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 51c0e412fd9c4909bb014ae370587bd2
I20260709 12:22:40.689719  2800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0e8a1a9224999bd5bcfc28b4c4bb8" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:40.695564  2876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0e8a1a9224999bd5bcfc28b4c4bb8" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:40.698840  2921 raft_consensus.cc:2804] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:40.699007  2876 raft_consensus.cc:3060] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.699270  2921 raft_consensus.cc:493] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.699630  2921 raft_consensus.cc:3060] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.699831  2725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2" is_pre_election: true
I20260709 12:22:40.700584  2725 raft_consensus.cc:2468] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 0.
I20260709 12:22:40.703708  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e" is_pre_election: true
I20260709 12:22:40.699276  2922 leader_election.cc:290] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.705753  2915 tablet_bootstrap.cc:654] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.669107  2958 consensus_queue.cc:237] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.694240  2955 raft_consensus.cc:515] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
W20260709 12:22:40.709029  2765 leader_election.cc:343] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:40.712824  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:40.713546  2875 raft_consensus.cc:3060] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.707327  2922 raft_consensus.cc:493] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.706225  2800 raft_consensus.cc:3060] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.715694  2922 raft_consensus.cc:515] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.717734  2922 leader_election.cc:290] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.718906  2767 leader_election.cc:304] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 697f4bdcd0944ad58ce468159b075fe8; no voters: 9bff7abdc5314c13a9534adf58b5002e
I20260709 12:22:40.726166  2725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:40.727669  2726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "553040872de34092969954cb33c7de3b" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2" is_pre_election: true
I20260709 12:22:40.728200  2725 raft_consensus.cc:3060] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.728549  2726 raft_consensus.cc:2468] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 0.
I20260709 12:22:40.724884  2874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "553040872de34092969954cb33c7de3b" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e" is_pre_election: true
I20260709 12:22:40.719226  2961 leader_election.cc:290] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.733412  2874 raft_consensus.cc:2468] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 0.
I20260709 12:22:40.735872  2765 leader_election.cc:304] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:40.725234  2922 raft_consensus.cc:2804] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:40.741132  2961 raft_consensus.cc:2804] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:40.740695  2800 raft_consensus.cc:2468] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 1.
I20260709 12:22:40.743917  2726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:40.740264  2922 raft_consensus.cc:493] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.746243  2916 tablet_bootstrap.cc:492] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2: No bootstrap required, opened a new log
I20260709 12:22:40.749786  2916 ts_tablet_manager.cc:1403] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2: Time spent bootstrapping tablet: real 0.194s	user 0.016s	sys 0.000s
I20260709 12:22:40.751551  2957 raft_consensus.cc:515] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.753095  2916 raft_consensus.cc:359] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:40.753858  2916 raft_consensus.cc:385] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.754141  2916 raft_consensus.cc:740] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.755051  2916 consensus_queue.cc:260] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:40.756850  2725 raft_consensus.cc:2468] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 1.
I20260709 12:22:40.757339  2916 ts_tablet_manager.cc:1434] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:22:40.758487  2916 tablet_bootstrap.cc:492] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2: Bootstrap starting.
I20260709 12:22:40.746871  2961 raft_consensus.cc:493] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.758136  2955 leader_election.cc:290] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485)
I20260709 12:22:40.758483  2922 raft_consensus.cc:3060] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.763643  2726 raft_consensus.cc:2393] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9bff7abdc5314c13a9534adf58b5002e in current term 1: Already voted for candidate 51c0e412fd9c4909bb014ae370587bd2 in this term.
I20260709 12:22:40.747310  2876 raft_consensus.cc:2468] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 1.
I20260709 12:22:40.765208  2917 tablet_bootstrap.cc:654] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.768599  2921 raft_consensus.cc:515] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.766057  2800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:40.771072  2692 leader_election.cc:304] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 697f4bdcd0944ad58ce468159b075fe8; no voters: 
I20260709 12:22:40.771682  2800 raft_consensus.cc:3060] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.772208  2958 raft_consensus.cc:2804] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:40.772745  2958 raft_consensus.cc:697] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Becoming Leader. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:40.762476  2961 raft_consensus.cc:3060] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.766575  2957 leader_election.cc:290] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 election: Requested vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.770848  2874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:40.775580  2915 tablet_bootstrap.cc:492] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e: No bootstrap required, opened a new log
I20260709 12:22:40.772910  2726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae17ace8aad40fe8ec2b304a4a97cb5" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:40.776325  2915 ts_tablet_manager.cc:1403] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e: Time spent bootstrapping tablet: real 0.191s	user 0.016s	sys 0.004s
I20260709 12:22:40.775609  2874 raft_consensus.cc:2393] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 51c0e412fd9c4909bb014ae370587bd2 in current term 1: Already voted for candidate 9bff7abdc5314c13a9534adf58b5002e in this term.
I20260709 12:22:40.775513  2767 leader_election.cc:304] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 697f4bdcd0944ad58ce468159b075fe8; no voters: 
I20260709 12:22:40.761538  2875 raft_consensus.cc:2468] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 1.
I20260709 12:22:40.779394  2965 raft_consensus.cc:2804] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:22:40.779438  2842 leader_election.cc:343] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:40.781152  2958 consensus_queue.cc:237] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.788673  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:40.791942  2915 raft_consensus.cc:359] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.792707  2915 raft_consensus.cc:385] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.792999  2915 raft_consensus.cc:740] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.796257  2915 consensus_queue.cc:260] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.798238  2692 leader_election.cc:304] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2; no voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e
I20260709 12:22:40.799450  2957 raft_consensus.cc:2749] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:22:40.800879  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae17ace8aad40fe8ec2b304a4a97cb5" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:40.801680  2799 raft_consensus.cc:3060] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.802038  2800 raft_consensus.cc:2468] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bff7abdc5314c13a9534adf58b5002e in term 1.
I20260709 12:22:40.803292  2842 leader_election.cc:304] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 51c0e412fd9c4909bb014ae370587bd2
I20260709 12:22:40.799216  2921 leader_election.cc:290] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485)
I20260709 12:22:40.811023  2955 raft_consensus.cc:2804] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:40.811645  2955 raft_consensus.cc:697] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 1 LEADER]: Becoming Leader. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Running, Role: LEADER
I20260709 12:22:40.814040  2915 ts_tablet_manager.cc:1434] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 12:22:40.815183  2921 raft_consensus.cc:493] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.826613  2921 raft_consensus.cc:515] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.812696  2955 consensus_queue.cc:237] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.832998  2921 leader_election.cc:290] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485)
I20260709 12:22:40.836069  2917 tablet_bootstrap.cc:492] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8: No bootstrap required, opened a new log
I20260709 12:22:40.836596  2917 ts_tablet_manager.cc:1403] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8: Time spent bootstrapping tablet: real 0.149s	user 0.008s	sys 0.028s
I20260709 12:22:40.806111  2922 raft_consensus.cc:515] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.840498  2916 tablet_bootstrap.cc:654] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:40.841118  2726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:40.842895  2726 raft_consensus.cc:3060] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.843808  2917 raft_consensus.cc:359] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:40.829258  2799 raft_consensus.cc:2468] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bff7abdc5314c13a9534adf58b5002e in term 1.
I20260709 12:22:40.844843  2917 raft_consensus.cc:385] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.845109  2917 raft_consensus.cc:740] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.846107  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:40.846582  2965 raft_consensus.cc:697] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Running, Role: LEADER
I20260709 12:22:40.846715  2875 raft_consensus.cc:3060] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.845876  2917 consensus_queue.cc:260] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:40.844077  2922 leader_election.cc:290] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.859205  2961 raft_consensus.cc:515] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.861805  2961 leader_election.cc:290] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.863407  2725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "553040872de34092969954cb33c7de3b" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:40.863713  2874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "553040872de34092969954cb33c7de3b" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:40.864256  2874 raft_consensus.cc:3060] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.864174  2725 raft_consensus.cc:3060] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:40.853981  2842 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:46485, user_credentials={real_user=slave}} blocked reactor thread for 50794.3us
I20260709 12:22:40.871771  2842 leader_election.cc:304] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 51c0e412fd9c4909bb014ae370587bd2
I20260709 12:22:40.847506  2965 consensus_queue.cc:237] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.873081  2724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2" is_pre_election: true
I20260709 12:22:40.873692  2922 raft_consensus.cc:493] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.878973  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8" is_pre_election: true
I20260709 12:22:40.879930  2799 raft_consensus.cc:2393] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9bff7abdc5314c13a9534adf58b5002e in current term 1: Already voted for candidate 697f4bdcd0944ad58ce468159b075fe8 in this term.
I20260709 12:22:40.882054  2922 raft_consensus.cc:515] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:40.883548  2842 leader_election.cc:304] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9bff7abdc5314c13a9534adf58b5002e; no voters: 51c0e412fd9c4909bb014ae370587bd2, 697f4bdcd0944ad58ce468159b075fe8
I20260709 12:22:40.885936  2917 ts_tablet_manager.cc:1434] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8: Time spent starting tablet: real 0.049s	user 0.017s	sys 0.016s
I20260709 12:22:40.885912  2876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e" is_pre_election: true
I20260709 12:22:40.886694  2876 raft_consensus.cc:2468] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 0.
I20260709 12:22:40.887892  2765 leader_election.cc:304] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:40.891880  2874 raft_consensus.cc:2468] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 1.
I20260709 12:22:40.895915  2765 leader_election.cc:304] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:40.898684  2979 raft_consensus.cc:2804] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:40.900206  2979 raft_consensus.cc:697] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Running, Role: LEADER
I20260709 12:22:40.902036  2958 raft_consensus.cc:493] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.901456  2979 consensus_queue.cc:237] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.902730  2958 raft_consensus.cc:515] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:40.907364  2958 leader_election.cc:290] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271), 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485)
I20260709 12:22:40.899161  2973 raft_consensus.cc:2804] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:40.907725  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8" is_pre_election: true
I20260709 12:22:40.908034  2977 raft_consensus.cc:2804] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:40.908216  2973 raft_consensus.cc:493] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.911682  2973 raft_consensus.cc:3060] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.911108  2623 catalog_manager.cc:5697] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "51c0e412fd9c4909bb014ae370587bd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.912294  2977 raft_consensus.cc:697] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 1 LEADER]: Becoming Leader. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Running, Role: LEADER
I20260709 12:22:40.909399  2726 raft_consensus.cc:2468] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 1.
I20260709 12:22:40.910362  2874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e" is_pre_election: true
I20260709 12:22:40.896709  2875 raft_consensus.cc:2468] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 1.
I20260709 12:22:40.915005  2976 raft_consensus.cc:2749] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:40.918828  2922 leader_election.cc:290] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271), 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425)
I20260709 12:22:40.912978  2623 catalog_manager.cc:5697] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "51c0e412fd9c4909bb014ae370587bd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.923072  2767 leader_election.cc:304] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 697f4bdcd0944ad58ce468159b075fe8; no voters: 
I20260709 12:22:40.923875  2725 raft_consensus.cc:2468] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 1.
I20260709 12:22:40.926705  2726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2" is_pre_election: true
I20260709 12:22:40.927552  2916 tablet_bootstrap.cc:492] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2: No bootstrap required, opened a new log
I20260709 12:22:40.928155  2916 ts_tablet_manager.cc:1403] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2: Time spent bootstrapping tablet: real 0.170s	user 0.014s	sys 0.009s
I20260709 12:22:40.927932  2726 raft_consensus.cc:2468] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 0.
I20260709 12:22:40.929953  2623 catalog_manager.cc:5697] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "51c0e412fd9c4909bb014ae370587bd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.931087  2916 raft_consensus.cc:359] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.931834  2916 raft_consensus.cc:385] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:40.932206  2916 raft_consensus.cc:740] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Initialized, Role: FOLLOWER
I20260709 12:22:40.921586  2874 raft_consensus.cc:2468] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 0.
I20260709 12:22:40.926002  2965 raft_consensus.cc:2804] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:40.936831  2965 raft_consensus.cc:697] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Running, Role: LEADER
I20260709 12:22:40.937052  2973 raft_consensus.cc:515] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:40.937863  2965 consensus_queue.cc:237] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.938961  2799 raft_consensus.cc:2393] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 51c0e412fd9c4909bb014ae370587bd2 in current term 1: Already voted for candidate 697f4bdcd0944ad58ce468159b075fe8 in this term.
I20260709 12:22:40.940234  2725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:40.940910  2725 raft_consensus.cc:3060] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.941205  2690 leader_election.cc:304] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:40.933713  2916 consensus_queue.cc:260] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.913779  2977 consensus_queue.cc:237] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.931874  2621 catalog_manager.cc:5697] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "697f4bdcd0944ad58ce468159b075fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.927439  2622 catalog_manager.cc:5697] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195). New cstate: current_term: 1 leader_uuid: "9bff7abdc5314c13a9534adf58b5002e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:40.953572  2874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:40.954517  2874 raft_consensus.cc:3060] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.957398  2916 ts_tablet_manager.cc:1434] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.001s
I20260709 12:22:40.958230  2973 leader_election.cc:290] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271), 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425)
I20260709 12:22:40.960358  2725 raft_consensus.cc:2468] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 1.
I20260709 12:22:40.960913  2958 raft_consensus.cc:2764] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:22:40.966480  2874 raft_consensus.cc:2468] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 1.
I20260709 12:22:40.968014  2765 leader_election.cc:304] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:40.969014  2973 raft_consensus.cc:2804] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:40.969517  2973 raft_consensus.cc:697] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Running, Role: LEADER
I20260709 12:22:40.970430  2973 consensus_queue.cc:237] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:40.976647  2624 catalog_manager.cc:5697] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "697f4bdcd0944ad58ce468159b075fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.978819  2624 catalog_manager.cc:5697] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "697f4bdcd0944ad58ce468159b075fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.987528  2958 raft_consensus.cc:493] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.988103  2958 raft_consensus.cc:515] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:40.990372  2621 catalog_manager.cc:5697] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195). New cstate: current_term: 1 leader_uuid: "9bff7abdc5314c13a9534adf58b5002e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:40.992728  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae17ace8aad40fe8ec2b304a4a97cb5" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f4bdcd0944ad58ce468159b075fe8" is_pre_election: true
I20260709 12:22:40.993712  2799 raft_consensus.cc:2393] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 51c0e412fd9c4909bb014ae370587bd2 in current term 1: Already voted for candidate 9bff7abdc5314c13a9534adf58b5002e in this term.
I20260709 12:22:40.990391  2958 leader_election.cc:290] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:40.998214  2874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae17ace8aad40fe8ec2b304a4a97cb5" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bff7abdc5314c13a9534adf58b5002e" is_pre_election: true
I20260709 12:22:41.000003  2690 leader_election.cc:304] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2; no voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e
I20260709 12:22:41.001549  2958 raft_consensus.cc:3060] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:41.007963  2624 catalog_manager.cc:5697] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "697f4bdcd0944ad58ce468159b075fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:41.009851  2958 raft_consensus.cc:2749] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:41.057201  2958 consensus_queue.cc:1048] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:41.059262  2799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc"
dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
mode: GRACEFUL
new_leader_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:45298
I20260709 12:22:41.060091  2799 raft_consensus.cc:606] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Received request to transfer leadership to TS 51c0e412fd9c4909bb014ae370587bd2
I20260709 12:22:41.070569  2799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc"
dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
mode: GRACEFUL
new_leader_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:45298
I20260709 12:22:41.071238  2799 raft_consensus.cc:606] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Received request to transfer leadership to TS 51c0e412fd9c4909bb014ae370587bd2
I20260709 12:22:41.078675  2957 consensus_queue.cc:1048] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:41.080101  2799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b"
dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
mode: GRACEFUL
new_leader_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:45298
I20260709 12:22:41.080808  2799 raft_consensus.cc:606] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Received request to transfer leadership to TS 51c0e412fd9c4909bb014ae370587bd2
I20260709 12:22:41.102075  2874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aae17ace8aad40fe8ec2b304a4a97cb5"
dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
mode: GRACEFUL
new_leader_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:60952
I20260709 12:22:41.102669  2874 raft_consensus.cc:606] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 1 LEADER]: Received request to transfer leadership to TS 51c0e412fd9c4909bb014ae370587bd2
I20260709 12:22:41.122869  2725 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53e4b64c0177467887dee4d5aa6e3a44"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
mode: GRACEFUL
new_leader_uuid: "697f4bdcd0944ad58ce468159b075fe8"
 from {username='slave'} at 127.0.0.1:42922
I20260709 12:22:41.123654  2725 raft_consensus.cc:606] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Received request to transfer leadership to TS 697f4bdcd0944ad58ce468159b075fe8
I20260709 12:22:41.130055  2874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623"
dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
mode: GRACEFUL
new_leader_uuid: "697f4bdcd0944ad58ce468159b075fe8"
 from {username='slave'} at 127.0.0.1:60952
I20260709 12:22:41.130662  2874 raft_consensus.cc:606] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 1 LEADER]: Received request to transfer leadership to TS 697f4bdcd0944ad58ce468159b075fe8
I20260709 12:22:41.135025  2957 consensus_queue.cc:1048] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:41.137547  2725 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad190a87809b4ea7ac8d72146be24a61"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
mode: GRACEFUL
new_leader_uuid: "697f4bdcd0944ad58ce468159b075fe8"
 from {username='slave'} at 127.0.0.1:42922
I20260709 12:22:41.138226  2725 raft_consensus.cc:606] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Received request to transfer leadership to TS 697f4bdcd0944ad58ce468159b075fe8
I20260709 12:22:41.145100  2725 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7b0e8a1a9224999bd5bcfc28b4c4bb8"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
mode: GRACEFUL
new_leader_uuid: "9bff7abdc5314c13a9534adf58b5002e"
 from {username='slave'} at 127.0.0.1:42922
I20260709 12:22:41.145823  2725 raft_consensus.cc:606] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Received request to transfer leadership to TS 9bff7abdc5314c13a9534adf58b5002e
I20260709 12:22:41.148041  2953 consensus_queue.cc:1048] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:41.163021  2923 consensus_queue.cc:1048] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:41.176395  2958 consensus_queue.cc:1048] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:41.186045  2958 consensus_queue.cc:1048] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:41.191263  2953 raft_consensus.cc:993] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2: : Instructing follower 697f4bdcd0944ad58ce468159b075fe8 to start an election
I20260709 12:22:41.191875  2953 raft_consensus.cc:1081] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Signalling peer 697f4bdcd0944ad58ce468159b075fe8 to start an election
I20260709 12:22:41.194049  2800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad190a87809b4ea7ac8d72146be24a61"
dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
 from {username='slave'} at 127.0.0.1:45292
I20260709 12:22:41.194742  2800 raft_consensus.cc:493] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:41.199397  2953 consensus_queue.cc:1048] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:41.195142  2800 raft_consensus.cc:3060] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.209578  2800 raft_consensus.cc:515] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.214308  2953 raft_consensus.cc:993] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2: : Instructing follower 9bff7abdc5314c13a9534adf58b5002e to start an election
I20260709 12:22:41.214902  2953 raft_consensus.cc:1081] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Signalling peer 9bff7abdc5314c13a9534adf58b5002e to start an election
I20260709 12:22:41.216979  2725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad190a87809b4ea7ac8d72146be24a61" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:41.217725  2725 raft_consensus.cc:3055] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:41.217859  2800 leader_election.cc:290] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:41.218112  2725 raft_consensus.cc:740] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:41.219108  2725 consensus_queue.cc:260] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.219782  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad190a87809b4ea7ac8d72146be24a61" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:41.220341  2725 raft_consensus.cc:3060] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.220526  2875 raft_consensus.cc:3060] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.228946  2725 raft_consensus.cc:2468] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 2.
I20260709 12:22:41.229485  2875 raft_consensus.cc:2468] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 2.
I20260709 12:22:41.230456  2767 leader_election.cc:304] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 697f4bdcd0944ad58ce468159b075fe8; no voters: 
I20260709 12:22:41.231462  2973 raft_consensus.cc:2804] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:41.231952  2973 raft_consensus.cc:697] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Running, Role: LEADER
I20260709 12:22:41.232821  2973 consensus_queue.cc:237] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.238062  2874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7b0e8a1a9224999bd5bcfc28b4c4bb8"
dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
 from {username='slave'} at 127.0.0.1:60930
I20260709 12:22:41.238811  2874 raft_consensus.cc:493] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:41.239260  2874 raft_consensus.cc:3060] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.243351  2621 catalog_manager.cc:5697] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 reported cstate change: term changed from 1 to 2, leader changed from 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193) to 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "697f4bdcd0944ad58ce468159b075fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:41.245512  2874 raft_consensus.cc:515] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.247975  2874 leader_election.cc:290] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 2 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485)
I20260709 12:22:41.249017  2725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0e8a1a9224999bd5bcfc28b4c4bb8" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:41.249333  2800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b0e8a1a9224999bd5bcfc28b4c4bb8" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:41.249753  2725 raft_consensus.cc:3055] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:41.250129  2725 raft_consensus.cc:740] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:41.250165  2800 raft_consensus.cc:3060] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.251123  2725 consensus_queue.cc:260] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.252509  2725 raft_consensus.cc:3060] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.257380  2800 raft_consensus.cc:2468] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bff7abdc5314c13a9534adf58b5002e in term 2.
I20260709 12:22:41.258764  2842 leader_election.cc:304] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:41.259284  2725 raft_consensus.cc:2468] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bff7abdc5314c13a9534adf58b5002e in term 2.
I20260709 12:22:41.259884  2977 raft_consensus.cc:2804] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:41.260473  2977 raft_consensus.cc:697] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [term 2 LEADER]: Becoming Leader. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Running, Role: LEADER
I20260709 12:22:41.261462  2977 consensus_queue.cc:237] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.269340  2621 catalog_manager.cc:5697] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e reported cstate change: term changed from 1 to 2, leader changed from 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193) to 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195). New cstate: current_term: 2 leader_uuid: "9bff7abdc5314c13a9534adf58b5002e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:41.298565  2922 consensus_queue.cc:1048] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:41.311230  2965 consensus_queue.cc:1048] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:41.318030  2976 consensus_queue.cc:1048] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:41.327240  2977 raft_consensus.cc:993] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e: : Instructing follower 697f4bdcd0944ad58ce468159b075fe8 to start an election
I20260709 12:22:41.327757  2977 raft_consensus.cc:1081] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 1 LEADER]: Signalling peer 697f4bdcd0944ad58ce468159b075fe8 to start an election
I20260709 12:22:41.331549  2799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623"
dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
 from {username='slave'} at 127.0.0.1:45294
I20260709 12:22:41.337007  2799 raft_consensus.cc:493] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:41.337486  2799 raft_consensus.cc:3060] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.343080  2921 consensus_queue.cc:1048] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:41.343545  2799 raft_consensus.cc:515] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.348191  2726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:41.349206  2799 leader_election.cc:290] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:41.351478  2874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:41.352149  2874 raft_consensus.cc:3055] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:41.352504  2874 raft_consensus.cc:740] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Running, Role: LEADER
I20260709 12:22:41.353562  2874 consensus_queue.cc:260] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.354806  2874 raft_consensus.cc:3060] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.361049  2921 consensus_queue.cc:1048] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:41.362025  2874 raft_consensus.cc:2468] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 2.
I20260709 12:22:41.363227  2765 leader_election.cc:304] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 51c0e412fd9c4909bb014ae370587bd2
I20260709 12:22:41.364153  2965 raft_consensus.cc:2804] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:41.364837  2965 raft_consensus.cc:697] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Running, Role: LEADER
I20260709 12:22:41.365741  2965 consensus_queue.cc:237] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.367116  2977 consensus_queue.cc:1048] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:41.375440  2622 catalog_manager.cc:5697] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 reported cstate change: term changed from 1 to 2, leader changed from 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195) to 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "697f4bdcd0944ad58ce468159b075fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:41.379276  2976 raft_consensus.cc:993] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e: : Instructing follower 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:41.380419  2976 raft_consensus.cc:1081] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 1 LEADER]: Signalling peer 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:41.383536  2726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aae17ace8aad40fe8ec2b304a4a97cb5"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:42908
I20260709 12:22:41.384207  2726 raft_consensus.cc:493] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:41.384598  2726 raft_consensus.cc:3060] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.393671  2726 raft_consensus.cc:515] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.396261  2726 leader_election.cc:290] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 2 election: Requested vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:41.397145  2725 raft_consensus.cc:1240] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Rejecting Update request from peer 9bff7abdc5314c13a9534adf58b5002e for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:41.397473  2874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae17ace8aad40fe8ec2b304a4a97cb5" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:41.397158  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae17ace8aad40fe8ec2b304a4a97cb5" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:41.398202  2799 raft_consensus.cc:3060] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.398201  2874 raft_consensus.cc:3055] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:41.398918  2874 raft_consensus.cc:740] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Running, Role: LEADER
I20260709 12:22:41.399924  2874 consensus_queue.cc:260] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.401257  2874 raft_consensus.cc:3060] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.405134  2799 raft_consensus.cc:2468] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 2.
I20260709 12:22:41.406925  2692 leader_election.cc:304] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 697f4bdcd0944ad58ce468159b075fe8; no voters: 
I20260709 12:22:41.408110  2923 raft_consensus.cc:2804] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:41.408672  2923 raft_consensus.cc:697] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Becoming Leader. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:41.409142  2874 raft_consensus.cc:2468] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 2.
I20260709 12:22:41.409559  2923 consensus_queue.cc:237] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.410265  2965 consensus_queue.cc:1048] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:41.418592  2973 raft_consensus.cc:993] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8: : Instructing follower 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:41.418231  2624 catalog_manager.cc:5697] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 reported cstate change: term changed from 1 to 2, leader changed from 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195) to 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "51c0e412fd9c4909bb014ae370587bd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:41.419132  2973 raft_consensus.cc:1081] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Signalling peer 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:41.421490  2726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:42900
I20260709 12:22:41.424168  2979 consensus_queue.cc:1048] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:41.424490  2726 raft_consensus.cc:493] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:41.424975  2726 raft_consensus.cc:3060] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.433282  2965 consensus_queue.cc:1048] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:41.434278  2726 raft_consensus.cc:515] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.438213  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:41.438918  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:41.439502  2875 raft_consensus.cc:3060] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.438918  2799 raft_consensus.cc:3055] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:41.443159  2799 raft_consensus.cc:740] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Running, Role: LEADER
I20260709 12:22:41.444108  2799 consensus_queue.cc:260] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.445392  2799 raft_consensus.cc:3060] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.447930  2875 raft_consensus.cc:2468] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 2.
I20260709 12:22:41.449013  2726 leader_election.cc:290] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 2 election: Requested vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:41.451979  2690 leader_election.cc:304] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:41.453009  2923 raft_consensus.cc:2804] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:41.453516  2923 raft_consensus.cc:697] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Becoming Leader. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:41.454418  2923 consensus_queue.cc:237] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.457257  2922 consensus_queue.cc:1048] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:41.458752  2922 consensus_queue.cc:1048] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:41.468029  2799 raft_consensus.cc:2468] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 2.
I20260709 12:22:41.474327  2965 raft_consensus.cc:993] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8: : Instructing follower 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:41.474902  2965 raft_consensus.cc:1081] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Signalling peer 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:41.474284  2621 catalog_manager.cc:5697] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 reported cstate change: term changed from 1 to 2, leader changed from 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194) to 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "51c0e412fd9c4909bb014ae370587bd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:41.477312  2726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:42900
I20260709 12:22:41.478026  2726 raft_consensus.cc:493] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:41.478435  2726 raft_consensus.cc:3060] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.486943  2726 raft_consensus.cc:515] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:41.489890  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:41.490464  2875 raft_consensus.cc:3060] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.491477  2726 leader_election.cc:290] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 2 election: Requested vote from peers 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271), 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485)
I20260709 12:22:41.492010  2965 consensus_queue.cc:1048] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:41.494275  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:41.494948  2799 raft_consensus.cc:3055] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:41.495324  2799 raft_consensus.cc:740] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Running, Role: LEADER
I20260709 12:22:41.496322  2799 consensus_queue.cc:260] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:41.497603  2799 raft_consensus.cc:3060] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.498024  2875 raft_consensus.cc:2468] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 2.
I20260709 12:22:41.499279  2690 leader_election.cc:304] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:41.500232  2923 raft_consensus.cc:2804] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:41.500743  2923 raft_consensus.cc:697] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Becoming Leader. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:41.501668  2923 consensus_queue.cc:237] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:41.506140  2799 raft_consensus.cc:2468] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 2.
I20260709 12:22:41.515626  2973 raft_consensus.cc:993] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8: : Instructing follower 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:41.516194  2965 raft_consensus.cc:1081] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Signalling peer 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:41.518040  2726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:42900
I20260709 12:22:41.518600  2726 raft_consensus.cc:493] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:41.518932  2726 raft_consensus.cc:3060] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.524173  2726 raft_consensus.cc:515] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.526105  2726 leader_election.cc:290] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 2 election: Requested vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:41.527216  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:41.527376  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:41.527935  2799 raft_consensus.cc:3055] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:41.528023  2875 raft_consensus.cc:3060] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.528353  2799 raft_consensus.cc:740] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Running, Role: LEADER
I20260709 12:22:41.529294  2799 consensus_queue.cc:260] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.530742  2799 raft_consensus.cc:3060] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.531083  2621 catalog_manager.cc:5697] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 reported cstate change: term changed from 1 to 2, leader changed from 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194) to 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "51c0e412fd9c4909bb014ae370587bd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:41.534252  2875 raft_consensus.cc:2468] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 2.
I20260709 12:22:41.535871  2690 leader_election.cc:304] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:41.536872  2953 raft_consensus.cc:2804] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:41.537376  2953 raft_consensus.cc:697] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Becoming Leader. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:41.538285  2953 consensus_queue.cc:237] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.539378  2799 raft_consensus.cc:2468] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 2.
I20260709 12:22:41.546793  2621 catalog_manager.cc:5697] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 reported cstate change: term changed from 1 to 2, leader changed from 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194) to 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "51c0e412fd9c4909bb014ae370587bd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:41.568760  2923 raft_consensus.cc:993] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2: : Instructing follower 697f4bdcd0944ad58ce468159b075fe8 to start an election
I20260709 12:22:41.569423  2953 raft_consensus.cc:1081] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Signalling peer 697f4bdcd0944ad58ce468159b075fe8 to start an election
I20260709 12:22:41.571605  2799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53e4b64c0177467887dee4d5aa6e3a44"
dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
 from {username='slave'} at 127.0.0.1:45292
I20260709 12:22:41.572443  2799 raft_consensus.cc:493] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:41.572814  2799 raft_consensus.cc:3060] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.579061  2799 raft_consensus.cc:515] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.580946  2799 leader_election.cc:290] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:41.582260  2726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e4b64c0177467887dee4d5aa6e3a44" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:41.582306  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e4b64c0177467887dee4d5aa6e3a44" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:41.583069  2875 raft_consensus.cc:3060] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.583141  2726 raft_consensus.cc:3055] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:41.583618  2726 raft_consensus.cc:740] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:41.584548  2726 consensus_queue.cc:260] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.585857  2726 raft_consensus.cc:3060] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.589113  2875 raft_consensus.cc:2468] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 2.
I20260709 12:22:41.590492  2765 leader_election.cc:304] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:41.591318  2965 raft_consensus.cc:2804] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:41.591909  2965 raft_consensus.cc:697] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Running, Role: LEADER
I20260709 12:22:41.592806  2965 consensus_queue.cc:237] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:41.594462  2726 raft_consensus.cc:2468] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 2.
I20260709 12:22:41.601015  2621 catalog_manager.cc:5697] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 reported cstate change: term changed from 1 to 2, leader changed from 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193) to 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "697f4bdcd0944ad58ce468159b075fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:41.656983  2726 raft_consensus.cc:1275] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Refusing update from remote peer 9bff7abdc5314c13a9534adf58b5002e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.658591  2921 consensus_queue.cc:1048] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:41.667481  2799 raft_consensus.cc:1275] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Refusing update from remote peer 9bff7abdc5314c13a9534adf58b5002e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.669243  2955 consensus_queue.cc:1048] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:41.820573  2875 raft_consensus.cc:1275] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Refusing update from remote peer 697f4bdcd0944ad58ce468159b075fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.821741  2726 raft_consensus.cc:1275] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Refusing update from remote peer 697f4bdcd0944ad58ce468159b075fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.822386  2922 consensus_queue.cc:1048] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:41.823422  2965 consensus_queue.cc:1048] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:41.840958  2875 raft_consensus.cc:1275] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Refusing update from remote peer 697f4bdcd0944ad58ce468159b075fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.842780  2973 consensus_queue.cc:1048] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:41.853583  2726 raft_consensus.cc:3060] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:41.862090  2726 raft_consensus.cc:1275] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Refusing update from remote peer 697f4bdcd0944ad58ce468159b075fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.863844  2973 consensus_queue.cc:1048] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:41.917590  2875 raft_consensus.cc:1275] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.919252  3012 consensus_queue.cc:1048] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:41.928737  2799 raft_consensus.cc:1275] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.930480  2953 consensus_queue.cc:1048] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:41.942323  2875 raft_consensus.cc:1275] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.944037  2953 consensus_queue.cc:1048] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:41.953702  2799 raft_consensus.cc:1275] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.955647  3012 consensus_queue.cc:1048] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:41.976482  2875 raft_consensus.cc:1275] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.977982  3012 consensus_queue.cc:1048] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:41.987495  2799 raft_consensus.cc:1275] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.990046  2923 consensus_queue.cc:1048] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:41.990715  2726 raft_consensus.cc:1275] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Refusing update from remote peer 697f4bdcd0944ad58ce468159b075fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.993257  2973 consensus_queue.cc:1048] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:42.006203  2875 raft_consensus.cc:1275] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Refusing update from remote peer 697f4bdcd0944ad58ce468159b075fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:42.008090  2979 consensus_queue.cc:1048] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:42.101820  2875 raft_consensus.cc:1275] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:42.103520  2953 consensus_queue.cc:1048] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:42.112018  2799 raft_consensus.cc:1275] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:42.114135  2923 consensus_queue.cc:1048] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:44.187951  2067 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:22:44.188933  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:44.207870  2726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
mode: GRACEFUL
new_leader_uuid: "9bff7abdc5314c13a9534adf58b5002e"
 from {username='slave'} at 127.0.0.1:42934
I20260709 12:22:44.208490  2726 raft_consensus.cc:606] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Received request to transfer leadership to TS 9bff7abdc5314c13a9534adf58b5002e
I20260709 12:22:44.211634  2726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
mode: GRACEFUL
new_leader_uuid: "9bff7abdc5314c13a9534adf58b5002e"
 from {username='slave'} at 127.0.0.1:42934
I20260709 12:22:44.212191  2726 raft_consensus.cc:606] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Received request to transfer leadership to TS 9bff7abdc5314c13a9534adf58b5002e
I20260709 12:22:44.213436  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:44.214197  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:44.342267  3016 raft_consensus.cc:993] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2: : Instructing follower 9bff7abdc5314c13a9534adf58b5002e to start an election
I20260709 12:22:44.342715  2923 raft_consensus.cc:1081] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Signalling peer 9bff7abdc5314c13a9534adf58b5002e to start an election
I20260709 12:22:44.344334  2875 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b"
dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
 from {username='slave'} at 127.0.0.1:60930
I20260709 12:22:44.344871  2875 raft_consensus.cc:493] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:44.345172  2875 raft_consensus.cc:3060] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:44.351110  2875 raft_consensus.cc:515] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:44.352902  2875 leader_election.cc:290] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 3 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485)
I20260709 12:22:44.353958  2726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:44.354272  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:44.354578  2726 raft_consensus.cc:3055] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:44.354920  2799 raft_consensus.cc:3060] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:44.354959  2726 raft_consensus.cc:740] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:44.355911  2726 consensus_queue.cc:260] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:44.357306  2726 raft_consensus.cc:3060] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:44.361343  2799 raft_consensus.cc:2468] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bff7abdc5314c13a9534adf58b5002e in term 3.
I20260709 12:22:44.362680  2842 leader_election.cc:304] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:44.363462  3014 raft_consensus.cc:2804] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:44.364580  3014 raft_consensus.cc:697] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 3 LEADER]: Becoming Leader. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Running, Role: LEADER
I20260709 12:22:44.364693  2726 raft_consensus.cc:2468] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bff7abdc5314c13a9534adf58b5002e in term 3.
I20260709 12:22:44.365479  3014 consensus_queue.cc:237] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:44.374470  2621 catalog_manager.cc:5697] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e reported cstate change: term changed from 2 to 3, leader changed from 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193) to 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195). New cstate: current_term: 3 leader_uuid: "9bff7abdc5314c13a9534adf58b5002e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:44.393934  2923 raft_consensus.cc:993] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2: : Instructing follower 9bff7abdc5314c13a9534adf58b5002e to start an election
I20260709 12:22:44.394415  3016 raft_consensus.cc:1081] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Signalling peer 9bff7abdc5314c13a9534adf58b5002e to start an election
I20260709 12:22:44.395888  2875 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc"
dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
 from {username='slave'} at 127.0.0.1:60930
I20260709 12:22:44.396307  2875 raft_consensus.cc:493] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:44.396538  2875 raft_consensus.cc:3060] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:44.401340  2875 raft_consensus.cc:515] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:44.402872  2875 leader_election.cc:290] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 3 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485)
I20260709 12:22:44.404094  2726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:44.404271  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:44.404635  2726 raft_consensus.cc:3055] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:44.404888  2799 raft_consensus.cc:3060] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:44.405017  2726 raft_consensus.cc:740] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:44.406018  2726 consensus_queue.cc:260] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:44.407045  2726 raft_consensus.cc:3060] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:44.410171  2799 raft_consensus.cc:2468] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bff7abdc5314c13a9534adf58b5002e in term 3.
I20260709 12:22:44.411619  2842 leader_election.cc:304] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:44.412441  3014 raft_consensus.cc:2804] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:44.412961  3014 raft_consensus.cc:697] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 3 LEADER]: Becoming Leader. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Running, Role: LEADER
I20260709 12:22:44.413546  2726 raft_consensus.cc:2468] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bff7abdc5314c13a9534adf58b5002e in term 3.
I20260709 12:22:44.413934  3014 consensus_queue.cc:237] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:44.422369  2621 catalog_manager.cc:5697] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e reported cstate change: term changed from 2 to 3, leader changed from 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193) to 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195). New cstate: current_term: 3 leader_uuid: "9bff7abdc5314c13a9534adf58b5002e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:44.828886  2726 raft_consensus.cc:1275] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Refusing update from remote peer 9bff7abdc5314c13a9534adf58b5002e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:44.830267  3014 consensus_queue.cc:1048] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:44.841846  2799 raft_consensus.cc:1275] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Refusing update from remote peer 9bff7abdc5314c13a9534adf58b5002e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:44.844250  3019 consensus_queue.cc:1048] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:44.851449  2800 raft_consensus.cc:1275] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Refusing update from remote peer 9bff7abdc5314c13a9534adf58b5002e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:44.853875  3019 consensus_queue.cc:1048] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:44.875536  2726 raft_consensus.cc:1275] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Refusing update from remote peer 9bff7abdc5314c13a9534adf58b5002e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:44.878340  3019 consensus_queue.cc:1048] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:45.215583  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:45.221818  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:45.222275  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:46.223320  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:46.229151  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:46.229554  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:47.230556  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:47.236161  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:47.236563  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:48.237835  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:48.244131  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:48.244678  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:49.245867  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:49.251210  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:49.251703  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:50.252977  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:50.258793  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:50.259358  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:51.260667  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:51.266759  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:51.267290  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:52.268651  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:52.274971  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:52.275383  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:53.276407  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:53.284672  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:53.285125  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:54.326761  2067 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:22:54.335990  2875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc"
dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
mode: GRACEFUL
new_leader_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:60952
I20260709 12:22:54.336817  2875 raft_consensus.cc:606] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 3 LEADER]: Received request to transfer leadership to TS 51c0e412fd9c4909bb014ae370587bd2
I20260709 12:22:54.343361  2875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b"
dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
mode: GRACEFUL
new_leader_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:60952
I20260709 12:22:54.343911  2875 raft_consensus.cc:606] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 3 LEADER]: Received request to transfer leadership to TS 51c0e412fd9c4909bb014ae370587bd2
I20260709 12:22:54.357100  2800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "553040872de34092969954cb33c7de3b"
dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
mode: GRACEFUL
new_leader_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:45298
I20260709 12:22:54.357767  2800 raft_consensus.cc:606] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Received request to transfer leadership to TS 51c0e412fd9c4909bb014ae370587bd2
I20260709 12:22:54.364465  2799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53e4b64c0177467887dee4d5aa6e3a44"
dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
mode: GRACEFUL
new_leader_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:45298
I20260709 12:22:54.365104  2799 raft_consensus.cc:606] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 LEADER]: Received request to transfer leadership to TS 51c0e412fd9c4909bb014ae370587bd2
I20260709 12:22:54.371121  2799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623"
dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
mode: GRACEFUL
new_leader_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:45298
I20260709 12:22:54.371695  2799 raft_consensus.cc:606] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 LEADER]: Received request to transfer leadership to TS 51c0e412fd9c4909bb014ae370587bd2
I20260709 12:22:54.404340  3044 raft_consensus.cc:993] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8: : Instructing follower 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:54.405150  3046 raft_consensus.cc:1081] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Signalling peer 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:54.407241  2726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "553040872de34092969954cb33c7de3b"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:42900
I20260709 12:22:54.407805  2726 raft_consensus.cc:493] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:54.408166  2726 raft_consensus.cc:3060] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:54.413981  2726 raft_consensus.cc:515] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:54.415969  2726 leader_election.cc:290] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 2 election: Requested vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:54.417387  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "553040872de34092969954cb33c7de3b" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:54.417387  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "553040872de34092969954cb33c7de3b" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:54.418510  2875 raft_consensus.cc:3060] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:54.418510  2799 raft_consensus.cc:3055] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:54.419021  2799 raft_consensus.cc:740] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Running, Role: LEADER
I20260709 12:22:54.420045  2799 consensus_queue.cc:260] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:54.421571  2799 raft_consensus.cc:3060] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:54.426051  2875 raft_consensus.cc:2468] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 2.
I20260709 12:22:54.427424  2690 leader_election.cc:304] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:54.428337  3037 raft_consensus.cc:2804] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:54.428471  2799 raft_consensus.cc:2468] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 2.
I20260709 12:22:54.429497  3037 raft_consensus.cc:697] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Becoming Leader. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:54.430569  3037 consensus_queue.cc:237] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:54.438869  2621 catalog_manager.cc:5697] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 reported cstate change: term changed from 1 to 2, leader changed from 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194) to 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "51c0e412fd9c4909bb014ae370587bd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:54.668397  3044 raft_consensus.cc:993] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8: : Instructing follower 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:54.669025  3046 raft_consensus.cc:1081] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 LEADER]: Signalling peer 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:54.670735  2726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53e4b64c0177467887dee4d5aa6e3a44"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:42900
I20260709 12:22:54.671173  2726 raft_consensus.cc:493] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:54.671458  2726 raft_consensus.cc:3060] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:54.676267  2726 raft_consensus.cc:515] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:54.677906  2726 leader_election.cc:290] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 3 election: Requested vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:54.679006  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e4b64c0177467887dee4d5aa6e3a44" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:54.678905  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e4b64c0177467887dee4d5aa6e3a44" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:54.679514  2875 raft_consensus.cc:3060] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:54.679627  2799 raft_consensus.cc:3055] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:54.680035  2799 raft_consensus.cc:740] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Running, Role: LEADER
I20260709 12:22:54.680835  2799 consensus_queue.cc:260] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:54.681933  2799 raft_consensus.cc:3060] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:54.684871  2875 raft_consensus.cc:2468] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 3.
I20260709 12:22:54.686148  2690 leader_election.cc:304] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:54.686918  3037 raft_consensus.cc:2804] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:54.687327  3037 raft_consensus.cc:697] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 3 LEADER]: Becoming Leader. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:54.688184  2799 raft_consensus.cc:2468] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 3.
I20260709 12:22:54.688150  3037 consensus_queue.cc:237] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:54.696957  3045 raft_consensus.cc:993] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e: : Instructing follower 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:54.697798  3054 raft_consensus.cc:1081] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 3 LEADER]: Signalling peer 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:54.697191  2621 catalog_manager.cc:5697] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 reported cstate change: term changed from 2 to 3, leader changed from 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194) to 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193). New cstate: current_term: 3 leader_uuid: "51c0e412fd9c4909bb014ae370587bd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:54.699905  2726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:42908
I20260709 12:22:54.700487  2726 raft_consensus.cc:493] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:54.700850  2726 raft_consensus.cc:3060] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:54.707639  2726 raft_consensus.cc:515] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:54.709365  2726 leader_election.cc:290] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 4 election: Requested vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:54.710203  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:54.710616  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4473c73827fc420a9222f7b3947ba58b" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:54.710806  2799 raft_consensus.cc:3060] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:54.711205  2875 raft_consensus.cc:3055] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:54.711469  2875 raft_consensus.cc:740] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Running, Role: LEADER
I20260709 12:22:54.712220  2875 consensus_queue.cc:260] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:54.713222  2875 raft_consensus.cc:3060] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:54.716262  2799 raft_consensus.cc:2468] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 4.
I20260709 12:22:54.717780  2692 leader_election.cc:304] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 697f4bdcd0944ad58ce468159b075fe8; no voters: 
I20260709 12:22:54.718818  3037 raft_consensus.cc:2804] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:54.719257  2875 raft_consensus.cc:2468] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 4.
I20260709 12:22:54.719349  3037 raft_consensus.cc:697] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 4 LEADER]: Becoming Leader. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:54.720487  3037 consensus_queue.cc:237] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:54.727948  2621 catalog_manager.cc:5697] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 reported cstate change: term changed from 3 to 4, leader changed from 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195) to 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193). New cstate: current_term: 4 leader_uuid: "51c0e412fd9c4909bb014ae370587bd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:54.822347  3044 raft_consensus.cc:993] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8: : Instructing follower 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:54.822887  3044 raft_consensus.cc:1081] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 LEADER]: Signalling peer 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:54.824658  2726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:42900
I20260709 12:22:54.825188  2726 raft_consensus.cc:493] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:54.825452  2726 raft_consensus.cc:3060] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:54.830929  2726 raft_consensus.cc:515] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:54.832787  2726 leader_election.cc:290] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 3 election: Requested vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:54.834263  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:54.834770  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64d4799f52d9401c8c86306eeb1a3623" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:54.835325  2875 raft_consensus.cc:3060] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:54.835424  2799 raft_consensus.cc:3055] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:54.836400  2799 raft_consensus.cc:740] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Running, Role: LEADER
I20260709 12:22:54.837419  2800 raft_consensus.cc:1275] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:54.837443  2799 consensus_queue.cc:260] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:54.839075  2799 raft_consensus.cc:3060] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:54.839118  3037 consensus_queue.cc:1048] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:54.844103  2875 raft_consensus.cc:2468] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 3.
I20260709 12:22:54.845964  2690 leader_election.cc:304] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:54.846884  3037 raft_consensus.cc:2804] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:54.848443  3037 raft_consensus.cc:697] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 3 LEADER]: Becoming Leader. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:54.849561  3037 consensus_queue.cc:237] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:54.847183  2799 raft_consensus.cc:2468] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 3.
I20260709 12:22:54.858812  2875 raft_consensus.cc:1275] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:54.860999  3058 consensus_queue.cc:1048] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:54.866314  2624 catalog_manager.cc:5697] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 reported cstate change: term changed from 2 to 3, leader changed from 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194) to 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193). New cstate: current_term: 3 leader_uuid: "51c0e412fd9c4909bb014ae370587bd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:54.907145  3045 raft_consensus.cc:993] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e: : Instructing follower 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:54.907722  3045 raft_consensus.cc:1081] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 3 LEADER]: Signalling peer 51c0e412fd9c4909bb014ae370587bd2 to start an election
I20260709 12:22:54.909576  2726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
 from {username='slave'} at 127.0.0.1:42908
I20260709 12:22:54.910211  2726 raft_consensus.cc:493] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:54.910533  2726 raft_consensus.cc:3060] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:54.915735  2726 raft_consensus.cc:515] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:54.917479  2726 leader_election.cc:290] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 4 election: Requested vote from peers 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:54.918428  2800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:54.918758  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc" candidate_uuid: "51c0e412fd9c4909bb014ae370587bd2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:54.919102  2800 raft_consensus.cc:3060] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:54.919224  2875 raft_consensus.cc:3055] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:54.919539  2875 raft_consensus.cc:740] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Running, Role: LEADER
I20260709 12:22:54.920378  2875 consensus_queue.cc:260] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:54.921571  2875 raft_consensus.cc:3060] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:54.926368  2800 raft_consensus.cc:2468] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 4.
I20260709 12:22:54.927791  2692 leader_election.cc:304] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 697f4bdcd0944ad58ce468159b075fe8; no voters: 
I20260709 12:22:54.928102  2875 raft_consensus.cc:2468] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c0e412fd9c4909bb014ae370587bd2 in term 4.
I20260709 12:22:54.928639  3058 raft_consensus.cc:2804] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:54.929193  3058 raft_consensus.cc:697] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 4 LEADER]: Becoming Leader. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:54.930053  3058 consensus_queue.cc:237] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:54.940557  2621 catalog_manager.cc:5697] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 reported cstate change: term changed from 3 to 4, leader changed from 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195) to 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193). New cstate: current_term: 4 leader_uuid: "51c0e412fd9c4909bb014ae370587bd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:55.128003  2800 raft_consensus.cc:1275] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 4 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:55.129510  3058 consensus_queue.cc:1048] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:55.140062  2875 raft_consensus.cc:1275] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 4 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:55.142354  3047 consensus_queue.cc:1048] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:55.195951  2875 raft_consensus.cc:1275] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:55.197613  3058 consensus_queue.cc:1048] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:55.209029  2800 raft_consensus.cc:1275] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:55.211282  3057 consensus_queue.cc:1048] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:55.277906  2875 raft_consensus.cc:1275] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:55.279472  3057 consensus_queue.cc:1048] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:55.289749  2800 raft_consensus.cc:1275] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:55.291282  3037 consensus_queue.cc:1048] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:55.375088  2875 raft_consensus.cc:1275] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 4 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:55.376634  3037 consensus_queue.cc:1048] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:55.386910  2800 raft_consensus.cc:1275] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 4 FOLLOWER]: Refusing update from remote peer 51c0e412fd9c4909bb014ae370587bd2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:55.389654  3037 consensus_queue.cc:1048] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:57.422297  2067 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:22:57.423127  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:57.432497  2726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
mode: GRACEFUL
new_leader_uuid: "9bff7abdc5314c13a9534adf58b5002e"
 from {username='slave'} at 127.0.0.1:42934
I20260709 12:22:57.433255  2726 raft_consensus.cc:606] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Received request to transfer leadership to TS 9bff7abdc5314c13a9534adf58b5002e
I20260709 12:22:57.436905  2726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ca5ce11aa9a430a8a06e20205ba2b1b"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
mode: GRACEFUL
new_leader_uuid: "9bff7abdc5314c13a9534adf58b5002e"
 from {username='slave'} at 127.0.0.1:42934
I20260709 12:22:57.437559  2726 raft_consensus.cc:606] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Received request to transfer leadership to TS 9bff7abdc5314c13a9534adf58b5002e
I20260709 12:22:57.440973  2726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aae17ace8aad40fe8ec2b304a4a97cb5"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
mode: GRACEFUL
new_leader_uuid: "697f4bdcd0944ad58ce468159b075fe8"
 from {username='slave'} at 127.0.0.1:42934
I20260709 12:22:57.441701  2726 raft_consensus.cc:606] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Received request to transfer leadership to TS 697f4bdcd0944ad58ce468159b075fe8
I20260709 12:22:57.444873  2726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc"
dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
mode: GRACEFUL
new_leader_uuid: "697f4bdcd0944ad58ce468159b075fe8"
 from {username='slave'} at 127.0.0.1:42934
I20260709 12:22:57.445570  2726 raft_consensus.cc:606] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 4 LEADER]: Received request to transfer leadership to TS 697f4bdcd0944ad58ce468159b075fe8
I20260709 12:22:57.445915  3057 raft_consensus.cc:993] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2: : Instructing follower 9bff7abdc5314c13a9534adf58b5002e to start an election
I20260709 12:22:57.446473  3074 raft_consensus.cc:1081] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Signalling peer 9bff7abdc5314c13a9534adf58b5002e to start an election
I20260709 12:22:57.446902  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:22:57.447674  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:57.448810  2875 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc"
dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
 from {username='slave'} at 127.0.0.1:60930
I20260709 12:22:57.449419  2875 raft_consensus.cc:493] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:57.449772  2875 raft_consensus.cc:3060] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:57.455072  2875 raft_consensus.cc:515] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:57.457310  2875 leader_election.cc:290] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 3 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485)
I20260709 12:22:57.458647  2726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:57.458891  2799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bb8c0788a7d4d0c8e19fbd44da9fbbc" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:57.459280  2726 raft_consensus.cc:3055] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:57.459452  2799 raft_consensus.cc:3060] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:57.459703  2726 raft_consensus.cc:740] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:57.460752  2726 consensus_queue.cc:260] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:57.461853  2726 raft_consensus.cc:3060] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:57.465173  2799 raft_consensus.cc:2468] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bff7abdc5314c13a9534adf58b5002e in term 3.
I20260709 12:22:57.466787  2842 leader_election.cc:304] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:57.467061  2726 raft_consensus.cc:2468] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bff7abdc5314c13a9534adf58b5002e in term 3.
I20260709 12:22:57.467620  3054 raft_consensus.cc:2804] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:57.468842  3054 raft_consensus.cc:697] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 3 LEADER]: Becoming Leader. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Running, Role: LEADER
I20260709 12:22:57.469985  3054 consensus_queue.cc:237] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } }
I20260709 12:22:57.477169  2624 catalog_manager.cc:5697] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e reported cstate change: term changed from 2 to 3, leader changed from 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193) to 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195). New cstate: current_term: 3 leader_uuid: "9bff7abdc5314c13a9534adf58b5002e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.859619  2726 raft_consensus.cc:1275] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Refusing update from remote peer 9bff7abdc5314c13a9534adf58b5002e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:57.861526  3057 raft_consensus.cc:993] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2: : Instructing follower 697f4bdcd0944ad58ce468159b075fe8 to start an election
I20260709 12:22:57.862176  3075 consensus_queue.cc:1048] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:57.863116  3074 raft_consensus.cc:1081] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Signalling peer 697f4bdcd0944ad58ce468159b075fe8 to start an election
I20260709 12:22:57.864599  2799 raft_consensus.cc:1275] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Refusing update from remote peer 9bff7abdc5314c13a9534adf58b5002e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:57.865475  2800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aae17ace8aad40fe8ec2b304a4a97cb5"
dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
 from {username='slave'} at 127.0.0.1:45292
I20260709 12:22:57.866344  2800 raft_consensus.cc:493] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:57.866792  2800 raft_consensus.cc:3060] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:57.868367  3075 consensus_queue.cc:1048] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:57.878897  3057 raft_consensus.cc:993] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2: : Instructing follower 697f4bdcd0944ad58ce468159b075fe8 to start an election
I20260709 12:22:57.882027  3074 raft_consensus.cc:1081] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 4 LEADER]: Signalling peer 697f4bdcd0944ad58ce468159b075fe8 to start an election
I20260709 12:22:57.886286  2801 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc"
dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
 from {username='slave'} at 127.0.0.1:45292
I20260709 12:22:57.886088  2800 raft_consensus.cc:515] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:57.887379  2801 raft_consensus.cc:493] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:57.887976  2801 raft_consensus.cc:3060] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:57.891063  2800 leader_election.cc:290] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:57.893903  2875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae17ace8aad40fe8ec2b304a4a97cb5" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:57.894173  3083 raft_consensus.cc:993] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2: : Instructing follower 9bff7abdc5314c13a9534adf58b5002e to start an election
I20260709 12:22:57.894616  2875 raft_consensus.cc:3060] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:57.895195  3057 raft_consensus.cc:1081] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Signalling peer 9bff7abdc5314c13a9534adf58b5002e to start an election
I20260709 12:22:57.896888  2725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae17ace8aad40fe8ec2b304a4a97cb5" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:57.897596  2725 raft_consensus.cc:3055] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:57.897953  2725 raft_consensus.cc:740] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:57.898200  2874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ca5ce11aa9a430a8a06e20205ba2b1b"
dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
 from {username='slave'} at 127.0.0.1:60930
I20260709 12:22:57.899088  2874 raft_consensus.cc:493] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:57.898970  2725 consensus_queue.cc:260] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:57.899649  2874 raft_consensus.cc:3060] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:57.900395  2725 raft_consensus.cc:3060] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:57.908067  2801 raft_consensus.cc:515] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:57.915344  2801 leader_election.cc:290] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195:46271)
I20260709 12:22:57.917229  2876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9bff7abdc5314c13a9534adf58b5002e"
I20260709 12:22:57.918071  2876 raft_consensus.cc:3060] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:57.908754  2875 raft_consensus.cc:2468] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 3.
I20260709 12:22:57.920173  2765 leader_election.cc:304] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:57.921725  3044 raft_consensus.cc:2804] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:57.908808  2874 raft_consensus.cc:515] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:57.910439  2725 raft_consensus.cc:2468] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 3.
I20260709 12:22:57.928957  2726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c45c2ae497034799b8ddab73de61c8dc" candidate_uuid: "697f4bdcd0944ad58ce468159b075fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:57.929754  2726 raft_consensus.cc:3055] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:57.930073  2726 raft_consensus.cc:740] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:57.929982  2874 leader_election.cc:290] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 2 election: Requested vote from peers 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425), 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194:46485)
I20260709 12:22:57.931208  2801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ca5ce11aa9a430a8a06e20205ba2b1b" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "697f4bdcd0944ad58ce468159b075fe8"
I20260709 12:22:57.927707  2876 raft_consensus.cc:2468] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 5.
I20260709 12:22:57.931929  2801 raft_consensus.cc:3060] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:57.932790  2765 leader_election.cc:304] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 697f4bdcd0944ad58ce468159b075fe8, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:57.932098  2726 consensus_queue.cc:260] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:57.937345  2726 raft_consensus.cc:3060] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:57.941339  2725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ca5ce11aa9a430a8a06e20205ba2b1b" candidate_uuid: "9bff7abdc5314c13a9534adf58b5002e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51c0e412fd9c4909bb014ae370587bd2"
I20260709 12:22:57.942073  2725 raft_consensus.cc:3055] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:57.942359  2725 raft_consensus.cc:740] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51c0e412fd9c4909bb014ae370587bd2, State: Running, Role: LEADER
I20260709 12:22:57.943244  2725 consensus_queue.cc:260] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:57.944702  2725 raft_consensus.cc:3060] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:57.945245  3092 raft_consensus.cc:2804] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:57.945894  3092 raft_consensus.cc:697] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 5 LEADER]: Becoming Leader. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Running, Role: LEADER
I20260709 12:22:57.946877  3044 raft_consensus.cc:697] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 3 LEADER]: Becoming Leader. State: Replica: 697f4bdcd0944ad58ce468159b075fe8, State: Running, Role: LEADER
I20260709 12:22:57.949317  3044 consensus_queue.cc:237] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:57.952463  2725 raft_consensus.cc:2468] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bff7abdc5314c13a9534adf58b5002e in term 2.
I20260709 12:22:57.953522  2801 raft_consensus.cc:2468] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bff7abdc5314c13a9534adf58b5002e in term 2.
I20260709 12:22:57.953478  3092 consensus_queue.cc:237] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:57.955101  2842 leader_election.cc:304] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c0e412fd9c4909bb014ae370587bd2, 9bff7abdc5314c13a9534adf58b5002e; no voters: 
I20260709 12:22:57.956791  3088 raft_consensus.cc:2804] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:57.957437  2726 raft_consensus.cc:2468] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 697f4bdcd0944ad58ce468159b075fe8 in term 5.
I20260709 12:22:57.959012  3088 raft_consensus.cc:697] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [term 2 LEADER]: Becoming Leader. State: Replica: 9bff7abdc5314c13a9534adf58b5002e, State: Running, Role: LEADER
I20260709 12:22:57.960126  3088 consensus_queue.cc:237] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } }
I20260709 12:22:57.970079  2624 catalog_manager.cc:5697] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 reported cstate change: term changed from 2 to 3, leader changed from 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193) to 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194). New cstate: current_term: 3 leader_uuid: "697f4bdcd0944ad58ce468159b075fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.971347  2621 catalog_manager.cc:5697] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e reported cstate change: term changed from 1 to 2, leader changed from 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193) to 9bff7abdc5314c13a9534adf58b5002e (127.2.4.195). New cstate: current_term: 2 leader_uuid: "9bff7abdc5314c13a9534adf58b5002e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:57.988103  2621 catalog_manager.cc:5697] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 reported cstate change: term changed from 4 to 5, leader changed from 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193) to 697f4bdcd0944ad58ce468159b075fe8 (127.2.4.194). New cstate: current_term: 5 leader_uuid: "697f4bdcd0944ad58ce468159b075fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:58.343297  2876 raft_consensus.cc:1275] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Refusing update from remote peer 697f4bdcd0944ad58ce468159b075fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:58.343935  2799 raft_consensus.cc:1275] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Refusing update from remote peer 9bff7abdc5314c13a9534adf58b5002e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:58.344916  3092 consensus_queue.cc:1048] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:58.345542  3088 consensus_queue.cc:1048] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f4bdcd0944ad58ce468159b075fe8" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:58.362035  2726 raft_consensus.cc:1275] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Refusing update from remote peer 697f4bdcd0944ad58ce468159b075fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:58.365247  3092 consensus_queue.cc:1048] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:58.369421  2725 raft_consensus.cc:1275] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Refusing update from remote peer 9bff7abdc5314c13a9534adf58b5002e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:58.375293  2876 raft_consensus.cc:1275] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 5 FOLLOWER]: Refusing update from remote peer 697f4bdcd0944ad58ce468159b075fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:58.378549  3044 consensus_queue.cc:1048] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bff7abdc5314c13a9534adf58b5002e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:58.382367  3075 consensus_queue.cc:1048] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:22:58.409799  2726 raft_consensus.cc:1275] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 5 FOLLOWER]: Refusing update from remote peer 697f4bdcd0944ad58ce468159b075fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:58.415055  3090 consensus_queue.cc:1048] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c0e412fd9c4909bb014ae370587bd2" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:58.449296  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:58.458271  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:58.459112  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:59.460526  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:59.468716  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:59.469331  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:00.536836  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:00.543823  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:00.544459  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:23:01.316010  2655 debug-util.cc:398] Leaking SignalData structure 0x7b0800054440 after lost signal to thread 2070
W20260709 12:23:01.317039  2655 debug-util.cc:398] Leaking SignalData structure 0x7b0800054b20 after lost signal to thread 2658
W20260709 12:23:01.318015  2655 debug-util.cc:398] Leaking SignalData structure 0x7b08000a0b80 after lost signal to thread 2750
W20260709 12:23:01.318904  2655 debug-util.cc:398] Leaking SignalData structure 0x7b080018da60 after lost signal to thread 2825
W20260709 12:23:01.319707  2655 debug-util.cc:398] Leaking SignalData structure 0x7b080008dc00 after lost signal to thread 2900
I20260709 12:23:01.545603  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:01.551952  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:01.552544  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:02.553821  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:02.560446  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:02.561091  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:03.562276  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:03.568693  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:03.569470  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:04.570698  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:04.576864  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:04.577435  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:05.578812  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:05.585196  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:05.585942  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:06.586983  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:06.592505  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:06.593019  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:07.649431  2067 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:23:07.650276  2067 tablet_server.cc:179] TabletServer@127.2.4.193:0 shutting down...
W20260709 12:23:07.673341  2842 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
W20260709 12:23:07.683411  2842 consensus_peers.cc:597] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e -> Peer 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425): Couldn't send request to peer 51c0e412fd9c4909bb014ae370587bd2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:07.705585  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:07.706924  2067 tablet_replica.cc:333] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2: stopping tablet replica
I20260709 12:23:07.708033  2067 raft_consensus.cc:2243] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.708811  2067 raft_consensus.cc:2272] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.711813  2067 tablet_replica.cc:333] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2: stopping tablet replica
I20260709 12:23:07.712534  2067 raft_consensus.cc:2243] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:23:07.713698  2067 raft_consensus.cc:2272] T 4473c73827fc420a9222f7b3947ba58b P 51c0e412fd9c4909bb014ae370587bd2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.716128  2067 tablet_replica.cc:333] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2: stopping tablet replica
I20260709 12:23:07.716852  2067 raft_consensus.cc:2243] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:07.718007  2067 raft_consensus.cc:2272] T 64d4799f52d9401c8c86306eeb1a3623 P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.720474  2067 tablet_replica.cc:333] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2: stopping tablet replica
I20260709 12:23:07.721149  2067 raft_consensus.cc:2243] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:07.722313  2067 raft_consensus.cc:2272] T 553040872de34092969954cb33c7de3b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.724812  2067 tablet_replica.cc:333] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2: stopping tablet replica
I20260709 12:23:07.725462  2067 raft_consensus.cc:2243] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:07.726835  2067 raft_consensus.cc:2272] T 53e4b64c0177467887dee4d5aa6e3a44 P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.729018  2067 tablet_replica.cc:333] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2: stopping tablet replica
I20260709 12:23:07.729717  2067 raft_consensus.cc:2243] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.730232  2067 raft_consensus.cc:2272] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.732343  2067 tablet_replica.cc:333] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2: stopping tablet replica
I20260709 12:23:07.732950  2067 raft_consensus.cc:2243] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.733556  2067 raft_consensus.cc:2272] T ad190a87809b4ea7ac8d72146be24a61 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.735656  2067 tablet_replica.cc:333] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2: stopping tablet replica
I20260709 12:23:07.736233  2067 raft_consensus.cc:2243] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.736824  2067 raft_consensus.cc:2272] T aae17ace8aad40fe8ec2b304a4a97cb5 P 51c0e412fd9c4909bb014ae370587bd2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.739004  2067 tablet_replica.cc:333] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2: stopping tablet replica
I20260709 12:23:07.739537  2067 raft_consensus.cc:2243] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.740041  2067 raft_consensus.cc:2272] T c45c2ae497034799b8ddab73de61c8dc P 51c0e412fd9c4909bb014ae370587bd2 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.742036  2067 tablet_replica.cc:333] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2: stopping tablet replica
I20260709 12:23:07.742604  2067 raft_consensus.cc:2243] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.743137  2067 raft_consensus.cc:2272] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 51c0e412fd9c4909bb014ae370587bd2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.780642  2067 tablet_server.cc:196] TabletServer@127.2.4.193:0 shutdown complete.
W20260709 12:23:07.782277  2767 consensus_peers.cc:597] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 -> Peer 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425): Couldn't send request to peer 51c0e412fd9c4909bb014ae370587bd2. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:35425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:07.795990  2842 consensus_peers.cc:597] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e -> Peer 51c0e412fd9c4909bb014ae370587bd2 (127.2.4.193:35425): Couldn't send request to peer 51c0e412fd9c4909bb014ae370587bd2. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:35425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:07.811260  2067 tablet_server.cc:179] TabletServer@127.2.4.194:0 shutting down...
I20260709 12:23:07.843890  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:07.845048  2067 tablet_replica.cc:333] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8: stopping tablet replica
I20260709 12:23:07.846071  2067 raft_consensus.cc:2243] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.846721  2067 raft_consensus.cc:2272] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.848865  2067 tablet_replica.cc:333] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8: stopping tablet replica
I20260709 12:23:07.849515  2067 raft_consensus.cc:2243] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.850033  2067 raft_consensus.cc:2272] T 553040872de34092969954cb33c7de3b P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.851900  2067 tablet_replica.cc:333] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8: stopping tablet replica
I20260709 12:23:07.852430  2067 raft_consensus.cc:2243] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.853096  2067 raft_consensus.cc:2272] T 4473c73827fc420a9222f7b3947ba58b P 697f4bdcd0944ad58ce468159b075fe8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.855185  2067 tablet_replica.cc:333] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8: stopping tablet replica
I20260709 12:23:07.855765  2067 raft_consensus.cc:2243] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.856277  2067 raft_consensus.cc:2272] T 64d4799f52d9401c8c86306eeb1a3623 P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.858460  2067 tablet_replica.cc:333] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8: stopping tablet replica
I20260709 12:23:07.858994  2067 raft_consensus.cc:2243] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:07.860004  2067 raft_consensus.cc:2272] T aae17ace8aad40fe8ec2b304a4a97cb5 P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.862036  2067 tablet_replica.cc:333] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8: stopping tablet replica
I20260709 12:23:07.862596  2067 raft_consensus.cc:2243] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.863142  2067 raft_consensus.cc:2272] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.865141  2067 tablet_replica.cc:333] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8: stopping tablet replica
I20260709 12:23:07.865727  2067 raft_consensus.cc:2243] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:23:07.866636  2067 raft_consensus.cc:2272] T c45c2ae497034799b8ddab73de61c8dc P 697f4bdcd0944ad58ce468159b075fe8 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.869067  2067 tablet_replica.cc:333] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8: stopping tablet replica
I20260709 12:23:07.869577  2067 raft_consensus.cc:2243] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:07.870577  2067 raft_consensus.cc:2272] T ad190a87809b4ea7ac8d72146be24a61 P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.872650  2067 tablet_replica.cc:333] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8: stopping tablet replica
I20260709 12:23:07.873168  2067 raft_consensus.cc:2243] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.873785  2067 raft_consensus.cc:2272] T 53e4b64c0177467887dee4d5aa6e3a44 P 697f4bdcd0944ad58ce468159b075fe8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.875694  2067 tablet_replica.cc:333] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8: stopping tablet replica
I20260709 12:23:07.876201  2067 raft_consensus.cc:2243] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.876904  2067 raft_consensus.cc:2272] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 697f4bdcd0944ad58ce468159b075fe8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.909826  2067 tablet_server.cc:196] TabletServer@127.2.4.194:0 shutdown complete.
I20260709 12:23:07.932335  2067 tablet_server.cc:179] TabletServer@127.2.4.195:0 shutting down...
I20260709 12:23:07.967303  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:07.968302  2067 tablet_replica.cc:333] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e: stopping tablet replica
I20260709 12:23:07.969460  2067 raft_consensus.cc:2243] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.970305  2067 raft_consensus.cc:2272] T 4473c73827fc420a9222f7b3947ba58b P 9bff7abdc5314c13a9534adf58b5002e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.972831  2067 tablet_replica.cc:333] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e: stopping tablet replica
I20260709 12:23:07.973482  2067 raft_consensus.cc:2243] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:07.974817  2067 raft_consensus.cc:2272] T b7b0e8a1a9224999bd5bcfc28b4c4bb8 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.977066  2067 tablet_replica.cc:333] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e: stopping tablet replica
I20260709 12:23:07.977679  2067 raft_consensus.cc:2243] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.978227  2067 raft_consensus.cc:2272] T c45c2ae497034799b8ddab73de61c8dc P 9bff7abdc5314c13a9534adf58b5002e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.980234  2067 tablet_replica.cc:333] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e: stopping tablet replica
I20260709 12:23:07.980821  2067 raft_consensus.cc:2243] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.981357  2067 raft_consensus.cc:2272] T aae17ace8aad40fe8ec2b304a4a97cb5 P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.983331  2067 tablet_replica.cc:333] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e: stopping tablet replica
I20260709 12:23:07.983922  2067 raft_consensus.cc:2243] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.984467  2067 raft_consensus.cc:2272] T 553040872de34092969954cb33c7de3b P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.986721  2067 tablet_replica.cc:333] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e: stopping tablet replica
I20260709 12:23:07.987326  2067 raft_consensus.cc:2243] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.987829  2067 raft_consensus.cc:2272] T ad190a87809b4ea7ac8d72146be24a61 P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.989953  2067 tablet_replica.cc:333] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e: stopping tablet replica
I20260709 12:23:07.990494  2067 raft_consensus.cc:2243] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.991021  2067 raft_consensus.cc:2272] T 64d4799f52d9401c8c86306eeb1a3623 P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.993038  2067 tablet_replica.cc:333] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e: stopping tablet replica
I20260709 12:23:07.993736  2067 raft_consensus.cc:2243] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:07.994697  2067 raft_consensus.cc:2272] T 4ca5ce11aa9a430a8a06e20205ba2b1b P 9bff7abdc5314c13a9534adf58b5002e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.996639  2067 tablet_replica.cc:333] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e: stopping tablet replica
I20260709 12:23:07.997169  2067 raft_consensus.cc:2243] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:07.997776  2067 raft_consensus.cc:2272] T 53e4b64c0177467887dee4d5aa6e3a44 P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:07.999692  2067 tablet_replica.cc:333] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e: stopping tablet replica
I20260709 12:23:08.000206  2067 raft_consensus.cc:2243] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:08.001169  2067 raft_consensus.cc:2272] T 0bb8c0788a7d4d0c8e19fbd44da9fbbc P 9bff7abdc5314c13a9534adf58b5002e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:08.036279  2067 tablet_server.cc:196] TabletServer@127.2.4.195:0 shutdown complete.
I20260709 12:23:08.060840  2067 master.cc:562] Master@127.2.4.254:38309 shutting down...
I20260709 12:23:08.091625  2067 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:08.092530  2067 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:08.092911  2067 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7cb8e5219c7f47ad849710af4312160e: stopping tablet replica
I20260709 12:23:08.133597  2067 master.cc:584] Master@127.2.4.254:38309 shutdown complete.
51c0e412fd9c4909bb014ae370587bd2  5
697f4bdcd0944ad58ce468159b075fe8  4
9bff7abdc5314c13a9534adf58b5002e  1
51c0e412fd9c4909bb014ae370587bd2  3
697f4bdcd0944ad58ce468159b075fe8  4
9bff7abdc5314c13a9534adf58b5002e  3
51c0e412fd9c4909bb014ae370587bd2  8
697f4bdcd0944ad58ce468159b075fe8  1
9bff7abdc5314c13a9534adf58b5002e  1
51c0e412fd9c4909bb014ae370587bd2  4
697f4bdcd0944ad58ce468159b075fe8  3
9bff7abdc5314c13a9534adf58b5002e  3
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (29137 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:23:08.188432  2067 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.4.254:39037
I20260709 12:23:08.190037  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:08.196946  3123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:08.197182  3122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:08.197481  3125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:08.198729  2067 server_base.cc:1061] running on GCE node
I20260709 12:23:08.199920  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:08.200150  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:08.200326  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599788200304 us; error 0 us; skew 500 ppm
I20260709 12:23:08.200915  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:08.203936  2067 webserver.cc:533] Webserver started at http://127.2.4.254:33283/ using document root <none> and password file <none>
I20260709 12:23:08.204581  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:08.204838  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:08.205156  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:08.207275  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/master-0-root/instance:
uuid: "6777153c831e452da47938f652649392"
format_stamp: "Formatted at 2026-07-09 12:23:08 on dist-test-slave-4qvp"
I20260709 12:23:08.212626  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 12:23:08.217163  3130 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:08.218339  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:23:08.218684  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/master-0-root
uuid: "6777153c831e452da47938f652649392"
format_stamp: "Formatted at 2026-07-09 12:23:08 on dist-test-slave-4qvp"
I20260709 12:23:08.218988  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:08.233762  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:08.235167  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:08.283841  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.254:39037
I20260709 12:23:08.283914  3181 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.254:39037 every 8 connection(s)
I20260709 12:23:08.288842  3182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:08.299544  3182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392: Bootstrap starting.
I20260709 12:23:08.304630  3182 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.309603  3182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392: No bootstrap required, opened a new log
I20260709 12:23:08.311961  3182 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6777153c831e452da47938f652649392" member_type: VOTER }
I20260709 12:23:08.312415  3182 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.312690  3182 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6777153c831e452da47938f652649392, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.313342  3182 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6777153c831e452da47938f652649392" member_type: VOTER }
I20260709 12:23:08.314026  3182 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:23:08.314292  3182 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:23:08.314599  3182 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.319327  3182 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6777153c831e452da47938f652649392" member_type: VOTER }
I20260709 12:23:08.319950  3182 leader_election.cc:304] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6777153c831e452da47938f652649392; no voters: 
I20260709 12:23:08.321489  3182 leader_election.cc:290] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:23:08.322085  3185 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:08.323741  3185 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [term 1 LEADER]: Becoming Leader. State: Replica: 6777153c831e452da47938f652649392, State: Running, Role: LEADER
I20260709 12:23:08.324714  3182 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:23:08.324692  3185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6777153c831e452da47938f652649392" member_type: VOTER }
I20260709 12:23:08.330897  3187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6777153c831e452da47938f652649392. Latest consensus state: current_term: 1 leader_uuid: "6777153c831e452da47938f652649392" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6777153c831e452da47938f652649392" member_type: VOTER } }
I20260709 12:23:08.332228  3186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6777153c831e452da47938f652649392" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6777153c831e452da47938f652649392" member_type: VOTER } }
I20260709 12:23:08.332785  3187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:08.332837  3186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:08.335877  3194 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:23:08.341825  3194 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:23:08.342940  2067 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:23:08.352176  3194 catalog_manager.cc:1383] Generated new cluster ID: d4319d9f29eb4df0bbe791a75a9d54af
I20260709 12:23:08.352557  3194 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:23:08.375138  3194 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:23:08.376542  3194 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:23:08.395315  3194 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392: Generated new TSK 0
I20260709 12:23:08.396111  3194 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:23:08.410207  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:08.418160  3203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:08.418562  3204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:08.420591  3206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:08.421440  2067 server_base.cc:1061] running on GCE node
I20260709 12:23:08.422701  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:08.422920  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:08.423228  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599788423183 us; error 0 us; skew 500 ppm
I20260709 12:23:08.423924  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:08.427064  2067 webserver.cc:533] Webserver started at http://127.2.4.193:33573/ using document root <none> and password file <none>
I20260709 12:23:08.427659  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:08.427840  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:08.428184  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:08.429440  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/instance:
uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
format_stamp: "Formatted at 2026-07-09 12:23:08 on dist-test-slave-4qvp"
I20260709 12:23:08.434682  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.002s
I20260709 12:23:08.438984  3211 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:08.439970  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:23:08.440426  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root
uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
format_stamp: "Formatted at 2026-07-09 12:23:08 on dist-test-slave-4qvp"
I20260709 12:23:08.440987  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:08.454286  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:08.455812  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:08.457680  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:08.460552  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:08.460803  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:08.461079  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:08.461274  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:08.518810  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.193:37779
I20260709 12:23:08.518965  3273 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.193:37779 every 8 connection(s)
I20260709 12:23:08.524364  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:08.534724  3278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:08.537109  3279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:08.538970  3281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:08.539667  2067 server_base.cc:1061] running on GCE node
I20260709 12:23:08.541702  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:08.542009  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:08.542255  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599788542233 us; error 0 us; skew 500 ppm
I20260709 12:23:08.543154  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:08.545696  3274 heartbeater.cc:344] Connected to a master server at 127.2.4.254:39037
I20260709 12:23:08.546551  3274 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:08.546521  2067 webserver.cc:533] Webserver started at http://127.2.4.194:32889/ using document root <none> and password file <none>
I20260709 12:23:08.547397  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:08.547667  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:08.547681  3274 heartbeater.cc:507] Master 127.2.4.254:39037 requested a full tablet report, sending...
I20260709 12:23:08.548125  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:08.549477  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/instance:
uuid: "8525f77f43714a58bf158d872c1db3e4"
format_stamp: "Formatted at 2026-07-09 12:23:08 on dist-test-slave-4qvp"
I20260709 12:23:08.550670  3147 ts_manager.cc:194] Registered new tserver with Master: 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:08.553153  3147 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54334
I20260709 12:23:08.556221  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:08.560536  3286 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:08.561532  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:23:08.562204  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root
uuid: "8525f77f43714a58bf158d872c1db3e4"
format_stamp: "Formatted at 2026-07-09 12:23:08 on dist-test-slave-4qvp"
I20260709 12:23:08.562515  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:08.578583  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:08.580266  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:08.582180  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:08.585222  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:08.585530  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:08.585835  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:08.586016  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:08.633718  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.194:33649
I20260709 12:23:08.633937  3348 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.194:33649 every 8 connection(s)
I20260709 12:23:08.639034  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:08.649058  3353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:08.648969  3352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:08.653029  2067 server_base.cc:1061] running on GCE node
W20260709 12:23:08.652976  3355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:08.654968  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:08.655298  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:08.655484  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599788655461 us; error 0 us; skew 500 ppm
I20260709 12:23:08.656217  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:08.656780  3349 heartbeater.cc:344] Connected to a master server at 127.2.4.254:39037
I20260709 12:23:08.657431  3349 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:08.658672  3349 heartbeater.cc:507] Master 127.2.4.254:39037 requested a full tablet report, sending...
I20260709 12:23:08.659969  2067 webserver.cc:533] Webserver started at http://127.2.4.195:32773/ using document root <none> and password file <none>
I20260709 12:23:08.660707  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:08.660980  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:08.661310  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:08.661424  3147 ts_manager.cc:194] Registered new tserver with Master: 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:08.663064  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/instance:
uuid: "c93989d3dd634ef585d77b03354121a4"
format_stamp: "Formatted at 2026-07-09 12:23:08 on dist-test-slave-4qvp"
I20260709 12:23:08.663484  3147 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54338
I20260709 12:23:08.668831  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.003s
I20260709 12:23:08.673177  3360 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:08.674265  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:23:08.674652  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root
uuid: "c93989d3dd634ef585d77b03354121a4"
format_stamp: "Formatted at 2026-07-09 12:23:08 on dist-test-slave-4qvp"
I20260709 12:23:08.675014  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:08.688124  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:08.689719  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:08.691423  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:08.694092  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:08.694346  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:08.694640  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:08.694808  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:08.746733  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.195:41453
I20260709 12:23:08.746850  3422 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.195:41453 every 8 connection(s)
I20260709 12:23:08.764595  3423 heartbeater.cc:344] Connected to a master server at 127.2.4.254:39037
I20260709 12:23:08.765403  3423 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:08.766444  3423 heartbeater.cc:507] Master 127.2.4.254:39037 requested a full tablet report, sending...
I20260709 12:23:08.769104  3147 ts_manager.cc:194] Registered new tserver with Master: c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:08.769569  2067 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019136531s
I20260709 12:23:08.770859  3147 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54346
I20260709 12:23:08.803212  3147 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54352:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:23:08.810828  3147 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:23:09.049953  3313 tablet_service.cc:1511] Processing CreateTablet for tablet 34c56ebe809a4c09b75ee18faf980d8c (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:23:09.051542  3313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c56ebe809a4c09b75ee18faf980d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.051677  3386 tablet_service.cc:1511] Processing CreateTablet for tablet 076c82502ea24caf81503a8f13fadf97 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:23:09.051834  3314 tablet_service.cc:1511] Processing CreateTablet for tablet a9cc3d1278674852b3b6c8c39f4f1b59 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:23:09.053167  3386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 076c82502ea24caf81503a8f13fadf97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.053423  3233 tablet_service.cc:1511] Processing CreateTablet for tablet af3368aec5b1412797545abe458a95fe (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:23:09.053514  3312 tablet_service.cc:1511] Processing CreateTablet for tablet 076c82502ea24caf81503a8f13fadf97 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:23:09.054862  3233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af3368aec5b1412797545abe458a95fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.056151  3309 tablet_service.cc:1511] Processing CreateTablet for tablet d639c2cd28f9418682dc27d0b0ba86b2 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:23:09.057410  3239 tablet_service.cc:1511] Processing CreateTablet for tablet a9cc3d1278674852b3b6c8c39f4f1b59 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:23:09.056151  3311 tablet_service.cc:1511] Processing CreateTablet for tablet dbb481337ada4c83945e213fb117ee7f (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:23:09.058995  3388 tablet_service.cc:1511] Processing CreateTablet for tablet a9cc3d1278674852b3b6c8c39f4f1b59 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:23:09.059306  3385 tablet_service.cc:1511] Processing CreateTablet for tablet dbb481337ada4c83945e213fb117ee7f (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:23:09.053542  3314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9cc3d1278674852b3b6c8c39f4f1b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.060767  3237 tablet_service.cc:1511] Processing CreateTablet for tablet 076c82502ea24caf81503a8f13fadf97 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:23:09.061781  3388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9cc3d1278674852b3b6c8c39f4f1b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.054859  3310 tablet_service.cc:1511] Processing CreateTablet for tablet 1b0ee883b3374a4eb86de5773524325b (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:23:09.062672  3385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbb481337ada4c83945e213fb117ee7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.063052  3310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b0ee883b3374a4eb86de5773524325b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.061247  3387 tablet_service.cc:1511] Processing CreateTablet for tablet 34c56ebe809a4c09b75ee18faf980d8c (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:23:09.064975  3235 tablet_service.cc:1511] Processing CreateTablet for tablet 1b0ee883b3374a4eb86de5773524325b (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:23:09.065753  3387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c56ebe809a4c09b75ee18faf980d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.066020  3309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d639c2cd28f9418682dc27d0b0ba86b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.067135  3234 tablet_service.cc:1511] Processing CreateTablet for tablet d639c2cd28f9418682dc27d0b0ba86b2 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:23:09.063021  3236 tablet_service.cc:1511] Processing CreateTablet for tablet dbb481337ada4c83945e213fb117ee7f (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:23:09.058943  3239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9cc3d1278674852b3b6c8c39f4f1b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.067503  3384 tablet_service.cc:1511] Processing CreateTablet for tablet 1b0ee883b3374a4eb86de5773524325b (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:23:09.068575  3311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbb481337ada4c83945e213fb117ee7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.069259  3384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b0ee883b3374a4eb86de5773524325b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.071291  3238 tablet_service.cc:1511] Processing CreateTablet for tablet 34c56ebe809a4c09b75ee18faf980d8c (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:23:09.073221  3383 tablet_service.cc:1511] Processing CreateTablet for tablet d639c2cd28f9418682dc27d0b0ba86b2 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:23:09.075577  3383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d639c2cd28f9418682dc27d0b0ba86b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.075960  3237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 076c82502ea24caf81503a8f13fadf97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.078075  3312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 076c82502ea24caf81503a8f13fadf97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.080642  3236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbb481337ada4c83945e213fb117ee7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.090471  3234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d639c2cd28f9418682dc27d0b0ba86b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.092666  3308 tablet_service.cc:1511] Processing CreateTablet for tablet af3368aec5b1412797545abe458a95fe (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:23:09.094404  3308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af3368aec5b1412797545abe458a95fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.094019  3382 tablet_service.cc:1511] Processing CreateTablet for tablet af3368aec5b1412797545abe458a95fe (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:23:09.095660  3382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af3368aec5b1412797545abe458a95fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.099308  3232 tablet_service.cc:1511] Processing CreateTablet for tablet a2be382cd47e41839958b59bc8e3c40a (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:23:09.100896  3232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2be382cd47e41839958b59bc8e3c40a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.102039  3238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c56ebe809a4c09b75ee18faf980d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.106011  3235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b0ee883b3374a4eb86de5773524325b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.119045  3307 tablet_service.cc:1511] Processing CreateTablet for tablet a2be382cd47e41839958b59bc8e3c40a (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:23:09.120698  3307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2be382cd47e41839958b59bc8e3c40a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.121227  3381 tablet_service.cc:1511] Processing CreateTablet for tablet a2be382cd47e41839958b59bc8e3c40a (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:23:09.122700  3381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2be382cd47e41839958b59bc8e3c40a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.124971  3306 tablet_service.cc:1511] Processing CreateTablet for tablet 84e58c891d3b452eb575499063d5d34f (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:23:09.127280  3306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84e58c891d3b452eb575499063d5d34f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.127058  3231 tablet_service.cc:1511] Processing CreateTablet for tablet 84e58c891d3b452eb575499063d5d34f (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:23:09.128597  3231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84e58c891d3b452eb575499063d5d34f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.171599  3380 tablet_service.cc:1511] Processing CreateTablet for tablet 84e58c891d3b452eb575499063d5d34f (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:23:09.177565  3439 tablet_bootstrap.cc:492] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:09.185086  3439 tablet_bootstrap.cc:654] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:09.195184  3380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84e58c891d3b452eb575499063d5d34f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.198650  3239 tablet_service.cc:1511] Processing CreateTablet for tablet bfb1e1f3b50843ffbc685b3a019ea8cb (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:23:09.214010  3440 tablet_bootstrap.cc:492] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:09.209743  3312 tablet_service.cc:1511] Processing CreateTablet for tablet bfb1e1f3b50843ffbc685b3a019ea8cb (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:23:09.215329  3312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb1e1f3b50843ffbc685b3a019ea8cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.228935  3383 tablet_service.cc:1511] Processing CreateTablet for tablet bfb1e1f3b50843ffbc685b3a019ea8cb (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:23:09.231019  3383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb1e1f3b50843ffbc685b3a019ea8cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.231468  3237 tablet_service.cc:1511] Processing CreateTablet for tablet 3af545b01cb24f1cacb0056f804a7692 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:23:09.234201  3313 tablet_service.cc:1511] Processing CreateTablet for tablet 3af545b01cb24f1cacb0056f804a7692 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:23:09.235600  3313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af545b01cb24f1cacb0056f804a7692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.235441  3233 tablet_service.cc:1511] Processing CreateTablet for tablet 118e1f84035044e1b2df505c59a79fdb (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:23:09.238132  3438 tablet_bootstrap.cc:492] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:09.241214  3305 tablet_service.cc:1511] Processing CreateTablet for tablet 118e1f84035044e1b2df505c59a79fdb (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:23:09.241986  3440 tablet_bootstrap.cc:654] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:09.243613  3381 tablet_service.cc:1511] Processing CreateTablet for tablet 118e1f84035044e1b2df505c59a79fdb (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:23:09.244951  3381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 118e1f84035044e1b2df505c59a79fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.248430  3238 tablet_service.cc:1511] Processing CreateTablet for tablet fd6cee8d33de46388ec38d683a789e94 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:23:09.250070  3305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 118e1f84035044e1b2df505c59a79fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.250623  3238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd6cee8d33de46388ec38d683a789e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.251343  3237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af545b01cb24f1cacb0056f804a7692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.254117  3233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 118e1f84035044e1b2df505c59a79fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.256354  3388 tablet_service.cc:1511] Processing CreateTablet for tablet fd6cee8d33de46388ec38d683a789e94 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:23:09.258076  3388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd6cee8d33de46388ec38d683a789e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.258983  3235 tablet_service.cc:1511] Processing CreateTablet for tablet 572d97394c304e81affa5ee75b52de28 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:23:09.257246  3387 tablet_service.cc:1511] Processing CreateTablet for tablet 3af545b01cb24f1cacb0056f804a7692 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:23:09.273541  3384 tablet_service.cc:1511] Processing CreateTablet for tablet 572d97394c304e81affa5ee75b52de28 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:23:09.274641  3314 tablet_service.cc:1511] Processing CreateTablet for tablet fd6cee8d33de46388ec38d683a789e94 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:23:09.270840  3306 tablet_service.cc:1511] Processing CreateTablet for tablet 572d97394c304e81affa5ee75b52de28 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:23:09.275691  3232 tablet_service.cc:1511] Processing CreateTablet for tablet 9253ca0fa41f43a7ac13ff17879c2ed3 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:23:09.278451  3310 tablet_service.cc:1511] Processing CreateTablet for tablet 9253ca0fa41f43a7ac13ff17879c2ed3 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:23:09.285844  3239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb1e1f3b50843ffbc685b3a019ea8cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.286479  3382 tablet_service.cc:1511] Processing CreateTablet for tablet 9253ca0fa41f43a7ac13ff17879c2ed3 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:23:09.286489  3310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9253ca0fa41f43a7ac13ff17879c2ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.288625  3306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 572d97394c304e81affa5ee75b52de28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.289954  3236 tablet_service.cc:1511] Processing CreateTablet for tablet 054906db16c94967bdef7c9dd5a47ea3 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:23:09.293968  3314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd6cee8d33de46388ec38d683a789e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.296804  3382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9253ca0fa41f43a7ac13ff17879c2ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.306171  3232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9253ca0fa41f43a7ac13ff17879c2ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.307139  3235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 572d97394c304e81affa5ee75b52de28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.317103  3387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af545b01cb24f1cacb0056f804a7692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.319885  3384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 572d97394c304e81affa5ee75b52de28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.331167  3308 tablet_service.cc:1511] Processing CreateTablet for tablet 054906db16c94967bdef7c9dd5a47ea3 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:23:09.333196  3308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054906db16c94967bdef7c9dd5a47ea3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:09.338017  3195 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:09.360102  3386 tablet_service.cc:1511] Processing CreateTablet for tablet 054906db16c94967bdef7c9dd5a47ea3 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:23:09.361496  3386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054906db16c94967bdef7c9dd5a47ea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.375401  3440 tablet_bootstrap.cc:492] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:09.375926  3440 ts_tablet_manager.cc:1403] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.162s	user 0.017s	sys 0.038s
I20260709 12:23:09.381265  3440 raft_consensus.cc:359] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:09.382061  3440 raft_consensus.cc:385] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:09.382418  3440 raft_consensus.cc:740] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:09.383412  3236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054906db16c94967bdef7c9dd5a47ea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.386255  3440 consensus_queue.cc:260] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:09.399767  3439 tablet_bootstrap.cc:492] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:09.400332  3439 ts_tablet_manager.cc:1403] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.223s	user 0.014s	sys 0.020s
I20260709 12:23:09.405985  3438 tablet_bootstrap.cc:654] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:09.418306  3233 tablet_service.cc:1511] Processing CreateTablet for tablet 2a558d863a564ddf9aa7c954dffaeafe (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:23:09.419771  3233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a558d863a564ddf9aa7c954dffaeafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.419873  3274 heartbeater.cc:499] Master 127.2.4.254:39037 was elected leader, sending a full tablet report...
I20260709 12:23:09.423449  3305 tablet_service.cc:1511] Processing CreateTablet for tablet 2a558d863a564ddf9aa7c954dffaeafe (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:23:09.424098  3238 tablet_service.cc:1511] Processing CreateTablet for tablet 1d167141acd1450192e10b3c44347860 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:23:09.425005  3305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a558d863a564ddf9aa7c954dffaeafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.425403  3238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d167141acd1450192e10b3c44347860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.406133  3440 ts_tablet_manager.cc:1434] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.030s	user 0.013s	sys 0.001s
I20260709 12:23:09.425275  3439 raft_consensus.cc:359] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:09.427968  3439 raft_consensus.cc:385] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:09.428318  3439 raft_consensus.cc:740] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:09.428089  3383 tablet_service.cc:1511] Processing CreateTablet for tablet 2a558d863a564ddf9aa7c954dffaeafe (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:23:09.429441  3383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a558d863a564ddf9aa7c954dffaeafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.451154  3440 tablet_bootstrap.cc:492] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:09.429246  3439 consensus_queue.cc:260] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:09.457880  3440 tablet_bootstrap.cc:654] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:09.466940  3381 tablet_service.cc:1511] Processing CreateTablet for tablet 1d167141acd1450192e10b3c44347860 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:23:09.468427  3381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d167141acd1450192e10b3c44347860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.469981  3314 tablet_service.cc:1511] Processing CreateTablet for tablet 1d167141acd1450192e10b3c44347860 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:23:09.472054  3438 tablet_bootstrap.cc:492] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:09.645336  3438 ts_tablet_manager.cc:1403] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.408s	user 0.010s	sys 0.008s
I20260709 12:23:09.648375  3438 raft_consensus.cc:359] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } }
I20260709 12:23:09.649108  3438 raft_consensus.cc:385] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:09.649478  3438 raft_consensus.cc:740] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:09.473595  3232 tablet_service.cc:1511] Processing CreateTablet for tablet 2bde98a54e9f47a0b9afe40221bcb336 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:23:09.725203  3232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bde98a54e9f47a0b9afe40221bcb336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.725325  3349 heartbeater.cc:499] Master 127.2.4.254:39037 was elected leader, sending a full tablet report...
I20260709 12:23:09.650745  3314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d167141acd1450192e10b3c44347860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.527141  3423 heartbeater.cc:499] Master 127.2.4.254:39037 was elected leader, sending a full tablet report...
I20260709 12:23:09.476106  3439 ts_tablet_manager.cc:1434] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.075s	user 0.008s	sys 0.000s
I20260709 12:23:09.743378  3439 tablet_bootstrap.cc:492] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:09.532933  3440 tablet_bootstrap.cc:492] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:09.763154  3440 ts_tablet_manager.cc:1403] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.335s	user 0.014s	sys 0.020s
I20260709 12:23:09.527020  3445 raft_consensus.cc:493] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.650714  3438 consensus_queue.cc:260] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } }
I20260709 12:23:09.741256  3305 tablet_service.cc:1511] Processing CreateTablet for tablet 2bde98a54e9f47a0b9afe40221bcb336 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:23:09.769346  3305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bde98a54e9f47a0b9afe40221bcb336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.769184  3440 raft_consensus.cc:359] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } }
I20260709 12:23:09.770021  3440 raft_consensus.cc:385] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:09.770331  3440 raft_consensus.cc:740] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:09.749951  3439 tablet_bootstrap.cc:654] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:09.759902  3443 raft_consensus.cc:493] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.774511  3443 raft_consensus.cc:515] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:09.775420  3383 tablet_service.cc:1511] Processing CreateTablet for tablet 0a5d938460364390a17cada19308b314 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:23:09.777698  3238 tablet_service.cc:1511] Processing CreateTablet for tablet 87fddcaae9194bef90d1047a518e45a6 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:23:09.778802  3443 leader_election.cc:290] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:09.779345  3238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87fddcaae9194bef90d1047a518e45a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.784070  3314 tablet_service.cc:1511] Processing CreateTablet for tablet 0a5d938460364390a17cada19308b314 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:23:09.786115  3314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a5d938460364390a17cada19308b314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.772992  3232 tablet_service.cc:1511] Processing CreateTablet for tablet 0a5d938460364390a17cada19308b314 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:23:09.787446  3232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a5d938460364390a17cada19308b314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.789875  3383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a5d938460364390a17cada19308b314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.791968  3311 tablet_service.cc:1511] Processing CreateTablet for tablet 87fddcaae9194bef90d1047a518e45a6 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:23:09.793859  3311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87fddcaae9194bef90d1047a518e45a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.766069  3445 raft_consensus.cc:515] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:09.777228  3440 consensus_queue.cc:260] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } }
I20260709 12:23:09.763888  3381 tablet_service.cc:1511] Processing CreateTablet for tablet 2bde98a54e9f47a0b9afe40221bcb336 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:23:09.800340  3381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bde98a54e9f47a0b9afe40221bcb336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.799825  3388 tablet_service.cc:1511] Processing CreateTablet for tablet 87fddcaae9194bef90d1047a518e45a6 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:23:09.801852  3388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87fddcaae9194bef90d1047a518e45a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.805979  3445 leader_election.cc:290] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:09.799901  3440 ts_tablet_manager.cc:1434] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 12:23:09.820627  3312 tablet_service.cc:1511] Processing CreateTablet for tablet dfabf95aa4ee4eae966abca58130bd91 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:23:09.822213  3312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfabf95aa4ee4eae966abca58130bd91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.822873  3440 tablet_bootstrap.cc:492] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:09.831894  3384 tablet_service.cc:1511] Processing CreateTablet for tablet dfabf95aa4ee4eae966abca58130bd91 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:23:09.833479  3384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfabf95aa4ee4eae966abca58130bd91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.851820  3233 tablet_service.cc:1511] Processing CreateTablet for tablet dfabf95aa4ee4eae966abca58130bd91 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:23:09.853286  3233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfabf95aa4ee4eae966abca58130bd91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.853356  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
W20260709 12:23:09.858536  3214 leader_election.cc:343] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:09.858358  3310 tablet_service.cc:1511] Processing CreateTablet for tablet b8131bbfd8374661bd9731b4a57468ee (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:23:09.858876  3443 raft_consensus.cc:493] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.859454  3443 raft_consensus.cc:515] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } }
I20260709 12:23:09.859926  3310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8131bbfd8374661bd9731b4a57468ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.860762  3236 tablet_service.cc:1511] Processing CreateTablet for tablet b8131bbfd8374661bd9731b4a57468ee (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:23:09.861999  3443 leader_election.cc:290] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:09.862277  3236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8131bbfd8374661bd9731b4a57468ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.865248  3438 ts_tablet_manager.cc:1434] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.220s	user 0.008s	sys 0.003s
I20260709 12:23:09.866472  3438 tablet_bootstrap.cc:492] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:09.872447  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9cc3d1278674852b3b6c8c39f4f1b59" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:09.875571  3439 tablet_bootstrap.cc:492] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:09.876174  3439 ts_tablet_manager.cc:1403] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.133s	user 0.008s	sys 0.002s
W20260709 12:23:09.876748  3214 leader_election.cc:343] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:09.878158  3382 tablet_service.cc:1511] Processing CreateTablet for tablet b8131bbfd8374661bd9731b4a57468ee (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:23:09.879504  3382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8131bbfd8374661bd9731b4a57468ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.879832  3439 raft_consensus.cc:359] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:09.880700  3439 raft_consensus.cc:385] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:09.881004  3439 raft_consensus.cc:740] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:09.896489  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b0ee883b3374a4eb86de5773524325b" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:09.898388  3362 leader_election.cc:343] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:09.902864  3440 tablet_bootstrap.cc:654] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:09.892297  3439 consensus_queue.cc:260] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:09.909999  3439 ts_tablet_manager.cc:1434] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.001s
I20260709 12:23:09.911092  3439 tablet_bootstrap.cc:492] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:09.917888  3439 tablet_bootstrap.cc:654] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:09.940760  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:09.942384  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9cc3d1278674852b3b6c8c39f4f1b59" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:09.943122  3323 raft_consensus.cc:2468] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:09.943431  3438 tablet_bootstrap.cc:654] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:09.945268  3215 leader_election.cc:343] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:09.945778  3215 leader_election.cc:304] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:09.947484  3443 raft_consensus.cc:2749] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:09.948954  3215 leader_election.cc:304] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: c93989d3dd634ef585d77b03354121a4
I20260709 12:23:09.949795  3440 tablet_bootstrap.cc:492] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:09.950241  3443 raft_consensus.cc:2804] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:09.950345  3440 ts_tablet_manager.cc:1403] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.130s	user 0.012s	sys 0.006s
I20260709 12:23:09.950711  3443 raft_consensus.cc:493] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:09.951108  3443 raft_consensus.cc:3060] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:09.954000  3440 raft_consensus.cc:359] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:09.954854  3440 raft_consensus.cc:385] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:09.955200  3440 raft_consensus.cc:740] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:09.955533  3233 tablet_service.cc:1511] Processing CreateTablet for tablet 5ba9db97635f4d28bf0f3079edf45a77 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:23:09.956046  3440 consensus_queue.cc:260] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:09.957789  3233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ba9db97635f4d28bf0f3079edf45a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.963174  3314 tablet_service.cc:1511] Processing CreateTablet for tablet 5ba9db97635f4d28bf0f3079edf45a77 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:23:09.963976  3388 tablet_service.cc:1511] Processing CreateTablet for tablet 5ba9db97635f4d28bf0f3079edf45a77 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:23:09.964840  3314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ba9db97635f4d28bf0f3079edf45a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.965281  3388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ba9db97635f4d28bf0f3079edf45a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.965464  3236 tablet_service.cc:1511] Processing CreateTablet for tablet d373e8b022274a08b08f00c0057ae3d3 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:23:09.967322  3236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d373e8b022274a08b08f00c0057ae3d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.971932  3232 tablet_service.cc:1511] Processing CreateTablet for tablet caed93336fbb415c82acd602adce7969 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:23:09.973491  3232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caed93336fbb415c82acd602adce7969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.978626  3310 tablet_service.cc:1511] Processing CreateTablet for tablet caed93336fbb415c82acd602adce7969 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:23:09.980901  3310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caed93336fbb415c82acd602adce7969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.982754  3383 tablet_service.cc:1511] Processing CreateTablet for tablet caed93336fbb415c82acd602adce7969 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:23:09.983072  3238 tablet_service.cc:1511] Processing CreateTablet for tablet c4a1156be8e3418790f41cd380136b93 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:23:09.984302  3238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4a1156be8e3418790f41cd380136b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.984661  3384 tablet_service.cc:1511] Processing CreateTablet for tablet d373e8b022274a08b08f00c0057ae3d3 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:23:09.984304  3383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caed93336fbb415c82acd602adce7969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.989734  3312 tablet_service.cc:1511] Processing CreateTablet for tablet c4a1156be8e3418790f41cd380136b93 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:23:09.989866  3311 tablet_service.cc:1511] Processing CreateTablet for tablet d373e8b022274a08b08f00c0057ae3d3 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:23:09.991437  3312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4a1156be8e3418790f41cd380136b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.992344  3384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d373e8b022274a08b08f00c0057ae3d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:09.994342  3381 tablet_service.cc:1511] Processing CreateTablet for tablet c4a1156be8e3418790f41cd380136b93 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:23:09.996306  3381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4a1156be8e3418790f41cd380136b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.017771  3311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d373e8b022274a08b08f00c0057ae3d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.018953  3239 tablet_service.cc:1511] Processing CreateTablet for tablet e8d7a917f21741e9827c30e5433a38e3 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:23:10.020471  3239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8d7a917f21741e9827c30e5433a38e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.021540  3386 tablet_service.cc:1511] Processing CreateTablet for tablet e8d7a917f21741e9827c30e5433a38e3 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:23:10.020494  3305 tablet_service.cc:1511] Processing CreateTablet for tablet e8d7a917f21741e9827c30e5433a38e3 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:23:10.023034  3386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8d7a917f21741e9827c30e5433a38e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.023233  3305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8d7a917f21741e9827c30e5433a38e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.027774  3439 tablet_bootstrap.cc:492] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:10.028354  3439 ts_tablet_manager.cc:1403] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.117s	user 0.015s	sys 0.024s
I20260709 12:23:10.029155  3445 raft_consensus.cc:493] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.029815  3445 raft_consensus.cc:515] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.034192  3445 leader_election.cc:290] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:10.034564  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbb481337ada4c83945e213fb117ee7f" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:10.036563  3362 leader_election.cc:343] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.047832  3439 raft_consensus.cc:359] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.062062  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b0ee883b3374a4eb86de5773524325b" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:10.063925  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbb481337ada4c83945e213fb117ee7f" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
W20260709 12:23:10.066226  3364 leader_election.cc:343] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.066804  3364 leader_election.cc:304] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
W20260709 12:23:10.068228  3364 leader_election.cc:343] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.068715  3364 leader_election.cc:304] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:10.069844  3445 raft_consensus.cc:2749] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:10.070591  3445 raft_consensus.cc:2749] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:10.073053  3438 tablet_bootstrap.cc:492] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:10.073899  3438 ts_tablet_manager.cc:1403] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.208s	user 0.004s	sys 0.013s
I20260709 12:23:10.062249  3443 raft_consensus.cc:515] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } }
I20260709 12:23:10.082505  3438 raft_consensus.cc:359] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.083861  3438 raft_consensus.cc:385] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.084406  3438 raft_consensus.cc:740] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.086158  3438 consensus_queue.cc:260] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.092099  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9cc3d1278674852b3b6c8c39f4f1b59" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:10.103289  3324 raft_consensus.cc:3060] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.092905  3443 leader_election.cc:290] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:10.102180  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9cc3d1278674852b3b6c8c39f4f1b59" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
W20260709 12:23:10.112066  3214 leader_election.cc:343] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.115463  3239 tablet_service.cc:1511] Processing CreateTablet for tablet c9145effc86e4e0db3de0ff579cd939d (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:23:10.117008  3239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9145effc86e4e0db3de0ff579cd939d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.071671  3439 raft_consensus.cc:385] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.126143  3439 raft_consensus.cc:740] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.127287  3439 consensus_queue.cc:260] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.130008  3439 ts_tablet_manager.cc:1434] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.101s	user 0.007s	sys 0.002s
I20260709 12:23:10.135352  3440 ts_tablet_manager.cc:1434] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.185s	user 0.042s	sys 0.033s
I20260709 12:23:10.141822  3439 tablet_bootstrap.cc:492] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:10.150213  3439 tablet_bootstrap.cc:654] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.151841  3314 tablet_service.cc:1511] Processing CreateTablet for tablet c9145effc86e4e0db3de0ff579cd939d (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:23:10.152714  3440 tablet_bootstrap.cc:492] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:10.156945  3445 raft_consensus.cc:493] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.160248  3445 raft_consensus.cc:515] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.162582  3445 leader_election.cc:290] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:10.167441  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2be382cd47e41839958b59bc8e3c40a" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:10.167817  3314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9145effc86e4e0db3de0ff579cd939d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:10.171180  3362 leader_election.cc:343] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:10.170989  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 102784us
I20260709 12:23:10.173257  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2be382cd47e41839958b59bc8e3c40a" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
W20260709 12:23:10.178879  3364 leader_election.cc:343] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.179508  3364 leader_election.cc:304] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:10.181064  3445 raft_consensus.cc:2749] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:10.187252  3386 tablet_service.cc:1511] Processing CreateTablet for tablet c9145effc86e4e0db3de0ff579cd939d (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:23:10.188684  3386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9145effc86e4e0db3de0ff579cd939d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.189174  3236 tablet_service.cc:1511] Processing CreateTablet for tablet 412da949752f49049ec6148896078f29 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:23:10.190712  3236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 412da949752f49049ec6148896078f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.192400  3448 raft_consensus.cc:493] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.192605  3324 raft_consensus.cc:2468] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:10.193027  3448 raft_consensus.cc:515] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.192987  3440 tablet_bootstrap.cc:654] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.201385  3448 leader_election.cc:290] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:10.202741  3215 leader_election.cc:304] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: c93989d3dd634ef585d77b03354121a4
I20260709 12:23:10.212134  3384 tablet_service.cc:1511] Processing CreateTablet for tablet 412da949752f49049ec6148896078f29 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:23:10.213707  3384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 412da949752f49049ec6148896078f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.213305  3461 raft_consensus.cc:2804] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:10.214499  3461 raft_consensus.cc:697] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:10.215822  3461 consensus_queue.cc:237] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } }
I20260709 12:23:10.205917  3438 ts_tablet_manager.cc:1434] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.132s	user 0.010s	sys 0.001s
I20260709 12:23:10.206588  3439 tablet_bootstrap.cc:492] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:10.219834  3439 ts_tablet_manager.cc:1403] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.078s	user 0.013s	sys 0.000s
I20260709 12:23:10.222307  3438 tablet_bootstrap.cc:492] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:10.219990  3311 tablet_service.cc:1511] Processing CreateTablet for tablet 412da949752f49049ec6148896078f29 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:23:10.220373  3233 tablet_service.cc:1511] Processing CreateTablet for tablet 00dbb3c83d794b5db75532d598235d9a (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:23:10.227730  3439 raft_consensus.cc:359] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.227730  3311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 412da949752f49049ec6148896078f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.234357  3233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00dbb3c83d794b5db75532d598235d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.238233  3438 tablet_bootstrap.cc:654] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.230850  3439 raft_consensus.cc:385] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.248497  3310 tablet_service.cc:1511] Processing CreateTablet for tablet 00dbb3c83d794b5db75532d598235d9a (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:23:10.250041  3310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00dbb3c83d794b5db75532d598235d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.251917  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c56ebe809a4c09b75ee18faf980d8c" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:10.254532  3288 leader_election.cc:343] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.265023  3386 tablet_service.cc:1511] Processing CreateTablet for tablet 00dbb3c83d794b5db75532d598235d9a (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:23:10.265025  3239 tablet_service.cc:1511] Processing CreateTablet for tablet 36377e446734493c8f3b1ff53d40d1cc (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:23:10.266525  3440 tablet_bootstrap.cc:492] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:10.266860  3239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36377e446734493c8f3b1ff53d40d1cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.267128  3440 ts_tablet_manager.cc:1403] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.126s	user 0.015s	sys 0.010s
I20260709 12:23:10.247548  3439 raft_consensus.cc:740] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.270586  3439 consensus_queue.cc:260] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.280193  3439 ts_tablet_manager.cc:1434] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.001s
I20260709 12:23:10.281296  3386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00dbb3c83d794b5db75532d598235d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.283249  3440 raft_consensus.cc:359] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.283401  3314 tablet_service.cc:1511] Processing CreateTablet for tablet 36377e446734493c8f3b1ff53d40d1cc (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:23:10.285099  3314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36377e446734493c8f3b1ff53d40d1cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.286197  3232 tablet_service.cc:1511] Processing CreateTablet for tablet d02f86799bc54984b41ede0f7db226e6 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:23:10.287698  3232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d02f86799bc54984b41ede0f7db226e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.287369  3312 tablet_service.cc:1511] Processing CreateTablet for tablet d02f86799bc54984b41ede0f7db226e6 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:23:10.288887  3312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d02f86799bc54984b41ede0f7db226e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.299228  3439 tablet_bootstrap.cc:492] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:10.284992  3384 tablet_service.cc:1511] Processing CreateTablet for tablet 36377e446734493c8f3b1ff53d40d1cc (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:23:10.302754  3440 raft_consensus.cc:385] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.303004  3384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36377e446734493c8f3b1ff53d40d1cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.303216  3440 raft_consensus.cc:740] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.304499  3440 consensus_queue.cc:260] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.310205  3440 ts_tablet_manager.cc:1434] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.001s
I20260709 12:23:10.311125  3438 tablet_bootstrap.cc:492] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:10.311465  3440 tablet_bootstrap.cc:492] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:10.311885  3438 ts_tablet_manager.cc:1403] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.090s	user 0.015s	sys 0.008s
I20260709 12:23:10.315181  3438 raft_consensus.cc:359] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.318125  3146 catalog_manager.cc:5697] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:10.321786  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c56ebe809a4c09b75ee18faf980d8c" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
W20260709 12:23:10.331224  3289 leader_election.cc:343] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.331773  3289 leader_election.cc:304] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:10.333088  3448 raft_consensus.cc:2749] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:10.324898  3383 tablet_service.cc:1511] Processing CreateTablet for tablet d02f86799bc54984b41ede0f7db226e6 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:23:10.334800  3383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d02f86799bc54984b41ede0f7db226e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.327718  3237 tablet_service.cc:1511] Processing CreateTablet for tablet b8f62d0c16794e8c941849d55bb4bb0b (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:23:10.338539  3237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f62d0c16794e8c941849d55bb4bb0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.342327  3381 tablet_service.cc:1511] Processing CreateTablet for tablet b8f62d0c16794e8c941849d55bb4bb0b (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:23:10.343797  3381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f62d0c16794e8c941849d55bb4bb0b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:10.343901  3195 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:10.348637  3440 tablet_bootstrap.cc:654] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.327823  3311 tablet_service.cc:1511] Processing CreateTablet for tablet b8f62d0c16794e8c941849d55bb4bb0b (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:23:10.328356  3439 tablet_bootstrap.cc:654] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.358009  3236 tablet_service.cc:1511] Processing CreateTablet for tablet a7494971ced1402da04932982df157fd (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:23:10.358632  3311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f62d0c16794e8c941849d55bb4bb0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.359004  3438 raft_consensus.cc:385] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.361729  3438 raft_consensus.cc:740] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.362891  3438 consensus_queue.cc:260] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.372892  3438 ts_tablet_manager.cc:1434] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.000s
I20260709 12:23:10.373153  3310 tablet_service.cc:1511] Processing CreateTablet for tablet a7494971ced1402da04932982df157fd (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:23:10.374575  3310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7494971ced1402da04932982df157fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.379114  3236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7494971ced1402da04932982df157fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.381920  3386 tablet_service.cc:1511] Processing CreateTablet for tablet a7494971ced1402da04932982df157fd (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:23:10.383755  3386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7494971ced1402da04932982df157fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.384970  3232 tablet_service.cc:1511] Processing CreateTablet for tablet e2884104b65f4a589e915fd4328b729c (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:23:10.386484  3232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2884104b65f4a589e915fd4328b729c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.379546  3438 tablet_bootstrap.cc:492] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:10.399408  3439 tablet_bootstrap.cc:492] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:10.399957  3439 ts_tablet_manager.cc:1403] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.101s	user 0.007s	sys 0.016s
I20260709 12:23:10.403056  3439 raft_consensus.cc:359] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.403815  3439 raft_consensus.cc:385] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.404106  3439 raft_consensus.cc:740] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.405099  3439 consensus_queue.cc:260] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.411967  3314 tablet_service.cc:1511] Processing CreateTablet for tablet e2884104b65f4a589e915fd4328b729c (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:23:10.413982  3314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2884104b65f4a589e915fd4328b729c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.422459  3448 raft_consensus.cc:493] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.421873  3237 tablet_service.cc:1511] Processing CreateTablet for tablet 3098f015fb154cf490e147a8e71f65b6 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:23:10.423074  3448 raft_consensus.cc:515] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.423647  3382 tablet_service.cc:1511] Processing CreateTablet for tablet e2884104b65f4a589e915fd4328b729c (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:23:10.424454  3237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3098f015fb154cf490e147a8e71f65b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.425105  3382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2884104b65f4a589e915fd4328b729c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.428534  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:10.429251  3398 raft_consensus.cc:2468] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:10.430711  3289 leader_election.cc:304] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:10.427243  3448 leader_election.cc:290] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:10.433316  3448 raft_consensus.cc:2804] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:10.433753  3448 raft_consensus.cc:493] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.434137  3448 raft_consensus.cc:3060] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.433352  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:10.434862  3249 raft_consensus.cc:2468] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:10.435043  3312 tablet_service.cc:1511] Processing CreateTablet for tablet 3098f015fb154cf490e147a8e71f65b6 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:23:10.436488  3312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3098f015fb154cf490e147a8e71f65b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.447144  3233 tablet_service.cc:1511] Processing CreateTablet for tablet 0634624f456740b48b77096645ab6b0b (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:23:10.448645  3233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0634624f456740b48b77096645ab6b0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.449716  3461 raft_consensus.cc:493] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.450205  3461 raft_consensus.cc:515] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.452396  3461 leader_election.cc:290] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:10.454746  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
W20260709 12:23:10.456393  3214 leader_election.cc:343] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.461397  3438 tablet_bootstrap.cc:654] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.462975  3439 ts_tablet_manager.cc:1434] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.063s	user 0.010s	sys 0.000s
I20260709 12:23:10.464294  3439 tablet_bootstrap.cc:492] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:10.471496  3383 tablet_service.cc:1511] Processing CreateTablet for tablet 0634624f456740b48b77096645ab6b0b (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:23:10.472404  3239 tablet_service.cc:1511] Processing CreateTablet for tablet 6c9b725784044c8da903f092febf7c05 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:23:10.473206  3383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0634624f456740b48b77096645ab6b0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.475128  3314 tablet_service.cc:1511] Processing CreateTablet for tablet 0634624f456740b48b77096645ab6b0b (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:23:10.478037  3314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0634624f456740b48b77096645ab6b0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.478317  3239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c9b725784044c8da903f092febf7c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.496265  3305 tablet_service.cc:1511] Processing CreateTablet for tablet 6c9b725784044c8da903f092febf7c05 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:23:10.497126  3384 tablet_service.cc:1511] Processing CreateTablet for tablet 6c9b725784044c8da903f092febf7c05 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:23:10.498021  3305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c9b725784044c8da903f092febf7c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.498428  3384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c9b725784044c8da903f092febf7c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.493335  3440 tablet_bootstrap.cc:492] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:10.502411  3440 ts_tablet_manager.cc:1403] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.191s	user 0.021s	sys 0.012s
I20260709 12:23:10.473425  3381 tablet_service.cc:1511] Processing CreateTablet for tablet 3098f015fb154cf490e147a8e71f65b6 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:23:10.518373  3381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3098f015fb154cf490e147a8e71f65b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.519786  3238 tablet_service.cc:1511] Processing CreateTablet for tablet d6a9849f17fd4313ab9ddbb06589c46f (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:23:10.521209  3238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6a9849f17fd4313ab9ddbb06589c46f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.521554  3438 tablet_bootstrap.cc:492] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:10.522250  3438 ts_tablet_manager.cc:1403] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.143s	user 0.017s	sys 0.016s
I20260709 12:23:10.523353  3440 raft_consensus.cc:359] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.524227  3440 raft_consensus.cc:385] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.524616  3440 raft_consensus.cc:740] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
W20260709 12:23:10.510417  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 79797.6us
I20260709 12:23:10.527316  3308 tablet_service.cc:1511] Processing CreateTablet for tablet d6a9849f17fd4313ab9ddbb06589c46f (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:23:10.528714  3308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6a9849f17fd4313ab9ddbb06589c46f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.530052  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:10.531270  3438 raft_consensus.cc:359] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.532061  3438 raft_consensus.cc:385] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.532384  3438 raft_consensus.cc:740] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.533181  3438 consensus_queue.cc:260] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.533397  3388 tablet_service.cc:1511] Processing CreateTablet for tablet d6a9849f17fd4313ab9ddbb06589c46f (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:23:10.535112  3388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6a9849f17fd4313ab9ddbb06589c46f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:10.536697  3215 leader_election.cc:343] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.537339  3215 leader_election.cc:304] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:10.538664  3461 raft_consensus.cc:2749] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:10.528730  3448 raft_consensus.cc:515] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.543278  3235 tablet_service.cc:1511] Processing CreateTablet for tablet b7048879a65244efa5a140c52d782803 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:23:10.544677  3235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7048879a65244efa5a140c52d782803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.525475  3440 consensus_queue.cc:260] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.548130  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:10.557065  3249 raft_consensus.cc:3060] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.549986  3448 leader_election.cc:290] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:10.551662  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:10.560113  3398 raft_consensus.cc:3060] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.556481  3439 tablet_bootstrap.cc:654] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.560679  3440 ts_tablet_manager.cc:1434] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.001s
I20260709 12:23:10.567596  3448 raft_consensus.cc:493] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.568192  3306 tablet_service.cc:1511] Processing CreateTablet for tablet b7048879a65244efa5a140c52d782803 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:23:10.568243  3448 raft_consensus.cc:515] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.569564  3387 tablet_service.cc:1511] Processing CreateTablet for tablet b7048879a65244efa5a140c52d782803 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:23:10.570026  3306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7048879a65244efa5a140c52d782803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.571019  3387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7048879a65244efa5a140c52d782803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.579267  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b0ee883b3374a4eb86de5773524325b" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:10.580082  3397 raft_consensus.cc:2468] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:10.582268  3289 leader_election.cc:304] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:10.583639  3473 raft_consensus.cc:2804] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:10.584024  3473 raft_consensus.cc:493] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.584383  3473 raft_consensus.cc:3060] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.593470  3461 raft_consensus.cc:493] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.594139  3461 raft_consensus.cc:515] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.595863  3440 tablet_bootstrap.cc:492] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:10.589859  3448 leader_election.cc:290] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:10.598027  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2be382cd47e41839958b59bc8e3c40a" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
W20260709 12:23:10.599831  3215 leader_election.cc:343] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.589563  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b0ee883b3374a4eb86de5773524325b" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:10.601820  3288 leader_election.cc:343] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.604705  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2be382cd47e41839958b59bc8e3c40a" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:10.605520  3397 raft_consensus.cc:2468] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:10.606993  3214 leader_election.cc:304] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:10.607896  3443 raft_consensus.cc:2804] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:10.608275  3443 raft_consensus.cc:493] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.608620  3443 raft_consensus.cc:3060] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.617121  3440 tablet_bootstrap.cc:654] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.617764  3461 leader_election.cc:290] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:10.617285  3443 raft_consensus.cc:515] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.621459  3462 raft_consensus.cc:493] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.622217  3462 raft_consensus.cc:515] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.626292  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3368aec5b1412797545abe458a95fe" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
W20260709 12:23:10.628019  3364 leader_election.cc:343] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.629453  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3368aec5b1412797545abe458a95fe" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:10.631362  3362 leader_election.cc:343] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.632007  3362 leader_election.cc:304] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:10.633172  3445 raft_consensus.cc:2749] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:10.634145  3462 leader_election.cc:290] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:10.637437  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2be382cd47e41839958b59bc8e3c40a" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
W20260709 12:23:10.639467  3215 leader_election.cc:343] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.588500  3438 ts_tablet_manager.cc:1434] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.062s	user 0.002s	sys 0.006s
I20260709 12:23:10.652293  3387 tablet_service.cc:1511] Processing CreateTablet for tablet 3015bb6b71f444dcb17062ac96b48cf5 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
W20260709 12:23:10.653589  3350 tablet.cc:2406] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:23:10.658602  3387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3015bb6b71f444dcb17062ac96b48cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.659806  3443 leader_election.cc:290] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:10.661317  3311 tablet_service.cc:1511] Processing CreateTablet for tablet 3015bb6b71f444dcb17062ac96b48cf5 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:23:10.662777  3311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3015bb6b71f444dcb17062ac96b48cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.664852  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2be382cd47e41839958b59bc8e3c40a" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:10.665591  3397 raft_consensus.cc:3060] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.670814  3236 tablet_service.cc:1511] Processing CreateTablet for tablet 3015bb6b71f444dcb17062ac96b48cf5 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:23:10.672807  3236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3015bb6b71f444dcb17062ac96b48cf5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:10.679798  3214 consensus_peers.cc:597] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Couldn't send request to peer c93989d3dd634ef585d77b03354121a4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:10.643602  3473 raft_consensus.cc:515] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.680575  3461 consensus_queue.cc:1048] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:10.682395  3473 leader_election.cc:290] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:10.637200  3249 raft_consensus.cc:2468] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:10.685556  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b0ee883b3374a4eb86de5773524325b" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:10.686679  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b0ee883b3374a4eb86de5773524325b" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:10.695844  3396 raft_consensus.cc:3060] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:10.700331  3288 leader_election.cc:343] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:10.709313  3345 debug-util.cc:398] Leaking SignalData structure 0x7b0800165800 after lost signal to thread 2070
W20260709 12:23:10.710907  3345 debug-util.cc:398] Leaking SignalData structure 0x7b0800164ee0 after lost signal to thread 3181
I20260709 12:23:10.710752  3288 leader_election.cc:304] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:10.702059  3239 tablet_service.cc:1511] Processing CreateTablet for tablet f85f93dbda424e3aa0102bc19a2ff837 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:23:10.713004  3239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f85f93dbda424e3aa0102bc19a2ff837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.718122  3473 raft_consensus.cc:2804] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:10.723713  3440 tablet_bootstrap.cc:492] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:10.724359  3440 ts_tablet_manager.cc:1403] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.129s	user 0.017s	sys 0.037s
I20260709 12:23:10.725807  3473 raft_consensus.cc:697] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:10.727080  3473 consensus_queue.cc:237] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.728279  3438 tablet_bootstrap.cc:492] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:10.728574  3306 tablet_service.cc:1511] Processing CreateTablet for tablet f85f93dbda424e3aa0102bc19a2ff837 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:23:10.730391  3306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f85f93dbda424e3aa0102bc19a2ff837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.729039  3440 raft_consensus.cc:359] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.731585  3440 raft_consensus.cc:385] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.731945  3440 raft_consensus.cc:740] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.733047  3384 tablet_service.cc:1511] Processing CreateTablet for tablet f85f93dbda424e3aa0102bc19a2ff837 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:23:10.734671  3384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f85f93dbda424e3aa0102bc19a2ff837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.734715  3238 tablet_service.cc:1511] Processing CreateTablet for tablet 6b70b620cf8243f4b39cf03eaae44881 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:23:10.728274  3439 tablet_bootstrap.cc:492] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:10.736508  3439 ts_tablet_manager.cc:1403] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.272s	user 0.038s	sys 0.037s
I20260709 12:23:10.736794  3238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b70b620cf8243f4b39cf03eaae44881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.740223  3440 consensus_queue.cc:260] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.745186  3312 tablet_service.cc:1511] Processing CreateTablet for tablet 6b70b620cf8243f4b39cf03eaae44881 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:23:10.747100  3312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b70b620cf8243f4b39cf03eaae44881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.690182  3398 raft_consensus.cc:2468] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:10.759203  3234 tablet_service.cc:1511] Processing CreateTablet for tablet 1f76214f21f642889fe174df15bb7047 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:23:10.760737  3234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f76214f21f642889fe174df15bb7047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.763186  3382 tablet_service.cc:1511] Processing CreateTablet for tablet 6b70b620cf8243f4b39cf03eaae44881 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:23:10.777949  3310 tablet_service.cc:1511] Processing CreateTablet for tablet 1f76214f21f642889fe174df15bb7047 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:23:10.779357  3310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f76214f21f642889fe174df15bb7047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.780552  3396 raft_consensus.cc:2468] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:10.775532  3397 raft_consensus.cc:2468] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:10.783406  3214 leader_election.cc:304] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:10.783854  3289 leader_election.cc:304] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:10.785950  3443 raft_consensus.cc:2804] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:10.786650  3443 raft_consensus.cc:697] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:10.787765  3388 tablet_service.cc:1511] Processing CreateTablet for tablet 1f76214f21f642889fe174df15bb7047 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:23:10.788326  3381 tablet_service.cc:1511] Processing CreateTablet for tablet 7a6d2799afd04cbe9bfaa74d467e66cb (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:23:10.787842  3443 consensus_queue.cc:237] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.791000  3448 raft_consensus.cc:2804] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:10.800594  3231 tablet_service.cc:1511] Processing CreateTablet for tablet 7a6d2799afd04cbe9bfaa74d467e66cb (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:23:10.802827  3233 tablet_service.cc:1511] Processing CreateTablet for tablet 9f37527cd3a34753b502172f81b808e0 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:23:10.794643  3386 tablet_service.cc:1511] Processing CreateTablet for tablet 16f6e94aa295499cb8358daafcb934b6 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:23:10.803603  3385 tablet_service.cc:1511] Processing CreateTablet for tablet f8a494da53f44c1d9780c046a155769a (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:23:10.803432  3380 tablet_service.cc:1511] Processing CreateTablet for tablet 9f37527cd3a34753b502172f81b808e0 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:23:10.794813  3230 tablet_service.cc:1511] Processing CreateTablet for tablet 16f6e94aa295499cb8358daafcb934b6 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:23:10.811465  3309 tablet_service.cc:1511] Processing CreateTablet for tablet 9f37527cd3a34753b502172f81b808e0 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:23:10.811976  3379 tablet_service.cc:1511] Processing CreateTablet for tablet 73563dc7466d4636a64ebd4d363935b1 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:23:10.814324  3307 tablet_service.cc:1511] Processing CreateTablet for tablet 16f6e94aa295499cb8358daafcb934b6 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:23:10.793080  3314 tablet_service.cc:1511] Processing CreateTablet for tablet 1f3795979cf042008b3447b47c75de09 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:23:10.796702  3313 tablet_service.cc:1511] Processing CreateTablet for tablet 7a6d2799afd04cbe9bfaa74d467e66cb (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:23:10.816773  3231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a6d2799afd04cbe9bfaa74d467e66cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.819120  3439 raft_consensus.cc:359] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.821573  3382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b70b620cf8243f4b39cf03eaae44881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.822225  3438 tablet_bootstrap.cc:654] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.830631  3383 tablet_service.cc:1511] Processing CreateTablet for tablet 1f3795979cf042008b3447b47c75de09 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:23:10.842780  3440 ts_tablet_manager.cc:1434] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.118s	user 0.011s	sys 0.000s
I20260709 12:23:10.874205  3439 raft_consensus.cc:385] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.880923  3439 raft_consensus.cc:740] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.882268  3439 consensus_queue.cc:260] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:10.884207  3230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16f6e94aa295499cb8358daafcb934b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.884624  3440 tablet_bootstrap.cc:492] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:10.886642  3439 ts_tablet_manager.cc:1434] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.150s	user 0.002s	sys 0.009s
I20260709 12:23:10.889726  3388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f76214f21f642889fe174df15bb7047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.890475  3380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f37527cd3a34753b502172f81b808e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.896509  3386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16f6e94aa295499cb8358daafcb934b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.910432  3233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f37527cd3a34753b502172f81b808e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.859579  3232 tablet_service.cc:1511] Processing CreateTablet for tablet 73563dc7466d4636a64ebd4d363935b1 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:23:10.860544  3237 tablet_service.cc:1511] Processing CreateTablet for tablet 1f3795979cf042008b3447b47c75de09 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:23:10.845541  3308 tablet_service.cc:1511] Processing CreateTablet for tablet f8a494da53f44c1d9780c046a155769a (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:23:10.919405  3381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a6d2799afd04cbe9bfaa74d467e66cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.926313  3439 tablet_bootstrap.cc:492] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:10.921404  3448 raft_consensus.cc:697] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:10.851740  3305 tablet_service.cc:1511] Processing CreateTablet for tablet 73563dc7466d4636a64ebd4d363935b1 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:23:10.940048  3385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8a494da53f44c1d9780c046a155769a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.942329  3379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73563dc7466d4636a64ebd4d363935b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.943091  3383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f3795979cf042008b3447b47c75de09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.958549  3384 tablet_service.cc:1511] Processing CreateTablet for tablet 11921fa3d80340c18f076a4d4499340b (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:23:10.960086  3384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11921fa3d80340c18f076a4d4499340b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:10.962299  3235 tablet_service.cc:1511] Processing CreateTablet for tablet f8a494da53f44c1d9780c046a155769a (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:23:10.968631  3440 tablet_bootstrap.cc:654] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.968815  3145 catalog_manager.cc:5697] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:10.989960  3448 consensus_queue.cc:237] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.003208  3384 tablet_service.cc:1511] Processing CreateTablet for tablet bb11c9e1f3b048769b242da5c7ae7cd7 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:23:11.001313  3380 tablet_service.cc:1511] Processing CreateTablet for tablet 7ff1130a2f274bccbc5b15389deacdc1 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:23:11.006364  3384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb11c9e1f3b048769b242da5c7ae7cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.014199  3380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff1130a2f274bccbc5b15389deacdc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.014322  3461 raft_consensus.cc:493] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.015029  3461 raft_consensus.cc:515] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.017793  3461 leader_election.cc:290] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:11.020637  3438 tablet_bootstrap.cc:492] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:11.020437  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c56ebe809a4c09b75ee18faf980d8c" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:11.021262  3438 ts_tablet_manager.cc:1403] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.318s	user 0.015s	sys 0.023s
W20260709 12:23:11.022491  3214 leader_election.cc:343] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.023190  3387 tablet_service.cc:1511] Processing CreateTablet for tablet 686eafc8b6bb49fbb5fd3dd0b68a3099 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:23:11.024770  3387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 686eafc8b6bb49fbb5fd3dd0b68a3099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.024446  3438 raft_consensus.cc:359] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.032544  3461 raft_consensus.cc:493] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.050590  3461 raft_consensus.cc:515] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.052856  3461 leader_election.cc:290] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:11.056425  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3368aec5b1412797545abe458a95fe" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:11.057348  3396 raft_consensus.cc:2468] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:11.059065  3214 leader_election.cc:304] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:11.060227  3461 raft_consensus.cc:2804] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:11.060664  3461 raft_consensus.cc:493] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.061030  3461 raft_consensus.cc:3060] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.068679  3439 tablet_bootstrap.cc:654] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.072871  3232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73563dc7466d4636a64ebd4d363935b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.050103  3438 raft_consensus.cc:385] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.074409  3438 raft_consensus.cc:740] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.075799  3237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f3795979cf042008b3447b47c75de09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.075909  3438 consensus_queue.cc:260] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.078449  3235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8a494da53f44c1d9780c046a155769a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.083209  3233 tablet_service.cc:1511] Processing CreateTablet for tablet 11921fa3d80340c18f076a4d4499340b (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:23:11.084523  3233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11921fa3d80340c18f076a4d4499340b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.080742  3381 tablet_service.cc:1511] Processing CreateTablet for tablet 8bcfe15edc4947b988fd204a12c79094 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:23:11.087118  3381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bcfe15edc4947b988fd204a12c79094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.087212  3238 tablet_service.cc:1511] Processing CreateTablet for tablet 7ff1130a2f274bccbc5b15389deacdc1 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:23:11.089054  3238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff1130a2f274bccbc5b15389deacdc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.091212  3239 tablet_service.cc:1511] Processing CreateTablet for tablet bb11c9e1f3b048769b242da5c7ae7cd7 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:23:11.092765  3239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb11c9e1f3b048769b242da5c7ae7cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.092841  3236 tablet_service.cc:1511] Processing CreateTablet for tablet 8bcfe15edc4947b988fd204a12c79094 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:23:11.094245  3236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bcfe15edc4947b988fd204a12c79094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.091214  3231 tablet_service.cc:1511] Processing CreateTablet for tablet 686eafc8b6bb49fbb5fd3dd0b68a3099 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:23:11.101922  3231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 686eafc8b6bb49fbb5fd3dd0b68a3099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.116111  3230 tablet_service.cc:1511] Processing CreateTablet for tablet c04a8b7530d5486a880a89a4ba0faac8 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:23:11.117307  3388 tablet_service.cc:1511] Processing CreateTablet for tablet fe4a63fc24af42ca862894414f658194 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:23:11.123682  3388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe4a63fc24af42ca862894414f658194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.123873  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3368aec5b1412797545abe458a95fe" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
W20260709 12:23:11.125846  3215 leader_election.cc:343] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.084870  3386 tablet_service.cc:1511] Processing CreateTablet for tablet c04a8b7530d5486a880a89a4ba0faac8 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:23:11.152753  3236 tablet_service.cc:1511] Processing CreateTablet for tablet 14193bb0c64048cf8ad561f477695970 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:23:11.093122  3379 tablet_service.cc:1511] Processing CreateTablet for tablet 14193bb0c64048cf8ad561f477695970 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:23:11.159893  3236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14193bb0c64048cf8ad561f477695970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.152915  3386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c04a8b7530d5486a880a89a4ba0faac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.172984  3230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c04a8b7530d5486a880a89a4ba0faac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.152886  3311 tablet_service.cc:1511] Processing CreateTablet for tablet 11921fa3d80340c18f076a4d4499340b (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:23:11.191397  3461 raft_consensus.cc:515] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.197757  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3368aec5b1412797545abe458a95fe" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
W20260709 12:23:11.199491  3215 leader_election.cc:343] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.209827  3238 tablet_service.cc:1511] Processing CreateTablet for tablet fe4a63fc24af42ca862894414f658194 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:23:11.211624  3238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe4a63fc24af42ca862894414f658194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.213953  3443 consensus_queue.cc:1048] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:11.221900  3448 consensus_queue.cc:1048] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:11.241146  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3368aec5b1412797545abe458a95fe" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:11.242429  3461 leader_election.cc:290] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:11.254117  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c56ebe809a4c09b75ee18faf980d8c" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:11.254752  3398 raft_consensus.cc:3060] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.255302  3323 raft_consensus.cc:2468] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:11.258653  3307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16f6e94aa295499cb8358daafcb934b6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:11.264963  3486 log.cc:927] Time spent T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:23:11.264364  3306 tablet_service.cc:1511] Processing CreateTablet for tablet 7ff1130a2f274bccbc5b15389deacdc1 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:23:11.266691  3440 tablet_bootstrap.cc:492] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:11.267217  3440 ts_tablet_manager.cc:1403] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.383s	user 0.053s	sys 0.061s
I20260709 12:23:11.268150  3310 tablet_service.cc:1511] Processing CreateTablet for tablet bb11c9e1f3b048769b242da5c7ae7cd7 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:23:11.270541  3305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73563dc7466d4636a64ebd4d363935b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.274503  3309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f37527cd3a34753b502172f81b808e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.277053  3313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a6d2799afd04cbe9bfaa74d467e66cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.277568  3440 raft_consensus.cc:359] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.278491  3440 raft_consensus.cc:385] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.282820  3215 leader_election.cc:304] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: c93989d3dd634ef585d77b03354121a4
I20260709 12:23:11.286758  3443 raft_consensus.cc:2804] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:11.286692  3448 consensus_queue.cc:1048] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:11.287385  3443 raft_consensus.cc:493] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.287806  3443 raft_consensus.cc:3060] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.288625  3398 raft_consensus.cc:2468] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:11.290177  3214 leader_election.cc:304] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:11.278822  3440 raft_consensus.cc:740] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.291241  3146 catalog_manager.cc:5697] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:11.300379  3439 tablet_bootstrap.cc:492] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:11.297581  3438 ts_tablet_manager.cc:1434] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.276s	user 0.010s	sys 0.009s
I20260709 12:23:11.328194  3439 ts_tablet_manager.cc:1403] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.402s	user 0.045s	sys 0.036s
I20260709 12:23:11.328649  3310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb11c9e1f3b048769b242da5c7ae7cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.336941  3439 raft_consensus.cc:359] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.327839  3461 raft_consensus.cc:2804] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:11.338258  3308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8a494da53f44c1d9780c046a155769a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.341070  3311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11921fa3d80340c18f076a4d4499340b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.340775  3309 tablet_service.cc:1511] Processing CreateTablet for tablet 8bcfe15edc4947b988fd204a12c79094 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:23:11.334594  3379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14193bb0c64048cf8ad561f477695970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.342365  3306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff1130a2f274bccbc5b15389deacdc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.340092  3438 tablet_bootstrap.cc:492] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:11.339227  3439 raft_consensus.cc:385] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.355702  3439 raft_consensus.cc:740] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.356956  3439 consensus_queue.cc:260] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.358829  3438 tablet_bootstrap.cc:654] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.359552  3439 ts_tablet_manager.cc:1434] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.003s
I20260709 12:23:11.334877  3312 tablet_service.cc:1511] Processing CreateTablet for tablet 686eafc8b6bb49fbb5fd3dd0b68a3099 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:23:11.346220  3314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f3795979cf042008b3447b47c75de09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.349193  3443 raft_consensus.cc:515] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:11.350476  3195 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:23:11.346846  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 56776us
I20260709 12:23:11.369562  3309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bcfe15edc4947b988fd204a12c79094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.372171  3312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 686eafc8b6bb49fbb5fd3dd0b68a3099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.376848  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c56ebe809a4c09b75ee18faf980d8c" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:11.377571  3323 raft_consensus.cc:3060] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.385417  3439 tablet_bootstrap.cc:492] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:11.366642  3461 raft_consensus.cc:697] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:11.397384  3310 tablet_service.cc:1511] Processing CreateTablet for tablet c04a8b7530d5486a880a89a4ba0faac8 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:23:11.385789  3462 raft_consensus.cc:493] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.402357  3145 catalog_manager.cc:5697] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:11.421211  3323 raft_consensus.cc:2468] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:11.405730  3443 leader_election.cc:290] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:11.416234  3462 raft_consensus.cc:515] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:11.421474  3215 consensus_peers.cc:597] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:11.330030  3440 consensus_queue.cc:260] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.416018  3310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c04a8b7530d5486a880a89a4ba0faac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.426139  3462 leader_election.cc:290] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:11.429114  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d639c2cd28f9418682dc27d0b0ba86b2" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:11.432834  3448 raft_consensus.cc:493] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.433580  3439 tablet_bootstrap.cc:654] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.433403  3448 raft_consensus.cc:515] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.431404  3314 tablet_service.cc:1511] Processing CreateTablet for tablet fe4a63fc24af42ca862894414f658194 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:23:11.436436  3314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe4a63fc24af42ca862894414f658194. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:11.437104  3364 leader_election.cc:343] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.437218  3448 leader_election.cc:290] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:11.442014  3215 leader_election.cc:304] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:11.443213  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:11.444023  3248 raft_consensus.cc:2468] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:11.444830  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c56ebe809a4c09b75ee18faf980d8c" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:11.445664  3398 raft_consensus.cc:3060] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.451069  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d639c2cd28f9418682dc27d0b0ba86b2" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:11.452805  3288 leader_election.cc:304] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
W20260709 12:23:11.454326  3362 leader_election.cc:343] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.455003  3362 leader_election.cc:304] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:11.454650  3494 raft_consensus.cc:2804] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:11.459592  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:11.461103  3481 raft_consensus.cc:2804] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:11.461614  3481 raft_consensus.cc:493] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.462131  3481 raft_consensus.cc:3060] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.461669  3461 consensus_queue.cc:237] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.469058  3440 ts_tablet_manager.cc:1434] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.202s	user 0.005s	sys 0.005s
I20260709 12:23:11.456745  3494 raft_consensus.cc:697] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:11.470455  3440 tablet_bootstrap.cc:492] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:11.471997  3310 tablet_service.cc:1511] Processing CreateTablet for tablet 14193bb0c64048cf8ad561f477695970 (DEFAULT_TABLE table=test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:23:11.474408  3310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14193bb0c64048cf8ad561f477695970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:11.470495  3494 consensus_queue.cc:237] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.494517  3438 tablet_bootstrap.cc:492] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:11.500166  3481 raft_consensus.cc:515] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.500938  3440 tablet_bootstrap.cc:654] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.510002  3438 ts_tablet_manager.cc:1403] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.170s	user 0.017s	sys 0.000s
I20260709 12:23:11.523756  3398 raft_consensus.cc:2468] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:11.527582  3462 raft_consensus.cc:493] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.528124  3462 raft_consensus.cc:515] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.530850  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:11.531864  3249 raft_consensus.cc:3060] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.536152  3462 leader_election.cc:290] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:11.541676  3439 tablet_bootstrap.cc:492] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:11.546118  3439 ts_tablet_manager.cc:1403] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.161s	user 0.018s	sys 0.000s
I20260709 12:23:11.529290  3438 raft_consensus.cc:359] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.549428  3439 raft_consensus.cc:359] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } }
I20260709 12:23:11.550122  3438 raft_consensus.cc:385] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.550302  3439 raft_consensus.cc:385] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.550510  3438 raft_consensus.cc:740] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.550760  3439 raft_consensus.cc:740] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.551451  3438 consensus_queue.cc:260] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.551779  3439 consensus_queue.cc:260] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } }
I20260709 12:23:11.554909  3462 raft_consensus.cc:2749] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:11.561247  3289 leader_election.cc:343] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.560678  3481 leader_election.cc:290] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:11.578766  3248 raft_consensus.cc:3060] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.580384  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c56ebe809a4c09b75ee18faf980d8c" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:11.581396  3323 raft_consensus.cc:2468] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:11.586836  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:11.573819  3438 ts_tablet_manager.cc:1434] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.048s	user 0.009s	sys 0.000s
I20260709 12:23:11.575549  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c56ebe809a4c09b75ee18faf980d8c" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:11.594449  3438 tablet_bootstrap.cc:492] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:11.613271  3364 leader_election.cc:304] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:11.614804  3498 raft_consensus.cc:2804] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:11.615254  3498 raft_consensus.cc:493] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.614360  3145 catalog_manager.cc:5697] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:11.615598  3498 raft_consensus.cc:3060] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:11.618486  3500 consensus_queue.cc:1048] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:23:11.621202  3289 leader_election.cc:343] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.605245  3249 raft_consensus.cc:2468] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:11.629040  3500 consensus_queue.cc:1048] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:11.633899  3288 leader_election.cc:304] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: c93989d3dd634ef585d77b03354121a4
I20260709 12:23:11.635280  3500 raft_consensus.cc:2804] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:11.635842  3500 raft_consensus.cc:697] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:11.636739  3500 consensus_queue.cc:237] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.648202  3440 tablet_bootstrap.cc:492] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:11.648857  3440 ts_tablet_manager.cc:1403] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.179s	user 0.017s	sys 0.057s
I20260709 12:23:11.609434  3438 tablet_bootstrap.cc:654] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.651947  3440 raft_consensus.cc:359] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.652757  3440 raft_consensus.cc:385] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.653072  3440 raft_consensus.cc:740] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.654171  3440 consensus_queue.cc:260] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.653182  3498 raft_consensus.cc:515] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.656791  3440 ts_tablet_manager.cc:1434] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:23:11.657262  3439 ts_tablet_manager.cc:1434] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.111s	user 0.006s	sys 0.025s
I20260709 12:23:11.657882  3440 tablet_bootstrap.cc:492] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:11.658445  3439 tablet_bootstrap.cc:492] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:11.660315  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c56ebe809a4c09b75ee18faf980d8c" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:11.660740  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c56ebe809a4c09b75ee18faf980d8c" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:11.654407  3144 catalog_manager.cc:5697] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:11.661433  3323 raft_consensus.cc:3060] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:11.664299  3145 catalog_manager.cc:5697] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:11.678949  3438 tablet_bootstrap.cc:492] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:11.680286  3438 ts_tablet_manager.cc:1403] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.086s	user 0.014s	sys 0.000s
I20260709 12:23:11.683521  3323 raft_consensus.cc:2468] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 2.
I20260709 12:23:11.683696  3438 raft_consensus.cc:359] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.684779  3364 leader_election.cc:304] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:11.686483  3438 raft_consensus.cc:385] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.686548  3504 raft_consensus.cc:2804] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:11.687285  3504 raft_consensus.cc:697] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:11.688189  3504 consensus_queue.cc:237] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.689702  3462 raft_consensus.cc:493] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.690215  3462 raft_consensus.cc:515] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } }
I20260709 12:23:11.686947  3438 raft_consensus.cc:740] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.664330  3498 leader_election.cc:290] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:11.692711  3438 consensus_queue.cc:260] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.693275  3462 leader_election.cc:290] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:11.696234  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9cc3d1278674852b3b6c8c39f4f1b59" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:11.697005  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9cc3d1278674852b3b6c8c39f4f1b59" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:11.702773  3438 ts_tablet_manager.cc:1434] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:23:11.703035  3362 leader_election.cc:304] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:11.703910  3438 tablet_bootstrap.cc:492] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:11.704203  3462 raft_consensus.cc:2749] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:11.705495  3440 tablet_bootstrap.cc:654] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.710979  3439 tablet_bootstrap.cc:654] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.710745  3438 tablet_bootstrap.cc:654] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.722538  3147 catalog_manager.cc:5697] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 2 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:11.727747  3438 tablet_bootstrap.cc:492] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:11.728421  3438 ts_tablet_manager.cc:1403] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.004s
I20260709 12:23:11.732874  3440 tablet_bootstrap.cc:492] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:11.733503  3440 ts_tablet_manager.cc:1403] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.076s	user 0.013s	sys 0.000s
I20260709 12:23:11.733731  3438 raft_consensus.cc:359] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.734572  3438 raft_consensus.cc:385] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.734869  3438 raft_consensus.cc:740] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.735977  3438 consensus_queue.cc:260] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.738153  3439 tablet_bootstrap.cc:492] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:11.738161  3438 ts_tablet_manager.cc:1434] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 12:23:11.739014  3439 ts_tablet_manager.cc:1403] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.081s	user 0.014s	sys 0.003s
I20260709 12:23:11.738839  3440 raft_consensus.cc:359] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.739576  3440 raft_consensus.cc:385] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.739842  3440 raft_consensus.cc:740] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.740662  3440 consensus_queue.cc:260] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.742471  3439 raft_consensus.cc:359] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.743505  3439 raft_consensus.cc:385] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.743862  3439 raft_consensus.cc:740] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.744874  3439 consensus_queue.cc:260] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.747321  3494 raft_consensus.cc:493] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.747614  3439 ts_tablet_manager.cc:1434] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:23:11.747948  3494 raft_consensus.cc:515] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.748970  3439 tablet_bootstrap.cc:492] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:11.752099  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbb481337ada4c83945e213fb117ee7f" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:11.752099  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbb481337ada4c83945e213fb117ee7f" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:11.754103  3396 raft_consensus.cc:2468] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:11.755393  3494 leader_election.cc:290] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:11.755585  3438 tablet_bootstrap.cc:492] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
W20260709 12:23:11.755735  3215 leader_election.cc:343] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.757995  3214 leader_election.cc:304] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:11.762475  3438 tablet_bootstrap.cc:654] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.761740  3440 ts_tablet_manager.cc:1434] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:23:11.764161  3494 raft_consensus.cc:2804] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:11.764799  3494 raft_consensus.cc:493] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.765141  3494 raft_consensus.cc:3060] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.777288  3440 tablet_bootstrap.cc:492] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:11.779160  3439 tablet_bootstrap.cc:654] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.780943  3494 raft_consensus.cc:515] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.784374  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbb481337ada4c83945e213fb117ee7f" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
W20260709 12:23:11.786048  3215 leader_election.cc:343] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.787979  3440 tablet_bootstrap.cc:654] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.788980  3438 tablet_bootstrap.cc:492] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:11.789595  3438 ts_tablet_manager.cc:1403] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.050s	user 0.016s	sys 0.016s
I20260709 12:23:11.790402  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbb481337ada4c83945e213fb117ee7f" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:11.791500  3396 raft_consensus.cc:3060] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.793336  3438 raft_consensus.cc:359] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.794360  3438 raft_consensus.cc:385] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.794742  3438 raft_consensus.cc:740] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.795800  3494 leader_election.cc:290] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:11.795733  3438 consensus_queue.cc:260] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.801275  3440 tablet_bootstrap.cc:492] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:11.803184  3440 ts_tablet_manager.cc:1403] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.002s
I20260709 12:23:11.802412  3438 ts_tablet_manager.cc:1434] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:23:11.805140  3438 tablet_bootstrap.cc:492] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:11.806226  3439 tablet_bootstrap.cc:492] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:11.808339  3396 raft_consensus.cc:2468] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:11.809072  3440 raft_consensus.cc:359] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.810019  3440 raft_consensus.cc:385] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.810328  3440 raft_consensus.cc:740] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.810684  3214 leader_election.cc:304] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:11.811090  3440 consensus_queue.cc:260] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.809115  3439 ts_tablet_manager.cc:1403] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.060s	user 0.011s	sys 0.001s
I20260709 12:23:11.812268  3461 raft_consensus.cc:2804] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:11.812816  3461 raft_consensus.cc:697] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:11.813506  3440 ts_tablet_manager.cc:1434] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 12:23:11.814720  3440 tablet_bootstrap.cc:492] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:11.815088  3461 consensus_queue.cc:237] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.821317  3439 raft_consensus.cc:359] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.822474  3439 raft_consensus.cc:385] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.822917  3439 raft_consensus.cc:740] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.823928  3438 tablet_bootstrap.cc:654] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.823911  3439 consensus_queue.cc:260] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.827735  3439 ts_tablet_manager.cc:1434] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:23:11.829501  3439 tablet_bootstrap.cc:492] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:11.831467  3440 tablet_bootstrap.cc:654] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.835312  3323 raft_consensus.cc:3060] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.843945  3500 raft_consensus.cc:493] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.844542  3500 raft_consensus.cc:515] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.846822  3440 tablet_bootstrap.cc:492] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:11.847733  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af545b01cb24f1cacb0056f804a7692" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:11.848417  3440 ts_tablet_manager.cc:1403] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.003s
W20260709 12:23:11.849462  3288 leader_election.cc:343] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.850651  3500 leader_election.cc:290] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:11.852357  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af545b01cb24f1cacb0056f804a7692" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
W20260709 12:23:11.854243  3289 leader_election.cc:343] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.854717  3289 leader_election.cc:304] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:11.855938  3500 raft_consensus.cc:2749] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:11.852375  3440 raft_consensus.cc:359] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.857345  3147 catalog_manager.cc:5697] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:11.857470  3440 raft_consensus.cc:385] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.859273  3440 raft_consensus.cc:740] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.861317  3440 consensus_queue.cc:260] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.866701  3439 tablet_bootstrap.cc:654] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.867509  3440 ts_tablet_manager.cc:1434] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:23:11.868803  3440 tablet_bootstrap.cc:492] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:11.883455  3440 tablet_bootstrap.cc:654] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.889393  3438 tablet_bootstrap.cc:492] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:11.889843  3439 tablet_bootstrap.cc:492] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:11.890125  3438 ts_tablet_manager.cc:1403] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.085s	user 0.020s	sys 0.018s
I20260709 12:23:11.890565  3439 ts_tablet_manager.cc:1403] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.061s	user 0.014s	sys 0.001s
I20260709 12:23:11.893868  3438 raft_consensus.cc:359] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.894692  3438 raft_consensus.cc:385] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.894974  3438 raft_consensus.cc:740] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.895442  3439 raft_consensus.cc:359] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.895669  3438 consensus_queue.cc:260] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.896382  3439 raft_consensus.cc:385] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.896698  3439 raft_consensus.cc:740] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.897567  3504 raft_consensus.cc:493] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.897389  3439 consensus_queue.cc:260] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.898296  3438 ts_tablet_manager.cc:1434] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:23:11.898077  3504 raft_consensus.cc:515] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.899492  3438 tablet_bootstrap.cc:492] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:11.899485  3439 ts_tablet_manager.cc:1434] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:11.900281  3504 leader_election.cc:290] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:11.903210  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:11.903739  3439 tablet_bootstrap.cc:492] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:11.905459  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:11.907806  3438 tablet_bootstrap.cc:654] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.910655  3439 tablet_bootstrap.cc:654] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:11.913415  3364 leader_election.cc:343] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:11.914883  3362 leader_election.cc:343] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.915565  3362 leader_election.cc:304] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:11.917282  3462 raft_consensus.cc:2749] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:11.923309  3439 tablet_bootstrap.cc:492] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:11.923245  3500 raft_consensus.cc:493] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.924131  3439 ts_tablet_manager.cc:1403] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.002s
I20260709 12:23:11.924328  3500 raft_consensus.cc:515] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.927211  3439 raft_consensus.cc:359] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.927959  3439 raft_consensus.cc:385] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.928241  3439 raft_consensus.cc:740] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.928313  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3368aec5b1412797545abe458a95fe" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:11.929733  3323 raft_consensus.cc:1240] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Rejecting Update request from peer 7cdaa0234c64450eb4ce4500b51b8a81 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:11.930154  3439 consensus_queue.cc:260] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.931102  3500 leader_election.cc:290] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:11.932940  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3368aec5b1412797545abe458a95fe" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:11.935963  3396 raft_consensus.cc:2393] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8525f77f43714a58bf158d872c1db3e4 in current term 1: Already voted for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in this term.
I20260709 12:23:11.938210  3289 leader_election.cc:304] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:11.938676  3439 ts_tablet_manager.cc:1434] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:23:11.939891  3439 tablet_bootstrap.cc:492] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:11.940608  3461 consensus_queue.cc:1059] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:11.941942  3461 raft_consensus.cc:3055] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:11.942385  3461 raft_consensus.cc:740] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:11.943512  3461 consensus_queue.cc:260] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.946621  3439 tablet_bootstrap.cc:654] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.950109  3461 raft_consensus.cc:3060] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:11.952412  3500 raft_consensus.cc:3060] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.956187  3440 tablet_bootstrap.cc:492] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:11.957187  3440 ts_tablet_manager.cc:1403] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.089s	user 0.031s	sys 0.004s
I20260709 12:23:11.986603  3443 raft_consensus.cc:493] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.987345  3443 raft_consensus.cc:515] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.993453  3440 raft_consensus.cc:359] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:11.994426  3440 raft_consensus.cc:385] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.994676  3443 leader_election.cc:290] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:11.994797  3440 raft_consensus.cc:740] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.995085  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:11.996059  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
W20260709 12:23:11.996809  3214 leader_election.cc:343] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.995863  3440 consensus_queue.cc:260] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:11.997383  3215 leader_election.cc:343] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.997830  3215 leader_election.cc:304] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:11.999410  3440 ts_tablet_manager.cc:1434] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.042s	user 0.017s	sys 0.009s
I20260709 12:23:12.000033  3443 raft_consensus.cc:2749] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:12.001155  3440 tablet_bootstrap.cc:492] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:12.003700  3443 raft_consensus.cc:493] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.004366  3438 tablet_bootstrap.cc:492] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:12.004304  3443 raft_consensus.cc:515] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.005263  3438 ts_tablet_manager.cc:1403] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.106s	user 0.011s	sys 0.013s
I20260709 12:23:12.007658  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd6cee8d33de46388ec38d683a789e94" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:12.008097  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd6cee8d33de46388ec38d683a789e94" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:12.009079  3440 tablet_bootstrap.cc:654] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:12.009809  3215 leader_election.cc:343] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:12.010474  3214 leader_election.cc:343] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:12.011023  3214 leader_election.cc:304] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:12.012090  3494 raft_consensus.cc:2749] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:12.012310  3439 tablet_bootstrap.cc:492] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:12.016546  3438 raft_consensus.cc:359] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.012880  3439 ts_tablet_manager.cc:1403] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.073s	user 0.009s	sys 0.020s
I20260709 12:23:12.018963  3443 leader_election.cc:290] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.025933  3438 raft_consensus.cc:385] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.026348  3438 raft_consensus.cc:740] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.027204  3438 consensus_queue.cc:260] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.028720  3439 raft_consensus.cc:359] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.029394  3439 raft_consensus.cc:385] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.029542  3438 ts_tablet_manager.cc:1434] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 12:23:12.029732  3439 raft_consensus.cc:740] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.031185  3438 tablet_bootstrap.cc:492] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:12.030855  3439 consensus_queue.cc:260] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.020134  3440 tablet_bootstrap.cc:492] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:12.022111  3500 raft_consensus.cc:2749] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:12.023417  3396 raft_consensus.cc:3060] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.036779  3439 ts_tablet_manager.cc:1434] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:23:12.037384  3440 ts_tablet_manager.cc:1403] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.000s
I20260709 12:23:12.037814  3443 consensus_queue.cc:1048] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:12.040609  3440 raft_consensus.cc:359] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.041496  3440 raft_consensus.cc:385] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.041805  3440 raft_consensus.cc:740] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.038208  3439 tablet_bootstrap.cc:492] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:12.042598  3440 consensus_queue.cc:260] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.043599  3461 consensus_queue.cc:1048] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:12.046648  3440 ts_tablet_manager.cc:1434] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:23:12.047690  3440 tablet_bootstrap.cc:492] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:12.053115  3439 tablet_bootstrap.cc:654] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.056047  3438 tablet_bootstrap.cc:654] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.066146  3440 tablet_bootstrap.cc:654] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.071352  3439 tablet_bootstrap.cc:492] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:12.072017  3439 ts_tablet_manager.cc:1403] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.002s
I20260709 12:23:12.074302  3438 tablet_bootstrap.cc:492] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:12.075397  3438 ts_tablet_manager.cc:1403] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.000s
I20260709 12:23:12.075466  3439 raft_consensus.cc:359] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.076504  3439 raft_consensus.cc:385] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.077088  3439 raft_consensus.cc:740] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.077721  3462 raft_consensus.cc:493] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.078358  3462 raft_consensus.cc:515] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.078212  3439 consensus_queue.cc:260] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.081782  3462 leader_election.cc:290] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:12.082439  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:12.082906  3439 ts_tablet_manager.cc:1434] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 12:23:12.084363  3439 tablet_bootstrap.cc:492] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:12.084667  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:12.085619  3247 raft_consensus.cc:2393] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c93989d3dd634ef585d77b03354121a4 in current term 1: Already voted for candidate 8525f77f43714a58bf158d872c1db3e4 in this term.
I20260709 12:23:12.087029  3362 leader_election.cc:304] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:12.088259  3438 raft_consensus.cc:359] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.090245  3438 raft_consensus.cc:385] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.090365  3481 raft_consensus.cc:493] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.088702  3462 raft_consensus.cc:3060] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.091879  3438 raft_consensus.cc:740] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.093323  3438 consensus_queue.cc:260] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.096783  3439 tablet_bootstrap.cc:654] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.091168  3481 raft_consensus.cc:515] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.096833  3438 ts_tablet_manager.cc:1434] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 12:23:12.099921  3438 tablet_bootstrap.cc:492] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:12.103093  3481 leader_election.cc:290] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.103338  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d639c2cd28f9418682dc27d0b0ba86b2" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:12.105084  3395 raft_consensus.cc:2468] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:12.106405  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d639c2cd28f9418682dc27d0b0ba86b2" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:12.110926  3247 raft_consensus.cc:2468] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:12.112457  3289 leader_election.cc:304] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:12.113706  3481 raft_consensus.cc:2804] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:12.114081  3481 raft_consensus.cc:493] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.114431  3481 raft_consensus.cc:3060] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.116357  3462 raft_consensus.cc:2749] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:12.132023  3438 tablet_bootstrap.cc:654] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.132081  3500 consensus_queue.cc:1048] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:12.141309  3481 raft_consensus.cc:515] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.162740  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d639c2cd28f9418682dc27d0b0ba86b2" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:12.163707  3247 raft_consensus.cc:3060] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.162998  3481 leader_election.cc:290] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.168917  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d639c2cd28f9418682dc27d0b0ba86b2" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:12.174677  3398 raft_consensus.cc:3060] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.192435  3439 tablet_bootstrap.cc:492] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:12.192986  3439 ts_tablet_manager.cc:1403] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.109s	user 0.013s	sys 0.019s
I20260709 12:23:12.197242  3440 tablet_bootstrap.cc:492] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:12.200963  3440 ts_tablet_manager.cc:1403] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.153s	user 0.039s	sys 0.022s
I20260709 12:23:12.194463  3504 raft_consensus.cc:493] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.195473  3398 raft_consensus.cc:2468] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:12.210887  3248 raft_consensus.cc:1275] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:23:12.221468  3248 pending_rounds.cc:85] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81: Aborting all ops after (but not including) 0
I20260709 12:23:12.221974  3248 pending_rounds.cc:107] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:23:12.222311  3248 raft_consensus.cc:2889] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:12.211256  3247 raft_consensus.cc:2468] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:12.212426  3500 consensus_queue.cc:1048] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:12.203295  3504 raft_consensus.cc:515] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.213817  3438 tablet_bootstrap.cc:492] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:12.227388  3438 ts_tablet_manager.cc:1403] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.128s	user 0.016s	sys 0.006s
I20260709 12:23:12.215574  3440 raft_consensus.cc:359] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.227367  3462 consensus_queue.cc:1048] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:12.228864  3504 leader_election.cc:290] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:12.228123  3440 raft_consensus.cc:385] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.230151  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054906db16c94967bdef7c9dd5a47ea3" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:12.230083  3440 raft_consensus.cc:740] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.231909  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054906db16c94967bdef7c9dd5a47ea3" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:12.232276  3440 consensus_queue.cc:260] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.232623  3324 raft_consensus.cc:2468] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:12.234148  3364 leader_election.cc:304] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:12.234709  3440 ts_tablet_manager.cc:1434] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.002s
I20260709 12:23:12.235281  3504 raft_consensus.cc:2804] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:12.235720  3504 raft_consensus.cc:493] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.215025  3439 raft_consensus.cc:359] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.237380  3439 raft_consensus.cc:385] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.237751  3439 raft_consensus.cc:740] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.238373  3288 leader_election.cc:304] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:12.238595  3439 consensus_queue.cc:260] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.239477  3481 raft_consensus.cc:2804] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:12.240072  3481 raft_consensus.cc:697] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
W20260709 12:23:12.235294  3362 leader_election.cc:343] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:12.236958  3438 raft_consensus.cc:359] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.241225  3439 ts_tablet_manager.cc:1434] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.000s
I20260709 12:23:12.241447  3438 raft_consensus.cc:385] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.237406  3440 tablet_bootstrap.cc:492] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:12.241866  3438 raft_consensus.cc:740] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.240760  3504 raft_consensus.cc:3060] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.243526  3481 consensus_queue.cc:237] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.244310  3438 consensus_queue.cc:260] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.246879  3439 tablet_bootstrap.cc:492] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:12.249662  3438 ts_tablet_manager.cc:1434] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:23:12.250087  3461 consensus_queue.cc:1048] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:12.250785  3438 tablet_bootstrap.cc:492] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:12.252192  3461 raft_consensus.cc:493] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.252738  3461 raft_consensus.cc:515] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.255025  3499 raft_consensus.cc:493] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.255702  3461 leader_election.cc:290] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.255584  3499 raft_consensus.cc:515] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.256284  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d97394c304e81affa5ee75b52de28" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:12.257241  3324 raft_consensus.cc:2468] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:12.259052  3499 leader_election.cc:290] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.259261  3215 leader_election.cc:304] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:12.260963  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d97394c304e81affa5ee75b52de28" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:12.264488  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9253ca0fa41f43a7ac13ff17879c2ed3" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:12.265604  3247 raft_consensus.cc:2468] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:12.268424  3288 leader_election.cc:304] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:12.269889  3499 raft_consensus.cc:2804] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:12.262537  3461 raft_consensus.cc:2804] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:12.270589  3461 raft_consensus.cc:493] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.270499  3499 raft_consensus.cc:493] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.271898  3499 raft_consensus.cc:3060] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.262959  3396 raft_consensus.cc:2468] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:12.277511  3443 raft_consensus.cc:493] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.278301  3443 raft_consensus.cc:515] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.283505  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af545b01cb24f1cacb0056f804a7692" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:12.284507  3324 raft_consensus.cc:2468] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:12.285887  3461 raft_consensus.cc:3060] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.281262  3443 leader_election.cc:290] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.286932  3215 leader_election.cc:304] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:12.304313  3443 raft_consensus.cc:2804] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:12.305994  3443 raft_consensus.cc:493] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.324532  3443 raft_consensus.cc:3060] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.338241  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af545b01cb24f1cacb0056f804a7692" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:12.339260  3396 raft_consensus.cc:2468] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:12.342654  3438 tablet_bootstrap.cc:654] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.349169  3439 tablet_bootstrap.cc:654] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.352496  3499 raft_consensus.cc:515] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.354135  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9253ca0fa41f43a7ac13ff17879c2ed3" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:12.329037  3504 raft_consensus.cc:515] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.365710  3440 tablet_bootstrap.cc:654] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.366079  3324 raft_consensus.cc:1275] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:23:12.359313  3462 raft_consensus.cc:493] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.360060  3481 raft_consensus.cc:493] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.357568  3541 raft_consensus.cc:493] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.363559  3504 leader_election.cc:290] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:12.383632  3144 catalog_manager.cc:5697] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:12.386974  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054906db16c94967bdef7c9dd5a47ea3" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:12.387351  3461 raft_consensus.cc:515] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.357762  3396 raft_consensus.cc:2468] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:12.387388  3443 raft_consensus.cc:515] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.390801  3462 raft_consensus.cc:515] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.393780  3481 raft_consensus.cc:515] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.395527  3461 leader_election.cc:290] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.394333  3443 leader_election.cc:290] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.396791  3481 leader_election.cc:290] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:23:12.398072  3195 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:12.399204  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:12.400187  3248 raft_consensus.cc:2468] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:12.402321  3288 leader_election.cc:304] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:12.405454  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af545b01cb24f1cacb0056f804a7692" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:12.406368  3322 raft_consensus.cc:3060] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.387882  3323 raft_consensus.cc:3060] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.411422  3542 consensus_queue.cc:1048] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:23:12.413968  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:12.423935  3438 tablet_bootstrap.cc:492] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:12.424363  3438 ts_tablet_manager.cc:1403] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.174s	user 0.022s	sys 0.015s
I20260709 12:23:12.417053  3481 raft_consensus.cc:2804] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:12.411495  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d97394c304e81affa5ee75b52de28" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:12.418391  3462 leader_election.cc:290] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:12.431710  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054906db16c94967bdef7c9dd5a47ea3" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:12.428262  3481 raft_consensus.cc:493] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.431772  3324 raft_consensus.cc:3060] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.419373  3541 raft_consensus.cc:515] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.438776  3481 raft_consensus.cc:3060] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.445376  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af545b01cb24f1cacb0056f804a7692" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:12.445959  3440 tablet_bootstrap.cc:492] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:12.446544  3440 ts_tablet_manager.cc:1403] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.209s	user 0.016s	sys 0.014s
I20260709 12:23:12.447190  3396 raft_consensus.cc:2468] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
W20260709 12:23:12.446368  3215 consensus_peers.cc:597] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:12.450690  3541 leader_election.cc:290] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:12.454924  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af545b01cb24f1cacb0056f804a7692" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:12.455587  3395 raft_consensus.cc:3060] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.455235  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9253ca0fa41f43a7ac13ff17879c2ed3" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:12.456284  3397 raft_consensus.cc:3060] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.458117  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9253ca0fa41f43a7ac13ff17879c2ed3" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:12.459030  3247 raft_consensus.cc:3060] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.463346  3248 raft_consensus.cc:2393] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c93989d3dd634ef585d77b03354121a4 in current term 1: Already voted for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in this term.
I20260709 12:23:12.474274  3321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af545b01cb24f1cacb0056f804a7692" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:12.475806  3438 raft_consensus.cc:359] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.476708  3438 raft_consensus.cc:385] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.477029  3438 raft_consensus.cc:740] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.477819  3438 consensus_queue.cc:260] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.480464  3438 ts_tablet_manager.cc:1434] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.056s	user 0.009s	sys 0.000s
I20260709 12:23:12.482504  3438 tablet_bootstrap.cc:492] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:12.487468  3439 tablet_bootstrap.cc:492] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:12.488138  3439 ts_tablet_manager.cc:1403] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.242s	user 0.013s	sys 0.010s
I20260709 12:23:12.494575  3397 raft_consensus.cc:2468] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:12.498227  3323 raft_consensus.cc:2468] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:12.501389  3322 raft_consensus.cc:2468] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:12.503677  3364 leader_election.cc:304] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:12.504320  3289 leader_election.cc:304] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:12.504727  3542 raft_consensus.cc:2804] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:12.504907  3215 leader_election.cc:304] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:12.461408  3499 leader_election.cc:290] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.505589  3542 raft_consensus.cc:697] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:12.506026  3461 raft_consensus.cc:2804] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:12.506556  3461 raft_consensus.cc:697] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:12.506026  3499 raft_consensus.cc:2804] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:12.503674  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:12.523734  3395 raft_consensus.cc:2468] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
W20260709 12:23:12.523962  3362 leader_election.cc:343] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:12.524993  3461 consensus_queue.cc:237] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.510118  3542 consensus_queue.cc:237] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.522202  3499 raft_consensus.cc:697] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:12.530941  3324 raft_consensus.cc:2468] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:12.522651  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d97394c304e81affa5ee75b52de28" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:12.532824  3499 consensus_queue.cc:237] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.538568  3440 raft_consensus.cc:359] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.548236  3440 raft_consensus.cc:385] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.548666  3440 raft_consensus.cc:740] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.554965  3440 consensus_queue.cc:260] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.529791  3439 raft_consensus.cc:359] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.532282  3481 raft_consensus.cc:515] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.545419  3247 raft_consensus.cc:2468] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:12.559499  3438 tablet_bootstrap.cc:654] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.525300  3362 leader_election.cc:304] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:12.557495  3439 raft_consensus.cc:385] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.554147  3397 raft_consensus.cc:3060] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.560763  3439 raft_consensus.cc:740] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.563058  3440 ts_tablet_manager.cc:1434] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.116s	user 0.010s	sys 0.000s
I20260709 12:23:12.563620  3322 raft_consensus.cc:2393] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c93989d3dd634ef585d77b03354121a4 in current term 1: Already voted for candidate 8525f77f43714a58bf158d872c1db3e4 in this term.
I20260709 12:23:12.561276  3215 leader_election.cc:304] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:12.566723  3440 tablet_bootstrap.cc:492] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:12.566648  3461 raft_consensus.cc:2804] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:12.567502  3461 raft_consensus.cc:697] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
W20260709 12:23:12.570535  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 45342.3us
I20260709 12:23:12.572463  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:12.573143  3247 raft_consensus.cc:2468] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:12.569566  3542 raft_consensus.cc:2749] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:12.576828  3439 consensus_queue.cc:260] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.581837  3145 catalog_manager.cc:5697] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:12.568434  3461 consensus_queue.cc:237] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.592840  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:12.592973  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:12.593617  3247 raft_consensus.cc:3060] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.593828  3396 raft_consensus.cc:3060] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.600731  3439 ts_tablet_manager.cc:1434] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.112s	user 0.009s	sys 0.000s
I20260709 12:23:12.602011  3439 tablet_bootstrap.cc:492] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:12.590835  3481 leader_election.cc:290] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.605957  3362 leader_election.cc:304] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:12.608227  3396 raft_consensus.cc:2468] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:12.611744  3542 raft_consensus.cc:2764] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:23:12.609719  3439 tablet_bootstrap.cc:654] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.615720  3289 leader_election.cc:304] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:12.617076  3499 raft_consensus.cc:2804] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:12.617865  3499 raft_consensus.cc:697] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:12.618193  3438 tablet_bootstrap.cc:492] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:12.621961  3438 ts_tablet_manager.cc:1403] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.140s	user 0.013s	sys 0.001s
I20260709 12:23:12.622438  3499 consensus_queue.cc:237] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.625993  3438 raft_consensus.cc:359] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.626797  3438 raft_consensus.cc:385] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.626116  3247 raft_consensus.cc:2468] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:12.627110  3438 raft_consensus.cc:740] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.628294  3438 consensus_queue.cc:260] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.631111  3438 ts_tablet_manager.cc:1434] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:23:12.632185  3438 tablet_bootstrap.cc:492] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:12.632997  3440 tablet_bootstrap.cc:654] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.649482  3397 raft_consensus.cc:2468] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:12.652014  3439 tablet_bootstrap.cc:492] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:12.652787  3439 ts_tablet_manager.cc:1403] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.001s
I20260709 12:23:12.659796  3439 raft_consensus.cc:359] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.663149  3439 raft_consensus.cc:385] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.663714  3439 raft_consensus.cc:740] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.664880  3439 consensus_queue.cc:260] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.667809  3481 consensus_queue.cc:1048] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:12.671386  3439 ts_tablet_manager.cc:1434] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:23:12.672405  3439 tablet_bootstrap.cc:492] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:12.683399  3147 catalog_manager.cc:5697] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:12.684697  3145 catalog_manager.cc:5697] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:12.703359  3440 tablet_bootstrap.cc:492] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:12.703845  3440 ts_tablet_manager.cc:1403] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.137s	user 0.020s	sys 0.000s
I20260709 12:23:12.707870  3438 tablet_bootstrap.cc:654] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.711639  3439 tablet_bootstrap.cc:654] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.713817  3440 raft_consensus.cc:359] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.714561  3440 raft_consensus.cc:385] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.714843  3440 raft_consensus.cc:740] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.715786  3440 consensus_queue.cc:260] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.718760  3440 ts_tablet_manager.cc:1434] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.001s
I20260709 12:23:12.719950  3440 tablet_bootstrap.cc:492] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:12.725440  3144 catalog_manager.cc:5697] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:12.728875  3144 catalog_manager.cc:5697] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:12.733924  3440 tablet_bootstrap.cc:654] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.734616  3499 consensus_queue.cc:1048] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:23:12.742348  3438 tablet_bootstrap.cc:492] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:12.742882  3438 ts_tablet_manager.cc:1403] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.111s	user 0.007s	sys 0.006s
I20260709 12:23:12.754802  3439 tablet_bootstrap.cc:492] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:12.755515  3439 ts_tablet_manager.cc:1403] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.083s	user 0.017s	sys 0.017s
I20260709 12:23:12.759349  3439 raft_consensus.cc:359] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.760067  3439 raft_consensus.cc:385] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.760353  3439 raft_consensus.cc:740] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.761088  3439 consensus_queue.cc:260] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.763425  3439 ts_tablet_manager.cc:1434] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:23:12.764411  3439 tablet_bootstrap.cc:492] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:12.766394  3438 raft_consensus.cc:359] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.768428  3438 raft_consensus.cc:385] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.768800  3438 raft_consensus.cc:740] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.769570  3438 consensus_queue.cc:260] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.770913  3439 tablet_bootstrap.cc:654] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.772356  3440 tablet_bootstrap.cc:492] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:12.772924  3440 ts_tablet_manager.cc:1403] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.053s	user 0.004s	sys 0.009s
I20260709 12:23:12.776656  3440 raft_consensus.cc:359] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.778508  3440 raft_consensus.cc:385] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.778951  3440 raft_consensus.cc:740] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.780331  3440 consensus_queue.cc:260] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.785001  3438 ts_tablet_manager.cc:1434] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.042s	user 0.004s	sys 0.003s
I20260709 12:23:12.786175  3438 tablet_bootstrap.cc:492] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:12.790874  3440 ts_tablet_manager.cc:1434] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:23:12.792529  3440 tablet_bootstrap.cc:492] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:12.801021  3440 tablet_bootstrap.cc:654] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.807646  3461 raft_consensus.cc:493] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.808779  3438 tablet_bootstrap.cc:654] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.808420  3461 raft_consensus.cc:515] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.812163  3461 leader_election.cc:290] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.812376  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054906db16c94967bdef7c9dd5a47ea3" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:12.813426  3320 raft_consensus.cc:2393] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cdaa0234c64450eb4ce4500b51b8a81 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:12.815932  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054906db16c94967bdef7c9dd5a47ea3" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:12.818419  3440 tablet_bootstrap.cc:492] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:12.818583  3214 leader_election.cc:304] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:12.819181  3440 ts_tablet_manager.cc:1403] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.010s
I20260709 12:23:12.819810  3461 raft_consensus.cc:3060] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.822328  3440 raft_consensus.cc:359] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.822721  3438 tablet_bootstrap.cc:492] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:12.823177  3440 raft_consensus.cc:385] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.823438  3438 ts_tablet_manager.cc:1403] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.037s	user 0.006s	sys 0.008s
I20260709 12:23:12.823562  3440 raft_consensus.cc:740] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.825143  3440 consensus_queue.cc:260] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.827847  3462 raft_consensus.cc:493] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.827479  3438 raft_consensus.cc:359] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.828344  3462 raft_consensus.cc:515] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.828703  3438 raft_consensus.cc:385] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.829077  3438 raft_consensus.cc:740] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.829917  3438 consensus_queue.cc:260] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.830578  3462 leader_election.cc:290] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:12.832448  3440 ts_tablet_manager.cc:1434] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:23:12.833421  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d167141acd1450192e10b3c44347860" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:12.834141  3247 raft_consensus.cc:2468] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:12.835518  3362 leader_election.cc:304] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:12.836356  3462 raft_consensus.cc:2804] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:12.836696  3462 raft_consensus.cc:493] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.836992  3462 raft_consensus.cc:3060] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.833422  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d167141acd1450192e10b3c44347860" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:12.834095  3461 raft_consensus.cc:2749] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:12.841739  3364 leader_election.cc:343] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:12.841799  3438 ts_tablet_manager.cc:1434] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.001s
I20260709 12:23:12.842401  3440 tablet_bootstrap.cc:492] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:12.846515  3439 tablet_bootstrap.cc:492] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:12.846845  3438 tablet_bootstrap.cc:492] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:12.850299  3440 tablet_bootstrap.cc:654] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.850549  3462 raft_consensus.cc:515] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.854101  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d167141acd1450192e10b3c44347860" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:12.854897  3438 tablet_bootstrap.cc:654] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.854877  3247 raft_consensus.cc:3060] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.864850  3439 ts_tablet_manager.cc:1403] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.101s	user 0.021s	sys 0.021s
I20260709 12:23:12.867579  3438 tablet_bootstrap.cc:492] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:12.868175  3438 ts_tablet_manager.cc:1403] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 12:23:12.871198  3438 raft_consensus.cc:359] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.872007  3438 raft_consensus.cc:385] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.871966  3247 raft_consensus.cc:2468] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:12.872581  3438 raft_consensus.cc:740] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.873410  3438 consensus_queue.cc:260] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.873947  3362 leader_election.cc:304] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:12.875211  3541 raft_consensus.cc:2804] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:12.875842  3541 raft_consensus.cc:697] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:12.876845  3541 consensus_queue.cc:237] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.880162  3438 ts_tablet_manager.cc:1434] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:23:12.881258  3438 tablet_bootstrap.cc:492] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:12.883916  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d167141acd1450192e10b3c44347860" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:12.884300  3462 leader_election.cc:290] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:12.884723  3320 raft_consensus.cc:3060] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.869061  3439 raft_consensus.cc:359] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.887248  3439 raft_consensus.cc:385] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.887612  3439 raft_consensus.cc:740] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.887648  3500 raft_consensus.cc:493] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.890347  3439 consensus_queue.cc:260] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.890899  3500 raft_consensus.cc:515] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.895630  3439 ts_tablet_manager.cc:1434] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.003s
I20260709 12:23:12.900241  3438 tablet_bootstrap.cc:654] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.902115  3439 tablet_bootstrap.cc:492] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:12.903288  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:12.904012  3397 raft_consensus.cc:2468] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:12.906545  3289 leader_election.cc:304] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:12.909221  3539 raft_consensus.cc:2804] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:12.909673  3539 raft_consensus.cc:493] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.901383  3500 leader_election.cc:290] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.915333  3500 raft_consensus.cc:493] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.915997  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:12.917186  3500 raft_consensus.cc:515] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.917495  3247 raft_consensus.cc:2468] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:12.919548  3500 leader_election.cc:290] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.921882  3540 raft_consensus.cc:493] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.913888  3539 raft_consensus.cc:3060] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.922407  3540 raft_consensus.cc:515] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.922859  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbb481337ada4c83945e213fb117ee7f" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:12.925768  3540 leader_election.cc:290] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:12.928647  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd6cee8d33de46388ec38d683a789e94" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:12.929311  3248 raft_consensus.cc:2468] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:12.931221  3362 leader_election.cc:304] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:12.933089  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbb481337ada4c83945e213fb117ee7f" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:12.920342  3146 catalog_manager.cc:5697] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:12.935604  3544 raft_consensus.cc:2804] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:12.935803  3289 leader_election.cc:304] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:12.936105  3544 raft_consensus.cc:493] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.936471  3544 raft_consensus.cc:3060] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.937992  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd6cee8d33de46388ec38d683a789e94" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:12.938809  3320 raft_consensus.cc:2468] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:12.940161  3499 raft_consensus.cc:2749] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:12.941277  3481 raft_consensus.cc:493] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.946653  3440 tablet_bootstrap.cc:492] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:12.947280  3440 ts_tablet_manager.cc:1403] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.105s	user 0.026s	sys 0.027s
I20260709 12:23:12.942028  3481 raft_consensus.cc:515] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.946451  3539 raft_consensus.cc:515] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.950835  3539 leader_election.cc:290] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.951778  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:12.951731  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:12.952066  3440 raft_consensus.cc:359] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.952589  3397 raft_consensus.cc:3060] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.953194  3440 raft_consensus.cc:385] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.952585  3247 raft_consensus.cc:3060] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.953617  3440 raft_consensus.cc:740] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.937337  3500 raft_consensus.cc:493] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.955427  3440 consensus_queue.cc:260] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.969408  3440 ts_tablet_manager.cc:1434] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:23:12.972747  3440 tablet_bootstrap.cc:492] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:12.969668  3481 leader_election.cc:290] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.971484  3438 tablet_bootstrap.cc:492] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:12.974319  3438 ts_tablet_manager.cc:1403] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.093s	user 0.023s	sys 0.037s
I20260709 12:23:12.965492  3494 consensus_queue.cc:1048] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:23:12.971487  3397 raft_consensus.cc:2468] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:12.977720  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:12.977850  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:12.978580  3248 raft_consensus.cc:2468] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:12.978719  3396 raft_consensus.cc:2468] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:12.980721  3288 leader_election.cc:304] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:12.956408  3500 raft_consensus.cc:515] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.981956  3322 raft_consensus.cc:2468] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:12.982149  3438 raft_consensus.cc:359] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.983140  3438 raft_consensus.cc:385] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.983433  3438 raft_consensus.cc:740] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.981990  3481 raft_consensus.cc:2804] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:12.983996  3481 raft_consensus.cc:493] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.984395  3481 raft_consensus.cc:3060] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.984186  3438 consensus_queue.cc:260] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:12.986639  3438 ts_tablet_manager.cc:1434] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:23:12.987699  3438 tablet_bootstrap.cc:492] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:12.992515  3500 leader_election.cc:290] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:12.996845  3439 tablet_bootstrap.cc:654] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.998356  3540 consensus_queue.cc:1048] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:13.000502  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd6cee8d33de46388ec38d683a789e94" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:13.000828  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd6cee8d33de46388ec38d683a789e94" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:13.001485  3541 raft_consensus.cc:493] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.001689  3248 raft_consensus.cc:2468] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:13.002172  3541 raft_consensus.cc:515] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.003922  3288 leader_election.cc:304] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:13.004705  3541 leader_election.cc:290] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:13.005234  3500 raft_consensus.cc:2804] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:13.006117  3500 raft_consensus.cc:493] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:13.006637  3500 raft_consensus.cc:3060] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:13.010062  3438 tablet_bootstrap.cc:654] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:13.010988  3289 leader_election.cc:304] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:13.011063  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bde98a54e9f47a0b9afe40221bcb336" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:13.011947  3322 raft_consensus.cc:2468] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:13.013487  3364 leader_election.cc:304] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:13.015765  3540 raft_consensus.cc:2804] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:13.016185  3540 raft_consensus.cc:493] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:13.005772  3544 raft_consensus.cc:515] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.018316  3395 raft_consensus.cc:2393] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8525f77f43714a58bf158d872c1db3e4 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:13.017666  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bde98a54e9f47a0b9afe40221bcb336" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:13.016422  3539 raft_consensus.cc:2804] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:13.027084  3539 raft_consensus.cc:697] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:13.027803  3544 leader_election.cc:290] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:13.016889  3481 raft_consensus.cc:515] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.023994  3322 raft_consensus.cc:3060] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:13.030980  3499 consensus_queue.cc:1048] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:13.024420  3247 raft_consensus.cc:2468] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:13.024900  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd6cee8d33de46388ec38d683a789e94" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:13.039575  3246 raft_consensus.cc:3060] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:13.020740  3248 raft_consensus.cc:2468] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:13.028407  3539 consensus_queue.cc:237] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.016687  3540 raft_consensus.cc:3060] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:13.030115  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd6cee8d33de46388ec38d683a789e94" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:13.046787  3499 consensus_queue.cc:1048] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:13.043371  3500 raft_consensus.cc:515] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.071858  3481 leader_election.cc:290] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:13.079233  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:13.080097  3397 raft_consensus.cc:3060] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:13.095240  3544 consensus_queue.cc:1048] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:13.101992  3320 raft_consensus.cc:2393] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c93989d3dd634ef585d77b03354121a4 in current term 1: Already voted for candidate 8525f77f43714a58bf158d872c1db3e4 in this term.
I20260709 12:23:13.114146  3397 raft_consensus.cc:2468] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:13.116966  3440 tablet_bootstrap.cc:654] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:13.121289  3246 raft_consensus.cc:2468] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:13.123068  3362 leader_election.cc:304] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:13.123997  3544 raft_consensus.cc:2804] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:13.124526  3544 raft_consensus.cc:697] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:13.125350  3544 consensus_queue.cc:237] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.124022  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:13.128459  3248 raft_consensus.cc:3060] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:13.147938  3439 tablet_bootstrap.cc:492] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:13.148576  3439 ts_tablet_manager.cc:1403] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.247s	user 0.007s	sys 0.025s
W20260709 12:23:13.184729  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 56597.9us
I20260709 12:23:13.195005  3438 tablet_bootstrap.cc:492] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:13.195647  3438 ts_tablet_manager.cc:1403] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.208s	user 0.017s	sys 0.016s
I20260709 12:23:13.199596  3438 raft_consensus.cc:359] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.200641  3438 raft_consensus.cc:385] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:13.201017  3438 raft_consensus.cc:740] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.201911  3438 consensus_queue.cc:260] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.211794  3500 leader_election.cc:290] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:13.215312  3439 raft_consensus.cc:359] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.222471  3439 raft_consensus.cc:385] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:13.255427  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd6cee8d33de46388ec38d683a789e94" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:13.256768  3249 raft_consensus.cc:2393] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8525f77f43714a58bf158d872c1db3e4 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:13.222957  3439 raft_consensus.cc:740] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.263199  3439 consensus_queue.cc:260] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.266346  3439 ts_tablet_manager.cc:1434] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.117s	user 0.009s	sys 0.002s
W20260709 12:23:13.267100  3575 log.cc:927] Time spent T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:23:13.267396  3544 raft_consensus.cc:493] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.268239  3544 raft_consensus.cc:515] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.270777  3580 consensus_queue.cc:1048] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.065s
W20260709 12:23:13.281608  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 92693.8us
I20260709 12:23:13.282097  3544 leader_election.cc:290] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:13.283205  3248 raft_consensus.cc:2468] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:13.285346  3481 raft_consensus.cc:493] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.287500  3439 tablet_bootstrap.cc:492] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:13.289587  3144 catalog_manager.cc:5697] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:13.284910  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 77346.2us
I20260709 12:23:13.289932  3540 raft_consensus.cc:515] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.292543  3481 raft_consensus.cc:515] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.310627  3481 leader_election.cc:290] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:13.315073  3540 leader_election.cc:290] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:13.318912  3438 ts_tablet_manager.cc:1434] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.123s	user 0.002s	sys 0.017s
I20260709 12:23:13.319904  3438 tablet_bootstrap.cc:492] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:13.313278  3288 leader_election.cc:304] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:13.329198  3440 tablet_bootstrap.cc:492] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:13.330338  3580 raft_consensus.cc:2804] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:13.330906  3580 raft_consensus.cc:697] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:13.331727  3580 consensus_queue.cc:237] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.338604  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:13.352931  3145 catalog_manager.cc:5697] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:13.378269  3540 consensus_queue.cc:1048] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
W20260709 12:23:13.433979  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 120696us
I20260709 12:23:13.435038  3438 tablet_bootstrap.cc:654] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:13.448164  3440 ts_tablet_manager.cc:1403] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.476s	user 0.039s	sys 0.028s
I20260709 12:23:13.447477  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5d938460364390a17cada19308b314" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:13.466068  3288 leader_election.cc:343] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:13.468448  3591 consensus_queue.cc:1048] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:13.475440  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bde98a54e9f47a0b9afe40221bcb336" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:13.476204  3248 raft_consensus.cc:3060] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:13.477353  3439 tablet_bootstrap.cc:654] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:13.481758  3362 leader_election.cc:343] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:13.484038  3195 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:13.495237  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5d938460364390a17cada19308b314" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:13.496996  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bde98a54e9f47a0b9afe40221bcb336" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:13.497076  3248 raft_consensus.cc:2468] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:13.499938  3324 raft_consensus.cc:3060] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:13.508857  3440 raft_consensus.cc:359] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.509933  3440 raft_consensus.cc:385] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:13.510280  3440 raft_consensus.cc:740] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.514591  3440 consensus_queue.cc:260] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.519068  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd6cee8d33de46388ec38d683a789e94" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:13.524494  3362 leader_election.cc:304] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:13.525332  3440 ts_tablet_manager.cc:1434] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.077s	user 0.008s	sys 0.000s
I20260709 12:23:13.531466  3440 tablet_bootstrap.cc:492] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:13.532819  3438 tablet_bootstrap.cc:492] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:13.533346  3438 ts_tablet_manager.cc:1403] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.214s	user 0.013s	sys 0.016s
I20260709 12:23:13.533741  3324 raft_consensus.cc:2468] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:13.529680  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:13.537706  3544 raft_consensus.cc:2804] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:13.538345  3544 raft_consensus.cc:697] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:13.539292  3544 consensus_queue.cc:237] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.549264  3145 catalog_manager.cc:5697] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:13.550832  3591 consensus_queue.cc:1048] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:13.554600  3591 consensus_queue.cc:1048] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:23:13.554557  3364 leader_election.cc:343] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:13.557087  3364 leader_election.cc:304] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:13.558490  3540 raft_consensus.cc:2749] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:13.576491  3439 tablet_bootstrap.cc:492] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:13.577260  3439 ts_tablet_manager.cc:1403] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.291s	user 0.014s	sys 0.013s
I20260709 12:23:13.547174  3438 raft_consensus.cc:359] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.579370  3438 raft_consensus.cc:385] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:13.579689  3438 raft_consensus.cc:740] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.581557  3593 consensus_queue.cc:1048] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:13.583673  3438 consensus_queue.cc:260] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.585055  3540 consensus_queue.cc:1048] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:13.586189  3438 ts_tablet_manager.cc:1434] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.053s	user 0.004s	sys 0.003s
I20260709 12:23:13.586293  3289 leader_election.cc:304] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:13.587379  3438 tablet_bootstrap.cc:492] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:13.590083  3439 raft_consensus.cc:359] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.590809  3439 raft_consensus.cc:385] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:13.591145  3439 raft_consensus.cc:740] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.591992  3580 raft_consensus.cc:2749] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:13.603122  3438 tablet_bootstrap.cc:654] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:13.591876  3439 consensus_queue.cc:260] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.607991  3439 ts_tablet_manager.cc:1434] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.030s	user 0.002s	sys 0.003s
I20260709 12:23:13.610246  3439 tablet_bootstrap.cc:492] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:13.605423  3440 tablet_bootstrap.cc:654] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:13.611068  3438 tablet_bootstrap.cc:492] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:13.611227  3593 consensus_queue.cc:1048] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:23:13.613294  3289 leader_election.cc:343] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:13.614043  3438 ts_tablet_manager.cc:1403] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.027s	user 0.005s	sys 0.007s
I20260709 12:23:13.613890  3289 leader_election.cc:304] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:13.616950  3439 tablet_bootstrap.cc:654] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:13.616998  3593 raft_consensus.cc:2749] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:13.617700  3438 raft_consensus.cc:359] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.618568  3438 raft_consensus.cc:385] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:13.618853  3438 raft_consensus.cc:740] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.619781  3438 consensus_queue.cc:260] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.622309  3438 ts_tablet_manager.cc:1434] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 12:23:13.623342  3438 tablet_bootstrap.cc:492] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:13.621089  3144 catalog_manager.cc:5697] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:13.630028  3438 tablet_bootstrap.cc:654] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:13.646456  3439 tablet_bootstrap.cc:492] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:13.647054  3439 ts_tablet_manager.cc:1403] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.014s
I20260709 12:23:13.649966  3439 raft_consensus.cc:359] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.650722  3439 raft_consensus.cc:385] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:13.651008  3439 raft_consensus.cc:740] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.651726  3439 consensus_queue.cc:260] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.654090  3439 ts_tablet_manager.cc:1434] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:13.655030  3439 tablet_bootstrap.cc:492] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:13.661388  3439 tablet_bootstrap.cc:654] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:13.668844  3593 consensus_queue.cc:1048] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:13.748034  3440 tablet_bootstrap.cc:492] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:13.763441  3580 raft_consensus.cc:493] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.808964  3440 ts_tablet_manager.cc:1403] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.278s	user 0.009s	sys 0.008s
I20260709 12:23:13.808614  3580 raft_consensus.cc:515] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.791869  3593 raft_consensus.cc:493] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.818007  3593 raft_consensus.cc:515] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.820662  3593 leader_election.cc:290] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:13.821837  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fddcaae9194bef90d1047a518e45a6" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:13.822672  3244 raft_consensus.cc:2468] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:13.823051  3580 leader_election.cc:290] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:13.810694  3438 tablet_bootstrap.cc:492] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:13.824360  3438 ts_tablet_manager.cc:1403] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.201s	user 0.066s	sys 0.093s
I20260709 12:23:13.824520  3288 leader_election.cc:304] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:13.826185  3580 raft_consensus.cc:2804] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:13.826619  3580 raft_consensus.cc:493] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:13.826972  3580 raft_consensus.cc:3060] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:13.828872  3438 raft_consensus.cc:359] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.829751  3438 raft_consensus.cc:385] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:13.835364  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:13.835425  3544 consensus_queue.cc:1048] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:13.836294  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fddcaae9194bef90d1047a518e45a6" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:13.836587  3542 consensus_queue.cc:1048] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:13.839154  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:13.841971  3438 raft_consensus.cc:740] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.845582  3397 raft_consensus.cc:2468] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
W20260709 12:23:13.845822  3288 leader_election.cc:343] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:13.847525  3438 consensus_queue.cc:260] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.855584  3438 ts_tablet_manager.cc:1434] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 12:23:13.856616  3438 tablet_bootstrap.cc:492] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:13.858592  3440 raft_consensus.cc:359] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.865993  3440 raft_consensus.cc:385] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:13.861334  3439 tablet_bootstrap.cc:492] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:13.867321  3439 ts_tablet_manager.cc:1403] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.212s	user 0.065s	sys 0.101s
I20260709 12:23:13.863068  3438 tablet_bootstrap.cc:654] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:13.864446  3289 leader_election.cc:343] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:13.859118  3580 raft_consensus.cc:515] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.870743  3439 raft_consensus.cc:359] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.874245  3439 raft_consensus.cc:385] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:13.872566  3580 leader_election.cc:290] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:13.873076  3289 leader_election.cc:304] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:13.875442  3439 raft_consensus.cc:740] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.876864  3580 raft_consensus.cc:2749] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:13.877600  3439 consensus_queue.cc:260] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.874047  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fddcaae9194bef90d1047a518e45a6" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:13.880241  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fddcaae9194bef90d1047a518e45a6" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:13.894804  3244 raft_consensus.cc:3060] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:13.895082  3394 raft_consensus.cc:3060] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:13.871632  3440 raft_consensus.cc:740] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.903282  3440 consensus_queue.cc:260] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.907560  3439 ts_tablet_manager.cc:1434] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 12:23:13.909078  3439 tablet_bootstrap.cc:492] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:13.914122  3244 raft_consensus.cc:2468] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:13.919929  3288 leader_election.cc:304] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:13.921736  3481 raft_consensus.cc:2804] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:13.922312  3481 raft_consensus.cc:697] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:13.922477  3394 raft_consensus.cc:2468] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:13.923218  3481 consensus_queue.cc:237] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.926648  3440 ts_tablet_manager.cc:1434] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.087s	user 0.007s	sys 0.001s
I20260709 12:23:13.927870  3440 tablet_bootstrap.cc:492] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:13.930542  3438 tablet_bootstrap.cc:492] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:13.931285  3438 ts_tablet_manager.cc:1403] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.075s	user 0.023s	sys 0.007s
I20260709 12:23:13.934020  3438 raft_consensus.cc:359] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.934957  3438 raft_consensus.cc:385] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:13.935274  3438 raft_consensus.cc:740] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.936509  3438 consensus_queue.cc:260] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.940898  3438 ts_tablet_manager.cc:1434] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:23:13.942145  3438 tablet_bootstrap.cc:492] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:13.960137  3438 tablet_bootstrap.cc:654] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:13.963500  3439 tablet_bootstrap.cc:654] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:13.969300  3440 tablet_bootstrap.cc:654] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:13.971443  3438 tablet_bootstrap.cc:492] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:13.972070  3438 ts_tablet_manager.cc:1403] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.030s	user 0.005s	sys 0.008s
I20260709 12:23:13.974785  3438 raft_consensus.cc:359] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.975683  3438 raft_consensus.cc:385] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:13.976006  3438 raft_consensus.cc:740] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:13.976725  3438 consensus_queue.cc:260] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:13.978899  3438 ts_tablet_manager.cc:1434] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 12:23:13.978736  3144 catalog_manager.cc:5697] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:13.981534  3438 tablet_bootstrap.cc:492] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:13.989498  3438 tablet_bootstrap.cc:654] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:13.992789  3544 consensus_queue.cc:1048] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:14.001540  3438 tablet_bootstrap.cc:492] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:14.002169  3438 ts_tablet_manager.cc:1403] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.000s
I20260709 12:23:14.005225  3438 raft_consensus.cc:359] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.006026  3438 raft_consensus.cc:385] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.006381  3438 raft_consensus.cc:740] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.007176  3438 consensus_queue.cc:260] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.009356  3438 ts_tablet_manager.cc:1434] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:14.013422  3438 tablet_bootstrap.cc:492] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:14.014792  3494 raft_consensus.cc:493] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.015303  3494 raft_consensus.cc:515] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.018951  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5d938460364390a17cada19308b314" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:14.019673  3394 raft_consensus.cc:2468] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:14.020502  3438 tablet_bootstrap.cc:654] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.017473  3494 leader_election.cc:290] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:14.021844  3214 leader_election.cc:304] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:14.023196  3494 raft_consensus.cc:2804] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:14.023633  3494 raft_consensus.cc:493] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.025701  3494 raft_consensus.cc:3060] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.027544  3481 consensus_queue.cc:1048] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:14.027953  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5d938460364390a17cada19308b314" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:14.028601  3323 raft_consensus.cc:2468] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:14.034636  3542 raft_consensus.cc:493] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.035331  3542 raft_consensus.cc:515] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.039304  3462 consensus_queue.cc:1048] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:14.048096  3542 leader_election.cc:290] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:14.049666  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:14.050563  3323 raft_consensus.cc:2468] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:14.052166  3364 leader_election.cc:304] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:14.054986  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:14.062108  3542 raft_consensus.cc:2804] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:14.062669  3542 raft_consensus.cc:493] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.063131  3542 raft_consensus.cc:3060] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:14.082845  3362 leader_election.cc:343] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:14.111704  3481 consensus_queue.cc:1048] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:14.136703  3439 tablet_bootstrap.cc:492] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:14.137336  3439 ts_tablet_manager.cc:1403] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.229s	user 0.017s	sys 0.003s
I20260709 12:23:14.145445  3439 raft_consensus.cc:359] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.146584  3439 raft_consensus.cc:385] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.146938  3439 raft_consensus.cc:740] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.148264  3439 consensus_queue.cc:260] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.157608  3439 ts_tablet_manager.cc:1434] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:23:14.159461  3439 tablet_bootstrap.cc:492] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:14.161605  3440 tablet_bootstrap.cc:492] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:14.162130  3438 tablet_bootstrap.cc:492] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:14.162330  3440 ts_tablet_manager.cc:1403] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.235s	user 0.063s	sys 0.080s
I20260709 12:23:14.162855  3438 ts_tablet_manager.cc:1403] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.150s	user 0.070s	sys 0.068s
I20260709 12:23:14.174399  3438 raft_consensus.cc:359] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.175172  3438 raft_consensus.cc:385] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.175691  3438 raft_consensus.cc:740] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.176409  3438 consensus_queue.cc:260] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.178745  3438 ts_tablet_manager.cc:1434] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:23:14.179728  3438 tablet_bootstrap.cc:492] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:14.192190  3494 raft_consensus.cc:515] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.195406  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5d938460364390a17cada19308b314" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:14.196039  3323 raft_consensus.cc:3060] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.201121  3440 raft_consensus.cc:359] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.201901  3440 raft_consensus.cc:385] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.202165  3440 raft_consensus.cc:740] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.202802  3440 consensus_queue.cc:260] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.206000  3580 raft_consensus.cc:493] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.206513  3580 raft_consensus.cc:515] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.208545  3580 leader_election.cc:290] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:14.209688  3439 tablet_bootstrap.cc:654] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.209684  3440 ts_tablet_manager.cc:1434] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 12:23:14.210260  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:14.210888  3440 tablet_bootstrap.cc:492] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
W20260709 12:23:14.211722  3289 leader_election.cc:343] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:14.213778  3245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:14.215250  3288 leader_election.cc:343] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:14.215734  3288 leader_election.cc:304] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:14.217116  3580 raft_consensus.cc:2749] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:14.216892  3542 raft_consensus.cc:515] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.220217  3323 raft_consensus.cc:2468] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:14.221557  3215 leader_election.cc:304] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:14.222601  3443 raft_consensus.cc:2804] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:14.223191  3443 raft_consensus.cc:697] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:14.224211  3443 consensus_queue.cc:237] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.215353  3438 tablet_bootstrap.cc:654] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.226327  3580 raft_consensus.cc:493] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.226984  3580 raft_consensus.cc:515] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.228050  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:14.225267  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5d938460364390a17cada19308b314" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:14.228709  3324 raft_consensus.cc:3060] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.228785  3397 raft_consensus.cc:3060] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.230865  3440 tablet_bootstrap.cc:654] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.223242  3494 leader_election.cc:290] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:14.233042  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4a1156be8e3418790f41cd380136b93" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:14.233811  3580 leader_election.cc:290] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:23:14.234903  3288 leader_election.cc:343] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:14.237447  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:14.238920  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4a1156be8e3418790f41cd380136b93" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
W20260709 12:23:14.240597  3289 leader_election.cc:343] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:14.241202  3289 leader_election.cc:304] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:14.240743  3542 leader_election.cc:290] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:14.241153  3324 raft_consensus.cc:2468] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:14.242857  3580 raft_consensus.cc:2749] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:14.243693  3364 leader_election.cc:304] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:14.244709  3462 raft_consensus.cc:2804] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:14.245230  3462 raft_consensus.cc:697] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:14.248248  3439 tablet_bootstrap.cc:492] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:14.248986  3439 ts_tablet_manager.cc:1403] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.090s	user 0.010s	sys 0.002s
I20260709 12:23:14.246411  3462 consensus_queue.cc:237] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.253454  3439 raft_consensus.cc:359] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.254913  3439 raft_consensus.cc:385] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.255308  3439 raft_consensus.cc:740] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.256258  3438 tablet_bootstrap.cc:492] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:14.256846  3438 ts_tablet_manager.cc:1403] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.077s	user 0.012s	sys 0.000s
I20260709 12:23:14.256412  3439 consensus_queue.cc:260] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.260003  3438 raft_consensus.cc:359] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:14.260617  3362 leader_election.cc:343] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:14.264323  3440 tablet_bootstrap.cc:492] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:14.264912  3440 ts_tablet_manager.cc:1403] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.054s	user 0.023s	sys 0.014s
I20260709 12:23:14.260907  3438 raft_consensus.cc:385] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.265753  3438 raft_consensus.cc:740] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.266610  3438 consensus_queue.cc:260] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.267995  3440 raft_consensus.cc:359] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.268709  3440 raft_consensus.cc:385] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.268821  3438 ts_tablet_manager.cc:1434] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:23:14.269796  3438 tablet_bootstrap.cc:492] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:14.260967  3439 ts_tablet_manager.cc:1434] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.004s
I20260709 12:23:14.274551  3439 tablet_bootstrap.cc:492] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:14.268993  3440 raft_consensus.cc:740] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.276129  3440 consensus_queue.cc:260] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.278774  3440 ts_tablet_manager.cc:1434] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.006s
I20260709 12:23:14.280339  3439 tablet_bootstrap.cc:654] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.280411  3440 tablet_bootstrap.cc:492] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:14.286324  3438 tablet_bootstrap.cc:654] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.287976  3440 tablet_bootstrap.cc:654] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.282442  3397 raft_consensus.cc:2468] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:14.292840  3580 raft_consensus.cc:493] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.301923  3580 raft_consensus.cc:515] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.307474  3144 catalog_manager.cc:5697] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:14.309505  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d373e8b022274a08b08f00c0057ae3d3" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:14.309860  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d373e8b022274a08b08f00c0057ae3d3" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
W20260709 12:23:14.312294  3288 leader_election.cc:343] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:14.310241  3580 leader_election.cc:290] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:23:14.313258  3289 leader_election.cc:343] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:14.316275  3289 leader_election.cc:304] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:14.320923  3542 raft_consensus.cc:493] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.321491  3542 raft_consensus.cc:515] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.326858  3438 tablet_bootstrap.cc:492] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:14.325191  3146 catalog_manager.cc:5697] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:14.325918  3588 raft_consensus.cc:493] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.327507  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:14.327545  3438 ts_tablet_manager.cc:1403] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.058s	user 0.012s	sys 0.011s
I20260709 12:23:14.328436  3324 raft_consensus.cc:2468] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:14.328322  3588 raft_consensus.cc:515] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.330235  3462 raft_consensus.cc:493] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.330765  3462 raft_consensus.cc:515] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.332261  3588 leader_election.cc:290] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:14.334565  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba9db97635f4d28bf0f3079edf45a77" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:14.337182  3288 leader_election.cc:343] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:14.325784  3542 leader_election.cc:290] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:14.332681  3462 leader_election.cc:290] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:14.332401  3364 leader_election.cc:304] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:14.338762  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:14.339522  3248 raft_consensus.cc:2468] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:14.340214  3462 raft_consensus.cc:2804] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:14.340906  3462 raft_consensus.cc:493] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.342007  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba9db97635f4d28bf0f3079edf45a77" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:14.343020  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba9db97635f4d28bf0f3079edf45a77" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:14.344677  3462 raft_consensus.cc:3060] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.344761  3324 raft_consensus.cc:2468] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
W20260709 12:23:14.346900  3362 leader_election.cc:343] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:14.329831  3580 raft_consensus.cc:2749] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:14.351521  3439 tablet_bootstrap.cc:492] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:14.354797  3494 raft_consensus.cc:493] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.354985  3439 ts_tablet_manager.cc:1403] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.081s	user 0.017s	sys 0.013s
I20260709 12:23:14.351972  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba9db97635f4d28bf0f3079edf45a77" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:14.355316  3494 raft_consensus.cc:515] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.362931  3494 leader_election.cc:290] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:14.362712  3439 raft_consensus.cc:359] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.364831  3439 raft_consensus.cc:385] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.365211  3439 raft_consensus.cc:740] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.365262  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:14.365726  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:14.366019  3439 consensus_queue.cc:260] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.366624  3323 raft_consensus.cc:2393] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cdaa0234c64450eb4ce4500b51b8a81 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:14.368392  3439 ts_tablet_manager.cc:1434] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:23:14.358897  3364 leader_election.cc:304] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:14.369663  3542 raft_consensus.cc:2804] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:14.370136  3542 raft_consensus.cc:493] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.370512  3542 raft_consensus.cc:3060] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.370569  3439 tablet_bootstrap.cc:492] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:14.365793  3438 raft_consensus.cc:359] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.375854  3438 raft_consensus.cc:385] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.376148  3438 raft_consensus.cc:740] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.376829  3438 consensus_queue.cc:260] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.376830  3440 tablet_bootstrap.cc:492] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:14.378353  3440 ts_tablet_manager.cc:1403] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.098s	user 0.031s	sys 0.022s
I20260709 12:23:14.378599  3438 ts_tablet_manager.cc:1434] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.000s
I20260709 12:23:14.379551  3438 tablet_bootstrap.cc:492] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:14.381747  3440 raft_consensus.cc:359] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.382474  3440 raft_consensus.cc:385] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.382750  3440 raft_consensus.cc:740] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.385463  3438 tablet_bootstrap.cc:654] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.385914  3588 raft_consensus.cc:493] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.386467  3588 raft_consensus.cc:515] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.386499  3440 consensus_queue.cc:260] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.389006  3588 leader_election.cc:290] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:14.389966  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d7a917f21741e9827c30e5433a38e3" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:14.390651  3248 raft_consensus.cc:2468] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:14.392051  3288 leader_election.cc:304] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:14.394428  3481 raft_consensus.cc:2804] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:14.394847  3481 raft_consensus.cc:493] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.395234  3481 raft_consensus.cc:3060] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.395298  3542 raft_consensus.cc:515] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.396787  3397 raft_consensus.cc:2393] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8525f77f43714a58bf158d872c1db3e4 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:14.398775  3440 ts_tablet_manager.cc:1434] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.005s
I20260709 12:23:14.399856  3440 tablet_bootstrap.cc:492] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:14.406644  3214 leader_election.cc:304] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:14.408010  3494 raft_consensus.cc:3060] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.408798  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d7a917f21741e9827c30e5433a38e3" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:14.409477  3588 consensus_queue.cc:1048] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:14.410306  3289 leader_election.cc:304] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:14.417382  3588 raft_consensus.cc:3060] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.424363  3494 raft_consensus.cc:2749] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:14.425220  3540 raft_consensus.cc:493] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.425740  3540 raft_consensus.cc:515] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.386019  3439 tablet_bootstrap.cc:654] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:14.421808  3289 leader_election.cc:343] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:14.405131  3462 raft_consensus.cc:515] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.444464  3542 leader_election.cc:290] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:14.445961  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba9db97635f4d28bf0f3079edf45a77" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:14.446995  3540 leader_election.cc:290] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:14.451308  3440 tablet_bootstrap.cc:654] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.461746  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caed93336fbb415c82acd602adce7969" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:14.470952  3462 leader_election.cc:290] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:23:14.474825  3362 leader_election.cc:343] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:14.476732  3362 leader_election.cc:343] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:14.481113  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 59342.4us
I20260709 12:23:14.511118  3439 tablet_bootstrap.cc:492] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:14.514501  3438 tablet_bootstrap.cc:492] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:14.642724  3438 ts_tablet_manager.cc:1403] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.263s	user 0.062s	sys 0.058s
I20260709 12:23:14.646584  3438 raft_consensus.cc:359] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.549980  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caed93336fbb415c82acd602adce7969" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:14.651320  3322 raft_consensus.cc:2468] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:14.648239  3439 ts_tablet_manager.cc:1403] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.278s	user 0.089s	sys 0.043s
I20260709 12:23:14.651925  3438 raft_consensus.cc:385] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.541811  3481 raft_consensus.cc:515] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.549548  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba9db97635f4d28bf0f3079edf45a77" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:14.589855  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:14.653188  3438 raft_consensus.cc:740] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.663443  3438 consensus_queue.cc:260] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.515640  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:14.663324  3320 raft_consensus.cc:3060] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.675259  3438 ts_tablet_manager.cc:1434] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.003s
I20260709 12:23:14.676579  3438 tablet_bootstrap.cc:492] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:14.659843  3462 consensus_queue.cc:1048] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:23:14.560562  3580 consensus_queue.cc:1048] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.117s
I20260709 12:23:14.687529  3439 raft_consensus.cc:359] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.689191  3439 raft_consensus.cc:385] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.689464  3439 raft_consensus.cc:740] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.683089  3248 raft_consensus.cc:3060] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.691869  3439 consensus_queue.cc:260] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.698475  3439 ts_tablet_manager.cc:1434] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
W20260709 12:23:14.685586  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 44050us
I20260709 12:23:14.705873  3439 tablet_bootstrap.cc:492] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:14.707231  3481 leader_election.cc:290] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:14.713804  3440 tablet_bootstrap.cc:492] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:14.714347  3440 ts_tablet_manager.cc:1403] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.315s	user 0.031s	sys 0.017s
I20260709 12:23:14.715978  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d7a917f21741e9827c30e5433a38e3" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:14.716621  3249 raft_consensus.cc:3060] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.720690  3440 raft_consensus.cc:359] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.721848  3440 raft_consensus.cc:385] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.722165  3440 raft_consensus.cc:740] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.725730  3249 raft_consensus.cc:2468] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:14.726015  3320 raft_consensus.cc:2468] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:14.735558  3440 consensus_queue.cc:260] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.752581  3288 leader_election.cc:304] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:14.759310  3440 ts_tablet_manager.cc:1434] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.004s
I20260709 12:23:14.760854  3440 tablet_bootstrap.cc:492] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:14.757197  3481 raft_consensus.cc:2804] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:14.763504  3542 raft_consensus.cc:493] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.771061  3542 raft_consensus.cc:515] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.773083  3542 leader_election.cc:290] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:14.773757  3438 tablet_bootstrap.cc:654] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.767441  3364 leader_election.cc:304] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:14.774850  3481 raft_consensus.cc:697] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:14.776262  3542 raft_consensus.cc:2804] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:14.782305  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4a1156be8e3418790f41cd380136b93" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:14.785920  3542 raft_consensus.cc:493] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.786432  3542 raft_consensus.cc:3060] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:14.787138  3362 leader_election.cc:343] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:14.783032  3481 consensus_queue.cc:237] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.789499  3364 leader_election.cc:304] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:14.793256  3462 raft_consensus.cc:2804] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:14.793911  3462 raft_consensus.cc:697] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:14.776564  3248 raft_consensus.cc:2468] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:14.780671  3588 raft_consensus.cc:2749] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:14.810029  3462 consensus_queue.cc:237] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.810702  3439 tablet_bootstrap.cc:654] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.816527  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4a1156be8e3418790f41cd380136b93" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:14.817305  3323 raft_consensus.cc:2468] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:14.812052  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d7a917f21741e9827c30e5433a38e3" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:14.833778  3494 consensus_queue.cc:1048] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:23:14.835387  3289 leader_election.cc:343] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:14.836663  3494 consensus_queue.cc:1048] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:14.837468  3438 tablet_bootstrap.cc:492] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:14.840246  3438 ts_tablet_manager.cc:1403] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.164s	user 0.013s	sys 0.008s
I20260709 12:23:14.845532  3438 raft_consensus.cc:359] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.846597  3438 raft_consensus.cc:385] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.847179  3438 raft_consensus.cc:740] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.847921  3438 consensus_queue.cc:260] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.848695  3544 consensus_queue.cc:1048] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:14.849123  3324 raft_consensus.cc:2468] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:14.856863  3438 ts_tablet_manager.cc:1434] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:23:14.858143  3438 tablet_bootstrap.cc:492] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
W20260709 12:23:14.878786  3642 log.cc:927] Time spent T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:23:14.860904  3364 leader_election.cc:304] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:14.895337  3144 catalog_manager.cc:5697] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:14.889897  3462 raft_consensus.cc:2804] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:14.905395  3462 raft_consensus.cc:493] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.907786  3440 tablet_bootstrap.cc:654] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.908998  3145 catalog_manager.cc:5697] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:14.905963  3462 raft_consensus.cc:3060] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.924551  3438 tablet_bootstrap.cc:654] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:14.915087  3195 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:14.908070  3542 raft_consensus.cc:515] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.916779  3439 tablet_bootstrap.cc:492] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:14.964996  3439 ts_tablet_manager.cc:1403] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.265s	user 0.041s	sys 0.010s
I20260709 12:23:14.946179  3364 leader_election.cc:304] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:14.976696  3440 tablet_bootstrap.cc:492] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:14.977276  3653 raft_consensus.cc:2804] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:14.978434  3440 ts_tablet_manager.cc:1403] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.218s	user 0.021s	sys 0.002s
I20260709 12:23:14.979665  3653 raft_consensus.cc:697] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:14.982658  3653 consensus_queue.cc:237] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.992091  3439 raft_consensus.cc:359] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.992871  3439 raft_consensus.cc:385] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.993194  3439 raft_consensus.cc:740] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.993958  3439 consensus_queue.cc:260] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:14.998605  3439 ts_tablet_manager.cc:1434] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 12:23:15.000020  3439 tablet_bootstrap.cc:492] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:15.014156  3481 raft_consensus.cc:493] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.023629  3542 leader_election.cc:290] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:15.017709  3588 raft_consensus.cc:493] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.046847  3438 tablet_bootstrap.cc:492] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:15.047489  3438 ts_tablet_manager.cc:1403] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.190s	user 0.023s	sys 0.004s
I20260709 12:23:15.047159  3440 raft_consensus.cc:359] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.047966  3440 raft_consensus.cc:385] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.048293  3440 raft_consensus.cc:740] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.042150  3588 raft_consensus.cc:515] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.049010  3440 consensus_queue.cc:260] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.052605  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caed93336fbb415c82acd602adce7969" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:15.067019  3323 raft_consensus.cc:3060] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.069033  3443 raft_consensus.cc:493] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.069743  3443 raft_consensus.cc:515] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.037046  3481 raft_consensus.cc:515] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.071870  3443 leader_election.cc:290] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:15.074326  3481 leader_election.cc:290] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:15.075611  3321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caed93336fbb415c82acd602adce7969" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:15.077836  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:15.089728  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:15.090123  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caed93336fbb415c82acd602adce7969" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:15.090891  3246 raft_consensus.cc:3060] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:15.093288  3288 leader_election.cc:343] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.094177  3323 raft_consensus.cc:2468] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:15.095610  3245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:15.096413  3289 leader_election.cc:343] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.096601  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caed93336fbb415c82acd602adce7969" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:15.099990  3396 raft_consensus.cc:2393] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cdaa0234c64450eb4ce4500b51b8a81 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:15.101384  3440 ts_tablet_manager.cc:1434] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.123s	user 0.005s	sys 0.003s
I20260709 12:23:15.102656  3440 tablet_bootstrap.cc:492] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:15.104357  3214 leader_election.cc:304] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:15.105389  3588 leader_election.cc:290] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:15.106433  3462 raft_consensus.cc:515] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.106998  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:15.109504  3439 tablet_bootstrap.cc:654] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.110175  3364 leader_election.cc:304] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:15.110872  3245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4a1156be8e3418790f41cd380136b93" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:15.111317  3544 raft_consensus.cc:2804] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:15.111877  3544 raft_consensus.cc:697] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
W20260709 12:23:15.112385  3362 leader_election.cc:343] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.112838  3544 consensus_queue.cc:237] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:15.099193  3288 leader_election.cc:343] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.114472  3246 raft_consensus.cc:2468] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:15.115087  3664 raft_consensus.cc:2749] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:15.115113  3462 leader_election.cc:290] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:15.118978  3438 raft_consensus.cc:359] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.121910  3438 raft_consensus.cc:385] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.122231  3438 raft_consensus.cc:740] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.122918  3438 consensus_queue.cc:260] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.125267  3438 ts_tablet_manager.cc:1434] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.077s	user 0.019s	sys 0.009s
I20260709 12:23:15.127581  3288 leader_election.cc:304] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:15.128569  3438 tablet_bootstrap.cc:492] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
W20260709 12:23:15.131829  3289 leader_election.cc:343] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.132428  3289 leader_election.cc:304] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:15.134176  3481 raft_consensus.cc:2749] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:15.131889  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4a1156be8e3418790f41cd380136b93" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:15.140100  3323 raft_consensus.cc:3060] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.134873  3481 raft_consensus.cc:2749] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:15.163393  3542 raft_consensus.cc:493] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.164072  3542 raft_consensus.cc:515] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.166504  3542 leader_election.cc:290] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:15.155367  3544 raft_consensus.cc:493] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.168391  3544 raft_consensus.cc:515] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.159219  3440 tablet_bootstrap.cc:654] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.169914  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d7a917f21741e9827c30e5433a38e3" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:15.172465  3245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d373e8b022274a08b08f00c0057ae3d3" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:15.171931  3144 catalog_manager.cc:5697] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:15.173130  3246 raft_consensus.cc:2393] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c93989d3dd634ef585d77b03354121a4 in current term 1: Already voted for candidate 8525f77f43714a58bf158d872c1db3e4 in this term.
I20260709 12:23:15.173238  3245 raft_consensus.cc:2468] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:15.173889  3144 catalog_manager.cc:5697] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:15.175037  3362 leader_election.cc:304] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:15.177173  3542 raft_consensus.cc:2804] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:15.177603  3542 raft_consensus.cc:493] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.178059  3542 raft_consensus.cc:3060] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.174377  3544 leader_election.cc:290] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:15.186329  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d7a917f21741e9827c30e5433a38e3" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:15.188247  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d373e8b022274a08b08f00c0057ae3d3" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:15.188884  3324 raft_consensus.cc:2468] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:15.195309  3364 leader_election.cc:304] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:15.196565  3544 raft_consensus.cc:3060] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.203035  3438 tablet_bootstrap.cc:654] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.221767  3542 raft_consensus.cc:515] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.224606  3481 consensus_queue.cc:1048] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:15.230229  3323 raft_consensus.cc:2468] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:15.225099  3542 leader_election.cc:290] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:15.226871  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d373e8b022274a08b08f00c0057ae3d3" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:15.235385  3324 raft_consensus.cc:3060] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.226871  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d373e8b022274a08b08f00c0057ae3d3" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:15.237282  3246 raft_consensus.cc:3060] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.238448  3364 leader_election.cc:304] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:15.239377  3542 raft_consensus.cc:2804] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:15.239908  3542 raft_consensus.cc:697] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:15.240739  3542 consensus_queue.cc:237] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.255394  3440 tablet_bootstrap.cc:492] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:15.256223  3440 ts_tablet_manager.cc:1403] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.154s	user 0.015s	sys 0.034s
I20260709 12:23:15.260821  3439 tablet_bootstrap.cc:492] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:15.261591  3439 ts_tablet_manager.cc:1403] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.262s	user 0.010s	sys 0.005s
I20260709 12:23:15.269055  3439 raft_consensus.cc:359] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.271427  3439 raft_consensus.cc:385] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.271811  3439 raft_consensus.cc:740] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.278687  3324 raft_consensus.cc:2468] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:15.292112  3364 leader_election.cc:304] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:15.293080  3653 consensus_queue.cc:1048] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:15.293509  3542 raft_consensus.cc:2804] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:15.294057  3542 raft_consensus.cc:697] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:15.279331  3440 raft_consensus.cc:359] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.295262  3440 raft_consensus.cc:385] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.295564  3440 raft_consensus.cc:740] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.296211  3440 consensus_queue.cc:260] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.297322  3542 consensus_queue.cc:237] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.302940  3440 ts_tablet_manager.cc:1434] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.046s	user 0.004s	sys 0.006s
I20260709 12:23:15.303996  3440 tablet_bootstrap.cc:492] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:15.281551  3246 raft_consensus.cc:2468] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:15.282068  3544 raft_consensus.cc:2749] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:15.287560  3439 consensus_queue.cc:260] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.311982  3439 ts_tablet_manager.cc:1434] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.000s
I20260709 12:23:15.313176  3439 tablet_bootstrap.cc:492] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:15.289460  3144 catalog_manager.cc:5697] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:15.288450  3438 tablet_bootstrap.cc:492] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:15.321998  3438 ts_tablet_manager.cc:1403] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.196s	user 0.024s	sys 0.025s
I20260709 12:23:15.324085  3588 consensus_queue.cc:1048] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:15.324818  3438 raft_consensus.cc:359] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.325599  3438 raft_consensus.cc:385] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.325917  3438 raft_consensus.cc:740] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.328130  3439 tablet_bootstrap.cc:654] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.326604  3438 consensus_queue.cc:260] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.339301  3438 ts_tablet_manager.cc:1434] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.002s
I20260709 12:23:15.340624  3438 tablet_bootstrap.cc:492] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:15.344491  3440 tablet_bootstrap.cc:654] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.356818  3438 tablet_bootstrap.cc:654] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.356834  3147 catalog_manager.cc:5697] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:15.361447  3439 tablet_bootstrap.cc:492] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:15.361967  3439 ts_tablet_manager.cc:1403] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.002s
I20260709 12:23:15.364401  3439 raft_consensus.cc:359] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.365295  3439 raft_consensus.cc:385] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.365562  3439 raft_consensus.cc:740] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.366436  3439 consensus_queue.cc:260] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.371981  3439 ts_tablet_manager.cc:1434] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:23:15.373147  3439 tablet_bootstrap.cc:492] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:15.380931  3439 tablet_bootstrap.cc:654] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.384382  3440 tablet_bootstrap.cc:492] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:15.384944  3440 ts_tablet_manager.cc:1403] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.081s	user 0.013s	sys 0.004s
I20260709 12:23:15.388366  3440 raft_consensus.cc:359] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.389045  3440 raft_consensus.cc:385] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.389315  3440 raft_consensus.cc:740] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.389986  3440 consensus_queue.cc:260] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.393215  3438 tablet_bootstrap.cc:492] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:15.393795  3438 ts_tablet_manager.cc:1403] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.053s	user 0.016s	sys 0.000s
I20260709 12:23:15.396641  3438 raft_consensus.cc:359] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.397332  3438 raft_consensus.cc:385] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.397600  3438 raft_consensus.cc:740] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.398893  3588 raft_consensus.cc:493] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.399453  3588 raft_consensus.cc:515] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.403908  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36377e446734493c8f3b1ff53d40d1cc" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:15.407109  3288 leader_election.cc:343] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.411279  3440 ts_tablet_manager.cc:1434] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.002s
I20260709 12:23:15.412156  3438 consensus_queue.cc:260] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.413564  3653 consensus_queue.cc:1048] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:15.414161  3588 leader_election.cc:290] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:15.414619  3438 ts_tablet_manager.cc:1434] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.002s
I20260709 12:23:15.415656  3438 tablet_bootstrap.cc:492] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:15.416837  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36377e446734493c8f3b1ff53d40d1cc" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:15.419124  3588 raft_consensus.cc:493] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:15.419436  3289 leader_election.cc:343] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.420127  3289 leader_election.cc:304] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:15.421275  3640 raft_consensus.cc:2749] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:15.419755  3588 raft_consensus.cc:515] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.432041  3588 leader_election.cc:290] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:15.432509  3440 tablet_bootstrap.cc:492] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:15.432478  3245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02f86799bc54984b41ede0f7db226e6" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:15.434656  3288 leader_election.cc:343] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.437831  3653 raft_consensus.cc:493] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.438432  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02f86799bc54984b41ede0f7db226e6" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:15.438462  3653 raft_consensus.cc:515] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.442128  3653 leader_election.cc:290] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:15.443145  3245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:15.444305  3289 leader_election.cc:343] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:15.444799  3362 leader_election.cc:343] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.445926  3440 tablet_bootstrap.cc:654] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.446069  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:15.446766  3323 raft_consensus.cc:2468] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:15.444809  3289 leader_election.cc:304] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:15.448014  3438 tablet_bootstrap.cc:654] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.448817  3364 leader_election.cc:304] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:15.449993  3653 raft_consensus.cc:2804] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:15.450397  3653 raft_consensus.cc:493] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.450765  3653 raft_consensus.cc:3060] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.448822  3588 raft_consensus.cc:2749] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:15.466986  3653 raft_consensus.cc:515] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.468832  3440 tablet_bootstrap.cc:492] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:15.469462  3440 ts_tablet_manager.cc:1403] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.037s	user 0.004s	sys 0.013s
I20260709 12:23:15.470353  3245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
W20260709 12:23:15.472043  3362 leader_election.cc:343] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.472481  3440 raft_consensus.cc:359] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.472678  3462 consensus_queue.cc:1048] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:15.475215  3440 raft_consensus.cc:385] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.475584  3440 raft_consensus.cc:740] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.476294  3440 consensus_queue.cc:260] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.485292  3440 ts_tablet_manager.cc:1434] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.002s
I20260709 12:23:15.485682  3439 tablet_bootstrap.cc:492] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:15.486265  3439 ts_tablet_manager.cc:1403] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.113s	user 0.015s	sys 0.009s
I20260709 12:23:15.487124  3440 tablet_bootstrap.cc:492] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:15.493180  3245 raft_consensus.cc:3060] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.506728  3439 raft_consensus.cc:359] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.507527  3439 raft_consensus.cc:385] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.507807  3439 raft_consensus.cc:740] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.508522  3439 consensus_queue.cc:260] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.510187  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:15.510962  3323 raft_consensus.cc:3060] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.504539  3653 leader_election.cc:290] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:15.516115  3653 consensus_queue.cc:1048] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:15.518450  3439 ts_tablet_manager.cc:1434] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.002s
I20260709 12:23:15.522171  3438 tablet_bootstrap.cc:492] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:15.522811  3438 ts_tablet_manager.cc:1403] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.107s	user 0.017s	sys 0.012s
I20260709 12:23:15.524376  3439 tablet_bootstrap.cc:492] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:15.529050  3462 raft_consensus.cc:493] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.527035  3438 raft_consensus.cc:359] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.529990  3438 raft_consensus.cc:385] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.530584  3438 raft_consensus.cc:740] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.529942  3462 raft_consensus.cc:515] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.531498  3438 consensus_queue.cc:260] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.534175  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:15.534878  3440 tablet_bootstrap.cc:654] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.535509  3462 leader_election.cc:290] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:15.538064  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:15.538766  3246 raft_consensus.cc:2468] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:15.539397  3324 raft_consensus.cc:2468] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:15.540542  3362 leader_election.cc:304] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:15.544243  3438 ts_tablet_manager.cc:1434] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:23:15.545740  3438 tablet_bootstrap.cc:492] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:15.545794  3462 raft_consensus.cc:2804] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:15.546271  3462 raft_consensus.cc:493] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.556242  3462 raft_consensus.cc:3060] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.562681  3439 tablet_bootstrap.cc:654] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.588755  3653 consensus_queue.cc:1048] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:15.603255  3323 raft_consensus.cc:2468] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:15.604930  3364 leader_election.cc:304] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:15.601500  3462 raft_consensus.cc:515] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.606067  3540 raft_consensus.cc:2804] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:15.606717  3540 raft_consensus.cc:697] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:15.608181  3440 tablet_bootstrap.cc:492] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:15.608727  3440 ts_tablet_manager.cc:1403] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.122s	user 0.013s	sys 0.001s
I20260709 12:23:15.607697  3540 consensus_queue.cc:237] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.616921  3440 raft_consensus.cc:359] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.617720  3440 raft_consensus.cc:385] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.618003  3440 raft_consensus.cc:740] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.619697  3440 consensus_queue.cc:260] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.621881  3440 ts_tablet_manager.cc:1434] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.002s
I20260709 12:23:15.621524  3544 consensus_queue.cc:1048] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:15.630508  3440 tablet_bootstrap.cc:492] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:15.632874  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:15.633780  3246 raft_consensus.cc:3060] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.639905  3440 tablet_bootstrap.cc:654] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.643411  3439 tablet_bootstrap.cc:492] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:15.642306  3462 leader_election.cc:290] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:15.643590  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:15.644223  3439 ts_tablet_manager.cc:1403] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.120s	user 0.022s	sys 0.004s
I20260709 12:23:15.644348  3324 raft_consensus.cc:3060] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.651886  3246 raft_consensus.cc:2468] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:15.651575  3439 raft_consensus.cc:359] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.652671  3439 raft_consensus.cc:385] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.653015  3439 raft_consensus.cc:740] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.653956  3439 consensus_queue.cc:260] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.656193  3362 leader_election.cc:304] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:15.656731  3439 ts_tablet_manager.cc:1434] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.006s
I20260709 12:23:15.657109  3653 raft_consensus.cc:2804] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:15.657567  3653 raft_consensus.cc:697] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:15.657768  3438 tablet_bootstrap.cc:654] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.658488  3653 consensus_queue.cc:237] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.663239  3439 tablet_bootstrap.cc:492] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:15.670859  3324 raft_consensus.cc:2468] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:15.678988  3147 catalog_manager.cc:5697] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:15.685837  3438 tablet_bootstrap.cc:492] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:15.686553  3438 ts_tablet_manager.cc:1403] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.141s	user 0.015s	sys 0.006s
I20260709 12:23:15.697870  3438 raft_consensus.cc:359] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.698837  3438 raft_consensus.cc:385] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.699200  3438 raft_consensus.cc:740] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.700013  3438 consensus_queue.cc:260] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.701288  3440 tablet_bootstrap.cc:492] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:15.701826  3440 ts_tablet_manager.cc:1403] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.072s	user 0.024s	sys 0.021s
I20260709 12:23:15.702550  3438 ts_tablet_manager.cc:1434] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:23:15.701514  3439 tablet_bootstrap.cc:654] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.703908  3438 tablet_bootstrap.cc:492] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:15.716264  3440 raft_consensus.cc:359] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.718179  3147 catalog_manager.cc:5697] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:15.729866  3440 raft_consensus.cc:385] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.730224  3440 raft_consensus.cc:740] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.729463  3438 tablet_bootstrap.cc:654] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.730985  3440 consensus_queue.cc:260] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.734287  3440 ts_tablet_manager.cc:1434] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.002s
I20260709 12:23:15.735745  3440 tablet_bootstrap.cc:492] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:15.740552  3439 tablet_bootstrap.cc:492] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:15.741411  3439 ts_tablet_manager.cc:1403] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.084s	user 0.007s	sys 0.005s
I20260709 12:23:15.745533  3439 raft_consensus.cc:359] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.746570  3439 raft_consensus.cc:385] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.746917  3439 raft_consensus.cc:740] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.750625  3439 consensus_queue.cc:260] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.754503  3440 tablet_bootstrap.cc:654] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.771557  3439 ts_tablet_manager.cc:1434] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.001s
I20260709 12:23:15.773303  3439 tablet_bootstrap.cc:492] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:15.776193  3540 raft_consensus.cc:493] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.776983  3540 raft_consensus.cc:515] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.780054  3540 leader_election.cc:290] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:15.782586  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f62d0c16794e8c941849d55bb4bb0b" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:15.783147  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f62d0c16794e8c941849d55bb4bb0b" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:15.784008  3323 raft_consensus.cc:2468] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
W20260709 12:23:15.784948  3362 leader_election.cc:343] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.786868  3540 raft_consensus.cc:493] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.787451  3540 raft_consensus.cc:515] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.789666  3540 leader_election.cc:290] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:15.786005  3364 leader_election.cc:304] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:15.791455  3438 tablet_bootstrap.cc:492] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:15.791440  3540 raft_consensus.cc:2804] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:15.792366  3438 ts_tablet_manager.cc:1403] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.089s	user 0.026s	sys 0.034s
I20260709 12:23:15.792752  3540 raft_consensus.cc:493] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.793147  3245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:15.793435  3540 raft_consensus.cc:3060] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.794575  3248 raft_consensus.cc:3060] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.795547  3438 raft_consensus.cc:359] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.796406  3438 raft_consensus.cc:385] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.796976  3438 raft_consensus.cc:740] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.797673  3438 consensus_queue.cc:260] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:15.800220  3362 leader_election.cc:343] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.803830  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:15.804561  3323 raft_consensus.cc:2468] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:15.804690  3438 ts_tablet_manager.cc:1434] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:23:15.805815  3438 tablet_bootstrap.cc:492] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:15.805974  3364 leader_election.cc:304] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:15.807240  3653 raft_consensus.cc:2804] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:15.807619  3653 raft_consensus.cc:493] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.807966  3653 raft_consensus.cc:3060] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.811816  3540 raft_consensus.cc:515] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.816895  3462 consensus_queue.cc:1048] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:15.820096  3540 leader_election.cc:290] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:15.822844  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f62d0c16794e8c941849d55bb4bb0b" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:15.823680  3323 raft_consensus.cc:3060] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.826392  3439 tablet_bootstrap.cc:654] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.834046  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f62d0c16794e8c941849d55bb4bb0b" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:15.832015  3323 raft_consensus.cc:2468] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:15.835872  3653 raft_consensus.cc:515] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.837090  3364 leader_election.cc:304] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
W20260709 12:23:15.838430  3362 leader_election.cc:343] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.838683  3540 raft_consensus.cc:2804] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:15.839298  3540 raft_consensus.cc:697] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:15.839501  3544 consensus_queue.cc:1048] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:15.840287  3540 consensus_queue.cc:237] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.843070  3438 tablet_bootstrap.cc:654] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.846524  3462 consensus_queue.cc:1048] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:15.852360  3439 tablet_bootstrap.cc:492] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:15.852953  3439 ts_tablet_manager.cc:1403] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.080s	user 0.012s	sys 0.001s
I20260709 12:23:15.864660  3640 raft_consensus.cc:493] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.865216  3640 raft_consensus.cc:515] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.868142  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:15.868142  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2884104b65f4a589e915fd4328b729c" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:15.869700  3288 leader_election.cc:343] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:15.869696  3362 leader_election.cc:343] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.860900  3439 raft_consensus.cc:359] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.871526  3439 raft_consensus.cc:385] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.871829  3439 raft_consensus.cc:740] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.872475  3439 consensus_queue.cc:260] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.872756  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2884104b65f4a589e915fd4328b729c" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
W20260709 12:23:15.874315  3289 leader_election.cc:343] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.874675  3289 leader_election.cc:304] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:15.875059  3439 ts_tablet_manager.cc:1434] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:23:15.875492  3539 raft_consensus.cc:2749] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:15.876011  3439 tablet_bootstrap.cc:492] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:15.876302  3640 leader_election.cc:290] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:15.879837  3653 leader_election.cc:290] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:15.880283  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:15.881956  3323 raft_consensus.cc:3060] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.889920  3440 tablet_bootstrap.cc:492] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:15.890808  3440 ts_tablet_manager.cc:1403] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.155s	user 0.028s	sys 0.012s
I20260709 12:23:15.895761  3323 raft_consensus.cc:2468] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:15.897300  3364 leader_election.cc:304] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:15.899313  3462 raft_consensus.cc:2804] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:15.907097  3462 raft_consensus.cc:697] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:15.908128  3462 consensus_queue.cc:237] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.916306  3438 tablet_bootstrap.cc:492] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:15.911173  3443 raft_consensus.cc:493] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.919162  3438 ts_tablet_manager.cc:1403] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.114s	user 0.022s	sys 0.015s
I20260709 12:23:15.919361  3443 raft_consensus.cc:515] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.921823  3438 raft_consensus.cc:359] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.922477  3443 leader_election.cc:290] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:15.923799  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:15.922534  3438 raft_consensus.cc:385] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.924835  3438 raft_consensus.cc:740] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.909526  3145 catalog_manager.cc:5697] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:15.926033  3653 consensus_queue.cc:1048] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:15.925459  3322 raft_consensus.cc:2393] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cdaa0234c64450eb4ce4500b51b8a81 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:15.930313  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:15.933233  3438 consensus_queue.cc:260] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.942231  3214 leader_election.cc:304] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:15.947598  3438 ts_tablet_manager.cc:1434] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:23:15.951642  3438 tablet_bootstrap.cc:492] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:15.964850  3664 raft_consensus.cc:3060] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.972235  3440 raft_consensus.cc:359] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.973014  3440 raft_consensus.cc:385] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.973282  3440 raft_consensus.cc:740] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.974496  3440 consensus_queue.cc:260] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:15.977151  3439 tablet_bootstrap.cc:654] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.994043  3440 ts_tablet_manager.cc:1434] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.103s	user 0.011s	sys 0.005s
I20260709 12:23:15.995302  3440 tablet_bootstrap.cc:492] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:15.991204  3539 raft_consensus.cc:493] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.005239  3640 raft_consensus.cc:493] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.005762  3539 raft_consensus.cc:515] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.006103  3640 raft_consensus.cc:515] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.032369  3438 tablet_bootstrap.cc:654] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.039573  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7494971ced1402da04932982df157fd" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
W20260709 12:23:16.041373  3289 leader_election.cc:343] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.043151  3640 leader_election.cc:290] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:16.037487  3539 leader_election.cc:290] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:16.044664  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0634624f456740b48b77096645ab6b0b" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:16.044664  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0634624f456740b48b77096645ab6b0b" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:16.046722  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7494971ced1402da04932982df157fd" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:16.047674  3288 leader_election.cc:343] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:16.048187  3289 leader_election.cc:343] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:16.048455  3288 leader_election.cc:343] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.048609  3289 leader_election.cc:304] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:16.048872  3288 leader_election.cc:304] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:16.049985  3539 raft_consensus.cc:2749] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:16.050622  3539 raft_consensus.cc:2749] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:16.054059  3664 raft_consensus.cc:2749] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:16.066746  3705 raft_consensus.cc:493] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.067293  3705 raft_consensus.cc:515] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.070571  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36377e446734493c8f3b1ff53d40d1cc" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:16.071344  3324 raft_consensus.cc:2468] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:16.072225  3705 leader_election.cc:290] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:16.072875  3364 leader_election.cc:304] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:16.074416  3705 raft_consensus.cc:2804] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.074867  3705 raft_consensus.cc:493] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.075208  3705 raft_consensus.cc:3060] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.076995  3440 tablet_bootstrap.cc:654] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.087211  3542 raft_consensus.cc:493] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.087815  3542 raft_consensus.cc:515] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.092360  3540 consensus_queue.cc:1048] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:16.090268  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36377e446734493c8f3b1ff53d40d1cc" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:16.094748  3248 raft_consensus.cc:2468] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:16.093477  3542 leader_election.cc:290] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:16.095588  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02f86799bc54984b41ede0f7db226e6" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:16.096803  3322 raft_consensus.cc:2468] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:16.097816  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02f86799bc54984b41ede0f7db226e6" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:16.100095  3438 tablet_bootstrap.cc:492] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:16.102802  3438 ts_tablet_manager.cc:1403] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.152s	user 0.008s	sys 0.009s
I20260709 12:23:16.105283  3364 leader_election.cc:304] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:16.104517  3248 raft_consensus.cc:2468] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:16.111574  3438 raft_consensus.cc:359] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.112277  3438 raft_consensus.cc:385] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.112576  3438 raft_consensus.cc:740] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.113271  3438 consensus_queue.cc:260] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.110608  3704 raft_consensus.cc:2804] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.114492  3704 raft_consensus.cc:493] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.114187  3705 raft_consensus.cc:515] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.114915  3704 raft_consensus.cc:3060] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.115078  3542 consensus_queue.cc:1048] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:16.115418  3438 ts_tablet_manager.cc:1434] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:23:16.118630  3438 tablet_bootstrap.cc:492] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:16.125092  3439 tablet_bootstrap.cc:492] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:16.125679  3439 ts_tablet_manager.cc:1403] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.250s	user 0.011s	sys 0.013s
I20260709 12:23:16.135025  3439 raft_consensus.cc:359] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.135943  3439 raft_consensus.cc:385] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.136242  3439 raft_consensus.cc:740] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.137053  3439 consensus_queue.cc:260] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.139595  3439 ts_tablet_manager.cc:1434] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 12:23:16.131958  3704 raft_consensus.cc:515] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.143178  3439 tablet_bootstrap.cc:492] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:16.145754  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36377e446734493c8f3b1ff53d40d1cc" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:16.149345  3248 raft_consensus.cc:3060] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.149966  3440 tablet_bootstrap.cc:492] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:16.150504  3440 ts_tablet_manager.cc:1403] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.155s	user 0.011s	sys 0.005s
I20260709 12:23:16.153141  3440 raft_consensus.cc:359] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.154848  3705 leader_election.cc:290] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:16.147964  3144 catalog_manager.cc:5697] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:16.159157  3440 raft_consensus.cc:385] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.160640  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36377e446734493c8f3b1ff53d40d1cc" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:16.161760  3440 raft_consensus.cc:740] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.162218  3322 raft_consensus.cc:3060] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.162663  3440 consensus_queue.cc:260] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:16.167649  3195 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:16.171545  3440 ts_tablet_manager.cc:1434] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:23:16.172022  3245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02f86799bc54984b41ede0f7db226e6" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:16.173611  3440 tablet_bootstrap.cc:492] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:16.175750  3544 raft_consensus.cc:493] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.172765  3245 raft_consensus.cc:3060] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.178702  3544 raft_consensus.cc:515] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.190934  3438 tablet_bootstrap.cc:654] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.192818  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02f86799bc54984b41ede0f7db226e6" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:16.193698  3323 raft_consensus.cc:3060] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.192605  3539 raft_consensus.cc:493] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.194427  3539 raft_consensus.cc:515] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.199060  3439 tablet_bootstrap.cc:654] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.191119  3704 leader_election.cc:290] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:16.191846  3248 raft_consensus.cc:2468] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:16.194870  3706 consensus_queue.cc:1048] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:16.226019  3544 leader_election.cc:290] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:16.226619  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7494971ced1402da04932982df157fd" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:16.228236  3462 consensus_queue.cc:1048] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:16.228864  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7494971ced1402da04932982df157fd" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:16.229614  3320 raft_consensus.cc:2468] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:16.230614  3539 leader_election.cc:290] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:16.232580  3364 leader_election.cc:304] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:16.244948  3323 raft_consensus.cc:2468] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:16.243001  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3098f015fb154cf490e147a8e71f65b6" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:16.247406  3288 leader_election.cc:343] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.241554  3462 raft_consensus.cc:2804] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.242506  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3098f015fb154cf490e147a8e71f65b6" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:16.249915  3462 raft_consensus.cc:493] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.250316  3462 raft_consensus.cc:3060] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:16.251677  3289 leader_election.cc:343] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.252122  3289 leader_election.cc:304] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:16.253661  3539 raft_consensus.cc:2749] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:16.262115  3362 leader_election.cc:304] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
W20260709 12:23:16.263439  3362 leader_election.cc:343] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.278194  3544 raft_consensus.cc:2804] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.278973  3544 raft_consensus.cc:697] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:16.280153  3544 consensus_queue.cc:237] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.287397  3322 raft_consensus.cc:2468] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:16.291019  3462 raft_consensus.cc:515] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.269876  3245 raft_consensus.cc:2468] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:16.367844  3462 leader_election.cc:290] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:16.369302  3362 leader_election.cc:304] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:16.372870  3544 raft_consensus.cc:2804] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.373535  3544 raft_consensus.cc:697] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:16.374547  3544 consensus_queue.cc:237] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.376482  3438 tablet_bootstrap.cc:492] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:16.377031  3438 ts_tablet_manager.cc:1403] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.259s	user 0.010s	sys 0.021s
I20260709 12:23:16.385057  3438 raft_consensus.cc:359] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.387070  3705 consensus_queue.cc:1048] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:16.396111  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7494971ced1402da04932982df157fd" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:16.396713  3322 raft_consensus.cc:3060] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.388754  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7494971ced1402da04932982df157fd" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
W20260709 12:23:16.400573  3362 consensus_peers.cc:597] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:16.392704  3438 raft_consensus.cc:385] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.402118  3438 raft_consensus.cc:740] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
W20260709 12:23:16.401125  3362 leader_election.cc:343] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.395533  3440 tablet_bootstrap.cc:654] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.411094  3147 catalog_manager.cc:5697] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:16.413484  3439 tablet_bootstrap.cc:492] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:16.414058  3439 ts_tablet_manager.cc:1403] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.274s	user 0.024s	sys 0.033s
I20260709 12:23:16.427295  3704 consensus_queue.cc:1048] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:16.406323  3438 consensus_queue.cc:260] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.431352  3438 ts_tablet_manager.cc:1434] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.054s	user 0.009s	sys 0.001s
I20260709 12:23:16.433040  3438 tablet_bootstrap.cc:492] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:16.431762  3322 raft_consensus.cc:2468] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:16.446477  3439 raft_consensus.cc:359] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.447566  3439 raft_consensus.cc:385] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.450404  3439 raft_consensus.cc:740] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.453722  3438 tablet_bootstrap.cc:654] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.452633  3439 consensus_queue.cc:260] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.466691  3439 ts_tablet_manager.cc:1434] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.000s
I20260709 12:23:16.468973  3704 raft_consensus.cc:493] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.469583  3704 raft_consensus.cc:515] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.472339  3704 leader_election.cc:290] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:16.476828  3245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9b725784044c8da903f092febf7c05" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:16.478008  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9b725784044c8da903f092febf7c05" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:16.478937  3320 raft_consensus.cc:2468] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:16.479800  3364 leader_election.cc:304] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:16.481369  3544 raft_consensus.cc:2804] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.482187  3544 raft_consensus.cc:697] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:16.488022  3664 raft_consensus.cc:493] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.490978  3664 raft_consensus.cc:515] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.495234  3664 leader_election.cc:290] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:16.492645  3440 tablet_bootstrap.cc:492] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:16.495219  3544 consensus_queue.cc:237] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.496438  3440 ts_tablet_manager.cc:1403] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.323s	user 0.014s	sys 0.011s
W20260709 12:23:16.489691  3362 leader_election.cc:343] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.498943  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:16.503639  3244 raft_consensus.cc:3060] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.496891  3439 tablet_bootstrap.cc:492] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:16.505748  3440 raft_consensus.cc:359] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.505194  3364 leader_election.cc:304] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:16.508188  3704 raft_consensus.cc:2804] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.510365  3704 raft_consensus.cc:493] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.510895  3704 raft_consensus.cc:3060] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.506531  3440 raft_consensus.cc:385] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.513880  3721 raft_consensus.cc:493] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.514997  3440 raft_consensus.cc:740] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.520885  3144 catalog_manager.cc:5697] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:16.522782  3440 consensus_queue.cc:260] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.523773  3721 raft_consensus.cc:515] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.525557  3438 tablet_bootstrap.cc:492] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:16.531019  3438 ts_tablet_manager.cc:1403] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.099s	user 0.019s	sys 0.019s
I20260709 12:23:16.532814  3439 tablet_bootstrap.cc:654] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.536288  3440 ts_tablet_manager.cc:1434] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 12:23:16.537410  3440 tablet_bootstrap.cc:492] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:16.551396  3704 raft_consensus.cc:515] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.549314  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:16.555985  3705 consensus_queue.cc:1048] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:16.556301  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9b725784044c8da903f092febf7c05" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:16.557260  3320 raft_consensus.cc:3060] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.559630  3214 leader_election.cc:304] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:16.560281  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9b725784044c8da903f092febf7c05" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:16.553450  3245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7048879a65244efa5a140c52d782803" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:16.564455  3658 raft_consensus.cc:2749] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:16.565079  3288 leader_election.cc:343] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.565853  3721 leader_election.cc:290] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:23:16.566799  3362 leader_election.cc:343] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.569110  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7048879a65244efa5a140c52d782803" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
W20260709 12:23:16.571991  3289 leader_election.cc:343] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.572439  3289 leader_election.cc:304] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:16.573807  3721 raft_consensus.cc:2749] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:16.553555  3704 leader_election.cc:290] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:16.579566  3320 raft_consensus.cc:2468] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:16.580797  3147 catalog_manager.cc:5697] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:16.583170  3364 leader_election.cc:304] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:16.583527  3440 tablet_bootstrap.cc:654] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.580163  3438 raft_consensus.cc:359] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.584656  3438 raft_consensus.cc:385] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.585804  3438 raft_consensus.cc:740] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.586570  3438 consensus_queue.cc:260] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.585070  3704 raft_consensus.cc:2804] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.589262  3704 raft_consensus.cc:697] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:16.590391  3438 ts_tablet_manager.cc:1434] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.059s	user 0.004s	sys 0.003s
I20260709 12:23:16.590118  3704 consensus_queue.cc:237] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.593180  3438 tablet_bootstrap.cc:492] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:16.632309  3439 tablet_bootstrap.cc:492] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:16.632956  3439 ts_tablet_manager.cc:1403] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.136s	user 0.010s	sys 0.004s
I20260709 12:23:16.638015  3439 raft_consensus.cc:359] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.638888  3439 raft_consensus.cc:385] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.639479  3439 raft_consensus.cc:740] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.643152  3439 consensus_queue.cc:260] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.645210  3439 ts_tablet_manager.cc:1434] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:23:16.646311  3439 tablet_bootstrap.cc:492] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:16.665743  3440 tablet_bootstrap.cc:492] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:16.667858  3440 ts_tablet_manager.cc:1403] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.131s	user 0.011s	sys 0.005s
I20260709 12:23:16.671628  3145 catalog_manager.cc:5697] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:16.672708  3439 tablet_bootstrap.cc:654] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.666417  3438 tablet_bootstrap.cc:654] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.681136  3440 raft_consensus.cc:359] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.684813  3440 raft_consensus.cc:385] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.685138  3440 raft_consensus.cc:740] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.686079  3440 consensus_queue.cc:260] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.688235  3440 ts_tablet_manager.cc:1434] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 12:23:16.704684  3440 tablet_bootstrap.cc:492] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:16.711604  3438 tablet_bootstrap.cc:492] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:16.712707  3438 ts_tablet_manager.cc:1403] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.120s	user 0.012s	sys 0.003s
I20260709 12:23:16.713912  3440 tablet_bootstrap.cc:654] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.714918  3704 consensus_queue.cc:1048] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:16.715898  3438 raft_consensus.cc:359] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.716746  3438 raft_consensus.cc:385] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.717021  3438 raft_consensus.cc:740] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.719396  3438 consensus_queue.cc:260] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.720728  3439 tablet_bootstrap.cc:492] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:16.721232  3439 ts_tablet_manager.cc:1403] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.075s	user 0.032s	sys 0.016s
I20260709 12:23:16.722033  3438 ts_tablet_manager.cc:1434] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:23:16.723100  3438 tablet_bootstrap.cc:492] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:16.724119  3439 raft_consensus.cc:359] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.725061  3439 raft_consensus.cc:385] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.725404  3439 raft_consensus.cc:740] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.726547  3439 consensus_queue.cc:260] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.729099  3439 ts_tablet_manager.cc:1434] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:16.729373  3438 tablet_bootstrap.cc:654] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.721230  3440 tablet_bootstrap.cc:492] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:16.730193  3440 ts_tablet_manager.cc:1403] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.007s
I20260709 12:23:16.733755  3440 raft_consensus.cc:359] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.734501  3440 raft_consensus.cc:385] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.734772  3440 raft_consensus.cc:740] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.735472  3440 consensus_queue.cc:260] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.738083  3440 ts_tablet_manager.cc:1434] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:23:16.739356  3440 tablet_bootstrap.cc:492] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:16.742678  3439 tablet_bootstrap.cc:492] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:16.743400  3438 tablet_bootstrap.cc:492] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:16.743937  3438 ts_tablet_manager.cc:1403] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.004s
I20260709 12:23:16.747762  3438 raft_consensus.cc:359] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.748483  3438 raft_consensus.cc:385] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.748765  3438 raft_consensus.cc:740] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.749764  3438 consensus_queue.cc:260] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.752038  3438 ts_tablet_manager.cc:1434] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:23:16.753044  3438 tablet_bootstrap.cc:492] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:16.754370  3440 tablet_bootstrap.cc:654] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.750177  3439 tablet_bootstrap.cc:654] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.760195  3438 tablet_bootstrap.cc:654] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.769599  3439 tablet_bootstrap.cc:492] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:16.770231  3439 ts_tablet_manager.cc:1403] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.000s
I20260709 12:23:16.773129  3439 raft_consensus.cc:359] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.773905  3439 raft_consensus.cc:385] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.774187  3439 raft_consensus.cc:740] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.775055  3439 consensus_queue.cc:260] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.777588  3439 ts_tablet_manager.cc:1434] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:16.778663  3439 tablet_bootstrap.cc:492] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:16.785791  3439 tablet_bootstrap.cc:654] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.805411  3440 tablet_bootstrap.cc:492] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:16.805985  3440 ts_tablet_manager.cc:1403] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.067s	user 0.016s	sys 0.002s
I20260709 12:23:16.808835  3440 raft_consensus.cc:359] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.809581  3440 raft_consensus.cc:385] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.809860  3440 raft_consensus.cc:740] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.810590  3440 consensus_queue.cc:260] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.811038  3540 consensus_queue.cc:1048] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:16.811991  3245 raft_consensus.cc:3060] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.813236  3440 ts_tablet_manager.cc:1434] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:23:16.820932  3438 tablet_bootstrap.cc:492] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:16.821554  3438 ts_tablet_manager.cc:1403] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.069s	user 0.012s	sys 0.016s
I20260709 12:23:16.824965  3438 raft_consensus.cc:359] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.834158  3440 tablet_bootstrap.cc:492] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:16.839159  3438 raft_consensus.cc:385] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.839510  3438 raft_consensus.cc:740] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.840471  3438 consensus_queue.cc:260] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.842751  3438 ts_tablet_manager.cc:1434] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:23:16.845495  3440 tablet_bootstrap.cc:654] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.852792  3438 tablet_bootstrap.cc:492] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:16.870074  3438 tablet_bootstrap.cc:654] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.872131  3439 tablet_bootstrap.cc:492] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:16.872728  3439 ts_tablet_manager.cc:1403] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.094s	user 0.027s	sys 0.012s
I20260709 12:23:16.875921  3440 tablet_bootstrap.cc:492] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:16.876530  3440 ts_tablet_manager.cc:1403] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.008s
I20260709 12:23:16.880788  3440 raft_consensus.cc:359] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.881572  3440 raft_consensus.cc:385] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.881862  3440 raft_consensus.cc:740] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.882519  3440 consensus_queue.cc:260] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.884696  3440 ts_tablet_manager.cc:1434] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:23:16.890883  3440 tablet_bootstrap.cc:492] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:16.884872  3439 raft_consensus.cc:359] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.893852  3439 raft_consensus.cc:385] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.894193  3439 raft_consensus.cc:740] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.894948  3439 consensus_queue.cc:260] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.897104  3439 ts_tablet_manager.cc:1434] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.002s
I20260709 12:23:16.897833  3440 tablet_bootstrap.cc:654] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.902453  3439 tablet_bootstrap.cc:492] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:16.903254  3542 consensus_queue.cc:1048] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:16.914095  3438 tablet_bootstrap.cc:492] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:16.914623  3542 raft_consensus.cc:493] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.917536  3439 tablet_bootstrap.cc:654] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.917025  3542 raft_consensus.cc:515] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.918260  3664 raft_consensus.cc:493] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.918848  3664 raft_consensus.cc:515] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.920454  3542 leader_election.cc:290] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:16.921440  3438 ts_tablet_manager.cc:1403] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.003s
I20260709 12:23:16.923141  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7048879a65244efa5a140c52d782803" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:16.923980  3320 raft_consensus.cc:2468] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:16.924036  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9b725784044c8da903f092febf7c05" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:16.925272  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9b725784044c8da903f092febf7c05" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:16.926101  3324 raft_consensus.cc:2393] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cdaa0234c64450eb4ce4500b51b8a81 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:16.926512  3658 raft_consensus.cc:493] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.927004  3658 raft_consensus.cc:515] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.928048  3215 leader_election.cc:304] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:16.928987  3364 leader_election.cc:304] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:16.929212  3658 leader_election.cc:290] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:16.930760  3705 raft_consensus.cc:2804] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.931245  3705 raft_consensus.cc:493] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.931689  3705 raft_consensus.cc:3060] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.934221  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7048879a65244efa5a140c52d782803" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:16.921267  3664 leader_election.cc:290] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:16.935894  3658 raft_consensus.cc:3060] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.945382  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7494971ced1402da04932982df157fd" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:16.946343  3320 raft_consensus.cc:2393] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cdaa0234c64450eb4ce4500b51b8a81 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:16.947381  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7494971ced1402da04932982df157fd" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:16.947646  3439 tablet_bootstrap.cc:492] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
W20260709 12:23:16.947883  3362 leader_election.cc:343] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.948168  3439 ts_tablet_manager.cc:1403] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.004s
I20260709 12:23:16.949959  3214 leader_election.cc:304] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:16.950810  3746 raft_consensus.cc:3060] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.952351  3440 tablet_bootstrap.cc:492] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:16.953003  3440 ts_tablet_manager.cc:1403] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.062s	user 0.013s	sys 0.012s
I20260709 12:23:16.954447  3721 raft_consensus.cc:493] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.955173  3721 raft_consensus.cc:515] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.956149  3440 raft_consensus.cc:359] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.957115  3440 raft_consensus.cc:385] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.957406  3440 raft_consensus.cc:740] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.958135  3440 consensus_queue.cc:260] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.959057  3439 raft_consensus.cc:359] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.959738  3439 raft_consensus.cc:385] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.959834  3721 leader_election.cc:290] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:16.960301  3440 ts_tablet_manager.cc:1434] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:16.961340  3440 tablet_bootstrap.cc:492] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:16.961782  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
W20260709 12:23:16.963295  3289 leader_election.cc:343] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.960039  3439 raft_consensus.cc:740] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.964560  3439 consensus_queue.cc:260] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.966815  3540 raft_consensus.cc:493] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.967556  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:16.967528  3540 raft_consensus.cc:515] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.970770  3540 leader_election.cc:290] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:16.970950  3439 ts_tablet_manager.cc:1434] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 12:23:16.971877  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0634624f456740b48b77096645ab6b0b" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:16.972642  3320 raft_consensus.cc:2468] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
W20260709 12:23:16.972834  3288 leader_election.cc:343] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.973356  3288 leader_election.cc:304] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:16.971917  3439 tablet_bootstrap.cc:492] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:16.974392  3364 leader_election.cc:304] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:16.975317  3540 raft_consensus.cc:2804] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.975689  3540 raft_consensus.cc:493] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.976022  3540 raft_consensus.cc:3060] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.976270  3640 raft_consensus.cc:2749] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:16.985587  3746 raft_consensus.cc:2749] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:16.988191  3440 tablet_bootstrap.cc:654] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.988057  3705 raft_consensus.cc:515] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:16.990717  3705 leader_election.cc:290] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:16.991356  3705 consensus_queue.cc:1048] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:16.992074  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7048879a65244efa5a140c52d782803" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:16.992777  3320 raft_consensus.cc:3060] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.994758  3658 raft_consensus.cc:2749] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.000455  3439 tablet_bootstrap.cc:654] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.000046  3438 raft_consensus.cc:359] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.000980  3438 raft_consensus.cc:385] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.001278  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0634624f456740b48b77096645ab6b0b" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:17.002003  3244 raft_consensus.cc:2468] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:17.000541  3540 raft_consensus.cc:515] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.001302  3438 raft_consensus.cc:740] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.011997  3542 consensus_queue.cc:1048] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:17.012809  3721 raft_consensus.cc:493] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.013540  3540 leader_election.cc:290] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:17.013401  3721 raft_consensus.cc:515] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.020190  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0634624f456740b48b77096645ab6b0b" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:17.020951  3323 raft_consensus.cc:3060] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.030555  3323 raft_consensus.cc:2468] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:17.022845  3438 consensus_queue.cc:260] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.031303  3443 raft_consensus.cc:493] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.035449  3443 raft_consensus.cc:515] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.037128  3721 leader_election.cc:290] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:17.038735  3320 raft_consensus.cc:2468] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:17.041743  3542 consensus_queue.cc:1048] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:23:17.044028  3440 tablet_bootstrap.cc:492] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:17.044551  3440 ts_tablet_manager.cc:1403] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.083s	user 0.010s	sys 0.010s
I20260709 12:23:17.045118  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3098f015fb154cf490e147a8e71f65b6" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:17.045563  3364 leader_election.cc:304] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:17.045894  3395 raft_consensus.cc:2468] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:17.046492  3462 raft_consensus.cc:2804] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.047075  3462 raft_consensus.cc:697] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:17.047528  3214 leader_election.cc:304] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:17.047946  3462 consensus_queue.cc:237] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.048519  3746 raft_consensus.cc:2804] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.048906  3746 raft_consensus.cc:493] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.049245  3746 raft_consensus.cc:3060] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.049564  3439 tablet_bootstrap.cc:492] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:17.050307  3439 ts_tablet_manager.cc:1403] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.079s	user 0.007s	sys 0.007s
I20260709 12:23:17.050498  3364 leader_election.cc:304] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:17.050861  3321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3098f015fb154cf490e147a8e71f65b6" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:17.051353  3704 raft_consensus.cc:2804] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.051571  3321 raft_consensus.cc:2468] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:17.051874  3704 raft_consensus.cc:697] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:17.051169  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3015bb6b71f444dcb17062ac96b48cf5" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
W20260709 12:23:17.053063  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 48398.6us
I20260709 12:23:17.054533  3440 raft_consensus.cc:359] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.052793  3704 consensus_queue.cc:237] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.058702  3438 ts_tablet_manager.cc:1434] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.136s	user 0.005s	sys 0.004s
I20260709 12:23:17.059183  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3015bb6b71f444dcb17062ac96b48cf5" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:17.059816  3438 tablet_bootstrap.cc:492] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:17.055485  3440 raft_consensus.cc:385] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.037616  3443 leader_election.cc:290] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:17.061371  3440 raft_consensus.cc:740] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.063349  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0634624f456740b48b77096645ab6b0b" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:17.066939  3249 raft_consensus.cc:3060] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.065423  3440 consensus_queue.cc:260] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.071445  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7048879a65244efa5a140c52d782803" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:17.072170  3244 raft_consensus.cc:3060] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:17.072779  3289 leader_election.cc:343] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.075615  3439 raft_consensus.cc:359] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.076414  3439 raft_consensus.cc:385] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.076666  3439 raft_consensus.cc:740] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.077297  3439 consensus_queue.cc:260] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.081964  3440 ts_tablet_manager.cc:1434] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
I20260709 12:23:17.090742  3440 tablet_bootstrap.cc:492] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:17.102869  3439 ts_tablet_manager.cc:1434] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.003s
I20260709 12:23:17.104921  3439 tablet_bootstrap.cc:492] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:17.116405  3588 raft_consensus.cc:493] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.117019  3588 raft_consensus.cc:515] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:17.122607  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 45957.4us
W20260709 12:23:17.124711  3288 leader_election.cc:343] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.099064  3544 consensus_queue.cc:1048] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:23:17.126631  3288 leader_election.cc:304] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:17.127493  3640 raft_consensus.cc:2749] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.121935  3244 raft_consensus.cc:2468] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:17.147926  3440 tablet_bootstrap.cc:654] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.171167  3438 tablet_bootstrap.cc:654] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.190788  3588 leader_election.cc:290] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:17.192049  3746 raft_consensus.cc:515] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.192621  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a9849f17fd4313ab9ddbb06589c46f" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:17.193348  3396 raft_consensus.cc:2468] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:17.194907  3289 leader_election.cc:304] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:17.204862  3588 raft_consensus.cc:2804] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.206310  3249 raft_consensus.cc:2468] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
W20260709 12:23:17.261521  3755 log.cc:927] Time spent T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:23:17.261327  3746 leader_election.cc:290] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:17.261643  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3098f015fb154cf490e147a8e71f65b6" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:17.264951  3324 raft_consensus.cc:3060] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.211503  3640 raft_consensus.cc:493] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.214102  3588 raft_consensus.cc:493] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.270740  3588 raft_consensus.cc:3060] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.296134  3324 raft_consensus.cc:2468] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:17.272557  3640 raft_consensus.cc:515] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.314245  3243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a9849f17fd4313ab9ddbb06589c46f" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:17.336253  3439 tablet_bootstrap.cc:654] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.342051  3757 raft_consensus.cc:493] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.342749  3757 raft_consensus.cc:515] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.344990  3757 leader_election.cc:290] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:17.345741  3588 raft_consensus.cc:515] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.342111  3147 catalog_manager.cc:5697] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.346908  3438 tablet_bootstrap.cc:492] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:17.347407  3147 catalog_manager.cc:5697] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.350056  3243 raft_consensus.cc:2468] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
W20260709 12:23:17.350369  3756 log.cc:927] Time spent T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4: Append to log took a long time: real 0.063s	user 0.002s	sys 0.000s
I20260709 12:23:17.350086  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3098f015fb154cf490e147a8e71f65b6" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:17.352375  3396 raft_consensus.cc:3060] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.352546  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a9849f17fd4313ab9ddbb06589c46f" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:17.347451  3438 ts_tablet_manager.cc:1403] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.288s	user 0.021s	sys 0.013s
I20260709 12:23:17.353197  3397 raft_consensus.cc:2468] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:17.359163  3215 leader_election.cc:304] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:17.365402  3757 raft_consensus.cc:2804] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.403292  3588 leader_election.cc:290] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:17.426301  3757 raft_consensus.cc:697] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:17.429102  3757 consensus_queue.cc:237] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.432157  3438 raft_consensus.cc:359] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.432931  3438 raft_consensus.cc:385] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.433224  3438 raft_consensus.cc:740] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.437855  3317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a9849f17fd4313ab9ddbb06589c46f" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:17.459157  3438 consensus_queue.cc:260] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.439522  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a9849f17fd4313ab9ddbb06589c46f" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:17.467864  3440 tablet_bootstrap.cc:492] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:17.468521  3440 ts_tablet_manager.cc:1403] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.378s	user 0.022s	sys 0.039s
I20260709 12:23:17.425488  3542 raft_consensus.cc:493] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.467263  3317 raft_consensus.cc:2393] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cdaa0234c64450eb4ce4500b51b8a81 in current term 1: Already voted for candidate 8525f77f43714a58bf158d872c1db3e4 in this term.
I20260709 12:23:17.483125  3640 leader_election.cc:290] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:17.475464  3247 raft_consensus.cc:3060] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:17.486383  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 122897us
W20260709 12:23:17.486974  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 109164us
I20260709 12:23:17.497870  3438 ts_tablet_manager.cc:1434] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.144s	user 0.014s	sys 0.001s
I20260709 12:23:17.480080  3542 raft_consensus.cc:515] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.515901  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f76214f21f642889fe174df15bb7047" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:17.517863  3288 leader_election.cc:343] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.503954  3705 raft_consensus.cc:493] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.521124  3440 raft_consensus.cc:359] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.524190  3440 raft_consensus.cc:385] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.524502  3440 raft_consensus.cc:740] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
W20260709 12:23:17.525298  3195 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:17.529431  3705 raft_consensus.cc:515] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.556957  3439 tablet_bootstrap.cc:492] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:17.590276  3439 ts_tablet_manager.cc:1403] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.486s	user 0.015s	sys 0.018s
I20260709 12:23:17.588114  3438 tablet_bootstrap.cc:492] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:17.591940  3721 raft_consensus.cc:493] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.592666  3721 raft_consensus.cc:515] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.596187  3721 leader_election.cc:290] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:17.587407  3440 consensus_queue.cc:260] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.553704  3758 raft_consensus.cc:493] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.606792  3438 tablet_bootstrap.cc:654] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.579495  3542 leader_election.cc:290] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:17.613696  3757 raft_consensus.cc:493] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.622049  3440 ts_tablet_manager.cc:1434] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.153s	user 0.007s	sys 0.002s
I20260709 12:23:17.623628  3440 tablet_bootstrap.cc:492] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:17.636415  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:17.638244  3288 leader_election.cc:343] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.617594  3757 raft_consensus.cc:515] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.606062  3758 raft_consensus.cc:515] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.632694  3247 raft_consensus.cc:2468] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:17.649873  3439 raft_consensus.cc:359] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.650616  3439 raft_consensus.cc:385] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.650843  3439 raft_consensus.cc:740] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.651437  3439 consensus_queue.cc:260] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.652869  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a9849f17fd4313ab9ddbb06589c46f" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:17.660156  3439 ts_tablet_manager.cc:1434] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.069s	user 0.009s	sys 0.000s
I20260709 12:23:17.658129  3397 raft_consensus.cc:3060] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.654805  3705 leader_election.cc:290] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:17.660612  3758 leader_election.cc:290] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:17.666342  3396 raft_consensus.cc:2468] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
W20260709 12:23:17.732311  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 373145us
I20260709 12:23:17.734838  3439 tablet_bootstrap.cc:492] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:17.745857  3757 leader_election.cc:290] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:17.759150  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b70b620cf8243f4b39cf03eaae44881" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:17.761610  3440 tablet_bootstrap.cc:654] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.766908  3438 tablet_bootstrap.cc:492] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:17.767992  3438 ts_tablet_manager.cc:1403] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.180s	user 0.013s	sys 0.007s
I20260709 12:23:17.770792  3776 consensus_queue.cc:1048] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:23:17.798940  3288 leader_election.cc:304] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:17.801529  3243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2884104b65f4a589e915fd4328b729c" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:17.802726  3243 raft_consensus.cc:2468] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:17.810534  3749 raft_consensus.cc:2804] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.811805  3397 raft_consensus.cc:2468] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:17.812958  3438 raft_consensus.cc:359] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.813911  3438 raft_consensus.cc:385] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.815807  3438 raft_consensus.cc:740] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.818694  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2884104b65f4a589e915fd4328b729c" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:17.820173  3396 raft_consensus.cc:2468] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:17.816700  3438 consensus_queue.cc:260] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.830982  3749 raft_consensus.cc:697] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:17.832407  3749 consensus_queue.cc:237] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:17.836911  3288 leader_election.cc:343] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.885692  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:17.886695  3438 ts_tablet_manager.cc:1434] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.118s	user 0.010s	sys 0.004s
I20260709 12:23:17.890854  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b70b620cf8243f4b39cf03eaae44881" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:17.891316  3438 tablet_bootstrap.cc:492] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:17.899459  3393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f76214f21f642889fe174df15bb7047" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:17.907385  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
W20260709 12:23:17.921319  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 55543.4us
I20260709 12:23:17.949035  3439 tablet_bootstrap.cc:654] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.959630  3440 tablet_bootstrap.cc:492] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:17.968741  3440 ts_tablet_manager.cc:1403] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.345s	user 0.018s	sys 0.008s
I20260709 12:23:17.970602  3768 raft_consensus.cc:493] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.975061  3144 catalog_manager.cc:5697] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:17.980439  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:17.981767  3322 raft_consensus.cc:2468] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:17.971400  3768 raft_consensus.cc:515] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:17.989699  3318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2884104b65f4a589e915fd4328b729c" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:17.990732  3318 raft_consensus.cc:2468] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:17.986745  3768 leader_election.cc:290] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:18.005795  3440 raft_consensus.cc:359] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.006830  3440 raft_consensus.cc:385] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.010303  3440 raft_consensus.cc:740] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.012537  3440 consensus_queue.cc:260] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.014626  3315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:18.019624  3438 tablet_bootstrap.cc:654] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:18.034086  3289 leader_election.cc:343] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.035000  3289 leader_election.cc:304] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:18.048890  3640 raft_consensus.cc:2749] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.054704  3440 ts_tablet_manager.cc:1434] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.086s	user 0.007s	sys 0.002s
I20260709 12:23:18.056747  3440 tablet_bootstrap.cc:492] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
W20260709 12:23:18.057251  3289 leader_election.cc:343] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.064767  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:18.067093  3289 leader_election.cc:304] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:18.068315  3749 raft_consensus.cc:2749] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.069134  3214 leader_election.cc:304] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:18.072016  3439 tablet_bootstrap.cc:492] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:18.074186  3439 ts_tablet_manager.cc:1403] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.340s	user 0.035s	sys 0.028s
W20260709 12:23:18.075062  3289 leader_election.cc:343] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.075621  3289 leader_election.cc:304] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:18.076706  3749 raft_consensus.cc:2749] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.075615  3664 raft_consensus.cc:2764] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:23:18.079362  3439 raft_consensus.cc:359] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.080456  3439 raft_consensus.cc:385] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.080772  3439 raft_consensus.cc:740] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.081565  3318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2884104b65f4a589e915fd4328b729c" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:18.082331  3318 raft_consensus.cc:2468] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:18.083545  3214 leader_election.cc:304] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:18.084039  3439 consensus_queue.cc:260] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.087805  3664 raft_consensus.cc:2804] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.088250  3664 raft_consensus.cc:493] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.088582  3664 raft_consensus.cc:3060] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.090006  3439 ts_tablet_manager.cc:1434] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 12:23:18.091408  3439 tablet_bootstrap.cc:492] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:18.086722  3145 catalog_manager.cc:5697] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.106161  3438 tablet_bootstrap.cc:492] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:18.106865  3438 ts_tablet_manager.cc:1403] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.216s	user 0.013s	sys 0.010s
I20260709 12:23:18.109138  3440 tablet_bootstrap.cc:654] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.108913  3768 consensus_queue.cc:1048] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:23:18.114629  3362 leader_election.cc:304] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:18.115772  3438 raft_consensus.cc:359] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.116684  3438 raft_consensus.cc:385] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.117025  3438 raft_consensus.cc:740] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.117264  3462 raft_consensus.cc:2804] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.117744  3462 raft_consensus.cc:493] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.118122  3462 raft_consensus.cc:3060] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.119781  3438 consensus_queue.cc:260] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.121975  3763 raft_consensus.cc:493] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.128436  3763 raft_consensus.cc:515] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.131193  3763 leader_election.cc:290] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:18.133320  3664 raft_consensus.cc:515] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.135291  3364 leader_election.cc:304] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:18.135699  3664 leader_election.cc:290] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:18.136715  3704 consensus_queue.cc:1048] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:18.133303  3462 raft_consensus.cc:515] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.138015  3776 raft_consensus.cc:2804] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.139407  3750 raft_consensus.cc:493] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.140347  3438 ts_tablet_manager.cc:1434] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.003s
I20260709 12:23:18.140841  3768 consensus_queue.cc:1048] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:23:18.137364  3364 leader_election.cc:343] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:18.140251  3362 leader_election.cc:343] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.147621  3664 consensus_queue.cc:1048] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:18.147883  3318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2884104b65f4a589e915fd4328b729c" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:18.148635  3318 raft_consensus.cc:3060] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.148979  3750 raft_consensus.cc:515] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.152983  3774 consensus_queue.cc:1048] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:23:18.153695  3438 tablet_bootstrap.cc:492] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:18.156893  3439 tablet_bootstrap.cc:654] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.144443  3776 raft_consensus.cc:493] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.169230  3462 leader_election.cc:290] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:18.168030  3776 raft_consensus.cc:3060] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.175431  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:18.176319  3319 raft_consensus.cc:2468] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:18.176383  3440 tablet_bootstrap.cc:492] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:18.178174  3440 ts_tablet_manager.cc:1403] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.122s	user 0.022s	sys 0.015s
I20260709 12:23:18.182646  3750 leader_election.cc:290] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:23:18.184288  3362 leader_election.cc:343] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.184829  3362 leader_election.cc:304] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:18.187186  3704 raft_consensus.cc:2749] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.190620  3776 raft_consensus.cc:515] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.193264  3776 leader_election.cc:290] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:18.203348  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2884104b65f4a589e915fd4328b729c" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:18.204012  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11c9e1f3b048769b242da5c7ae7cd7" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:18.213074  3215 leader_election.cc:304] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:18.214078  3757 raft_consensus.cc:2804] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.214471  3757 raft_consensus.cc:493] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.214824  3757 raft_consensus.cc:3060] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.214812  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11c9e1f3b048769b242da5c7ae7cd7" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:18.209152  3768 raft_consensus.cc:493] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:18.219715  3289 leader_election.cc:343] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.218773  3768 raft_consensus.cc:515] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.222187  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2884104b65f4a589e915fd4328b729c" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:18.223206  3397 raft_consensus.cc:2393] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7cdaa0234c64450eb4ce4500b51b8a81 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:18.223882  3438 tablet_bootstrap.cc:654] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.224082  3768 leader_election.cc:290] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:18.227113  3243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:18.228047  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:18.228714  3320 raft_consensus.cc:3060] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:18.234304  3288 leader_election.cc:343] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.229291  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:18.234886  3288 leader_election.cc:304] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
W20260709 12:23:18.242070  3362 leader_election.cc:343] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.219940  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2884104b65f4a589e915fd4328b729c" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:18.248258  3750 raft_consensus.cc:2749] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.248162  3664 consensus_queue.cc:1048] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:18.250018  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:18.245498  3394 raft_consensus.cc:2468] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:18.255074  3243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:18.266176  3440 raft_consensus.cc:359] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.278309  3440 raft_consensus.cc:385] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.278666  3440 raft_consensus.cc:740] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.279419  3440 consensus_queue.cc:260] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.270412  3244 raft_consensus.cc:2393] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c93989d3dd634ef585d77b03354121a4 in current term 1: Already voted for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in this term.
I20260709 12:23:18.262372  3319 raft_consensus.cc:2468] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:18.284456  3318 raft_consensus.cc:2468] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:18.284915  3364 leader_election.cc:304] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:18.291644  3362 leader_election.cc:304] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:18.293618  3774 raft_consensus.cc:2749] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.298197  3440 ts_tablet_manager.cc:1434] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.120s	user 0.009s	sys 0.000s
I20260709 12:23:18.298514  3215 leader_election.cc:304] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: c93989d3dd634ef585d77b03354121a4
I20260709 12:23:18.299345  3462 raft_consensus.cc:2804] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.290434  3320 raft_consensus.cc:2468] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:18.312412  3364 leader_election.cc:304] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:18.314007  3705 raft_consensus.cc:2804] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.302994  3763 raft_consensus.cc:2804] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.316541  3462 raft_consensus.cc:493] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.317117  3462 raft_consensus.cc:3060] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.318686  3757 raft_consensus.cc:515] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.320145  3243 raft_consensus.cc:2393] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c93989d3dd634ef585d77b03354121a4 in current term 1: Already voted for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in this term.
I20260709 12:23:18.323559  3440 tablet_bootstrap.cc:492] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:18.326081  3705 raft_consensus.cc:697] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:18.327224  3705 consensus_queue.cc:237] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.332304  3757 leader_election.cc:290] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:18.333930  3439 tablet_bootstrap.cc:492] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:18.334451  3439 ts_tablet_manager.cc:1403] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.243s	user 0.020s	sys 0.009s
I20260709 12:23:18.339619  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:18.317108  3763 raft_consensus.cc:697] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:18.354182  3439 raft_consensus.cc:359] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.359246  3439 raft_consensus.cc:385] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.359588  3439 raft_consensus.cc:740] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.360386  3439 consensus_queue.cc:260] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.355185  3318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:18.361443  3318 raft_consensus.cc:3060] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.363245  3763 consensus_queue.cc:237] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.357784  3462 raft_consensus.cc:515] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.376083  3394 raft_consensus.cc:2393] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7cdaa0234c64450eb4ce4500b51b8a81 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:18.380759  3439 ts_tablet_manager.cc:1434] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 12:23:18.389379  3462 leader_election.cc:290] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:18.383844  3243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:18.424862  3439 tablet_bootstrap.cc:492] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:18.429361  3318 raft_consensus.cc:2468] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:18.430883  3243 raft_consensus.cc:2393] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c93989d3dd634ef585d77b03354121a4 in current term 1: Already voted for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in this term.
I20260709 12:23:18.432015  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:18.432966  3316 raft_consensus.cc:2393] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c93989d3dd634ef585d77b03354121a4 in current term 1: Already voted for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in this term.
I20260709 12:23:18.439146  3440 tablet_bootstrap.cc:654] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.442520  3640 consensus_queue.cc:1048] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:18.459998  3439 tablet_bootstrap.cc:654] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.462476  3438 tablet_bootstrap.cc:492] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:18.463280  3438 ts_tablet_manager.cc:1403] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.310s	user 0.020s	sys 0.028s
I20260709 12:23:18.490968  3438 raft_consensus.cc:359] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.491755  3438 raft_consensus.cc:385] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.492069  3438 raft_consensus.cc:740] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.492703  3438 consensus_queue.cc:260] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:18.496335  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 61824.5us
I20260709 12:23:18.519929  3804 consensus_queue.cc:1048] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:18.519426  3147 catalog_manager.cc:5697] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.539633  3438 ts_tablet_manager.cc:1434] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.076s	user 0.008s	sys 0.002s
I20260709 12:23:18.541126  3438 tablet_bootstrap.cc:492] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:18.552119  3215 leader_election.cc:304] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: c93989d3dd634ef585d77b03354121a4
I20260709 12:23:18.553666  3757 raft_consensus.cc:2804] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.554219  3757 raft_consensus.cc:697] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:18.555146  3757 consensus_queue.cc:237] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.567624  3364 leader_election.cc:304] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:18.569386  3462 raft_consensus.cc:2749] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.580181  3440 tablet_bootstrap.cc:492] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:18.583413  3440 ts_tablet_manager.cc:1403] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.266s	user 0.015s	sys 0.010s
I20260709 12:23:18.587138  3749 raft_consensus.cc:493] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.587845  3749 raft_consensus.cc:515] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.616384  3439 tablet_bootstrap.cc:492] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:18.617012  3439 ts_tablet_manager.cc:1403] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.210s	user 0.023s	sys 0.032s
I20260709 12:23:18.621873  3749 leader_election.cc:290] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:18.623457  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3795979cf042008b3447b47c75de09" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:18.630084  3288 leader_election.cc:343] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.631330  3440 raft_consensus.cc:359] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.634590  3721 raft_consensus.cc:493] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.635303  3721 raft_consensus.cc:515] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.631820  3439 raft_consensus.cc:359] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.636222  3439 raft_consensus.cc:385] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.636588  3439 raft_consensus.cc:740] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.637418  3439 consensus_queue.cc:260] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.639901  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04a8b7530d5486a880a89a4ba0faac8" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:18.642163  3288 leader_election.cc:343] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.645783  3440 raft_consensus.cc:385] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.646241  3440 raft_consensus.cc:740] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.650117  3721 leader_election.cc:290] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:18.656376  3439 ts_tablet_manager.cc:1434] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.003s
I20260709 12:23:18.656016  3440 consensus_queue.cc:260] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.656658  3147 catalog_manager.cc:5697] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:18.664707  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3795979cf042008b3447b47c75de09" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:18.667146  3393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04a8b7530d5486a880a89a4ba0faac8" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
W20260709 12:23:18.667757  3289 leader_election.cc:343] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.668663  3289 leader_election.cc:304] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:18.676410  3805 raft_consensus.cc:2749] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:18.682456  3289 leader_election.cc:343] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.683017  3289 leader_election.cc:304] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:18.683813  3805 raft_consensus.cc:2749] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.684285  3439 tablet_bootstrap.cc:492] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:18.690078  3438 tablet_bootstrap.cc:654] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.692615  3440 ts_tablet_manager.cc:1434] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.109s	user 0.006s	sys 0.002s
I20260709 12:23:18.694033  3440 tablet_bootstrap.cc:492] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:18.694842  3762 raft_consensus.cc:493] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.695418  3762 raft_consensus.cc:515] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:18.699251  3195 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:18.700742  3762 leader_election.cc:290] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:18.704762  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:18.714048  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:18.714962  3320 raft_consensus.cc:2393] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cdaa0234c64450eb4ce4500b51b8a81 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:18.716667  3439 tablet_bootstrap.cc:654] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.716154  3144 catalog_manager.cc:5697] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.721271  3215 leader_election.cc:304] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:18.724009  3763 raft_consensus.cc:3060] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.725780  3440 tablet_bootstrap.cc:654] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.740556  3762 raft_consensus.cc:493] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.741140  3762 raft_consensus.cc:515] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.746619  3393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b70b620cf8243f4b39cf03eaae44881" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:18.747437  3393 raft_consensus.cc:2468] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:18.748955  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b70b620cf8243f4b39cf03eaae44881" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:18.749720  3320 raft_consensus.cc:2468] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:18.748994  3763 raft_consensus.cc:2749] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.755755  3214 leader_election.cc:304] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:18.756706  3757 raft_consensus.cc:2804] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.757102  3757 raft_consensus.cc:493] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.757455  3757 raft_consensus.cc:3060] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.743335  3762 leader_election.cc:290] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:18.764581  3768 raft_consensus.cc:493] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.765618  3768 raft_consensus.cc:515] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.771672  3768 leader_election.cc:290] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:18.772467  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3015bb6b71f444dcb17062ac96b48cf5" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:18.773865  3362 leader_election.cc:343] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.777372  3757 raft_consensus.cc:515] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.783870  3393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b70b620cf8243f4b39cf03eaae44881" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:18.784680  3393 raft_consensus.cc:3060] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.785281  3757 leader_election.cc:290] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:18.788987  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3015bb6b71f444dcb17062ac96b48cf5" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:18.789721  3319 raft_consensus.cc:2468] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:18.791668  3438 tablet_bootstrap.cc:492] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:18.792135  3438 ts_tablet_manager.cc:1403] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.251s	user 0.009s	sys 0.007s
I20260709 12:23:18.793161  3439 tablet_bootstrap.cc:492] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:18.794217  3439 ts_tablet_manager.cc:1403] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.110s	user 0.019s	sys 0.030s
I20260709 12:23:18.788149  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b70b620cf8243f4b39cf03eaae44881" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:18.795390  3320 raft_consensus.cc:3060] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.800161  3364 leader_election.cc:304] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:18.802043  3774 raft_consensus.cc:2804] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.802522  3774 raft_consensus.cc:493] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.802891  3774 raft_consensus.cc:3060] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.820797  3774 raft_consensus.cc:515] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.823856  3774 leader_election.cc:290] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:18.825701  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3015bb6b71f444dcb17062ac96b48cf5" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:18.829082  3440 tablet_bootstrap.cc:492] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:18.829814  3440 ts_tablet_manager.cc:1403] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.136s	user 0.016s	sys 0.000s
I20260709 12:23:18.831386  3439 raft_consensus.cc:359] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:18.827769  3362 leader_election.cc:343] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.832244  3439 raft_consensus.cc:385] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.833820  3439 raft_consensus.cc:740] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.834723  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3015bb6b71f444dcb17062ac96b48cf5" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:18.835453  3319 raft_consensus.cc:3060] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.837570  3462 consensus_queue.cc:1048] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:18.839363  3440 raft_consensus.cc:359] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.840277  3440 raft_consensus.cc:385] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.840636  3440 raft_consensus.cc:740] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.841750  3438 raft_consensus.cc:359] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.842625  3438 raft_consensus.cc:385] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.841493  3440 consensus_queue.cc:260] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.842955  3438 raft_consensus.cc:740] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.843891  3438 consensus_queue.cc:260] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.846809  3439 consensus_queue.cc:260] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.832629  3393 raft_consensus.cc:2468] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:18.851574  3438 ts_tablet_manager.cc:1434] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.002s
I20260709 12:23:18.853441  3214 leader_election.cc:304] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:18.854408  3757 raft_consensus.cc:2804] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.855006  3757 raft_consensus.cc:697] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:18.855907  3757 consensus_queue.cc:237] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.839927  3320 raft_consensus.cc:2468] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:18.862774  3439 ts_tablet_manager.cc:1434] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.001s
I20260709 12:23:18.863272  3440 ts_tablet_manager.cc:1434] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 12:23:18.864059  3319 raft_consensus.cc:2468] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:18.864715  3440 tablet_bootstrap.cc:492] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:18.865653  3364 leader_election.cc:304] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:18.866796  3774 raft_consensus.cc:2804] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.867650  3774 raft_consensus.cc:697] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:18.868491  3774 consensus_queue.cc:237] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:18.866537  3439 tablet_bootstrap.cc:492] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:18.874760  3438 tablet_bootstrap.cc:492] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:18.884861  3704 consensus_queue.cc:1048] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:23:18.894778  3438 tablet_bootstrap.cc:654] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.941054  3440 tablet_bootstrap.cc:654] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.954988  3439 tablet_bootstrap.cc:654] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.936581  3763 consensus_queue.cc:1048] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:23:18.979736  3438 tablet_bootstrap.cc:492] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:18.980221  3438 ts_tablet_manager.cc:1403] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.106s	user 0.007s	sys 0.011s
I20260709 12:23:18.988049  3762 consensus_queue.cc:1048] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.997655  3439 tablet_bootstrap.cc:492] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
W20260709 12:23:19.000016  3816 log.cc:927] Time spent T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4: Append to log took a long time: real 0.083s	user 0.002s	sys 0.002s
I20260709 12:23:18.998821  3438 raft_consensus.cc:359] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.001541  3438 raft_consensus.cc:385] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.001953  3438 raft_consensus.cc:740] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.002857  3438 consensus_queue.cc:260] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.003858  3439 ts_tablet_manager.cc:1403] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.140s	user 0.015s	sys 0.001s
I20260709 12:23:19.004782  3757 consensus_queue.cc:1048] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.005704  3438 ts_tablet_manager.cc:1434] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 12:23:19.006920  3438 tablet_bootstrap.cc:492] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:19.007004  3439 raft_consensus.cc:359] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.016376  3440 tablet_bootstrap.cc:492] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:19.021296  3440 ts_tablet_manager.cc:1403] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.157s	user 0.008s	sys 0.011s
I20260709 12:23:19.019869  3757 raft_consensus.cc:493] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.021989  3439 raft_consensus.cc:385] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.023056  3439 raft_consensus.cc:740] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.023618  3439 consensus_queue.cc:260] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.040994  3721 raft_consensus.cc:493] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.042018  3774 raft_consensus.cc:493] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.042505  3774 raft_consensus.cc:515] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.045915  3774 leader_election.cc:290] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:19.052788  3317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:19.053224  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:19.050529  3440 raft_consensus.cc:359] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.054591  3440 raft_consensus.cc:385] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.054989  3440 raft_consensus.cc:740] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
W20260709 12:23:19.057281  3364 leader_election.cc:343] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.028853  3144 catalog_manager.cc:5697] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.027362  3757 raft_consensus.cc:515] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.041585  3721 raft_consensus.cc:515] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.061851  3757 leader_election.cc:290] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:19.062837  3721 leader_election.cc:290] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:19.065212  3440 consensus_queue.cc:260] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.066849  3247 raft_consensus.cc:2468] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:19.069998  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:19.079012  3439 ts_tablet_manager.cc:1434] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.075s	user 0.004s	sys 0.004s
I20260709 12:23:19.080224  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4a63fc24af42ca862894414f658194" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
W20260709 12:23:19.082362  3289 leader_election.cc:343] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.083226  3462 raft_consensus.cc:493] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.083730  3462 raft_consensus.cc:515] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.093439  3462 leader_election.cc:290] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:19.077046  3763 consensus_queue.cc:1048] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:23:19.097812  3243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:19.092662  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:19.100152  3440 ts_tablet_manager.cc:1434] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.078s	user 0.010s	sys 0.000s
W20260709 12:23:19.102385  3215 leader_election.cc:343] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:19.103125  3362 leader_election.cc:343] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.103727  3362 leader_election.cc:304] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:19.099457  3317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:19.094488  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4a63fc24af42ca862894414f658194" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:19.078637  3397 raft_consensus.cc:2468] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
W20260709 12:23:19.108086  3288 leader_election.cc:343] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.109839  3462 raft_consensus.cc:2804] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.109998  3288 leader_election.cc:304] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
W20260709 12:23:19.111780  3364 leader_election.cc:343] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.112373  3364 leader_election.cc:304] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:19.114913  3462 raft_consensus.cc:493] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.115335  3462 raft_consensus.cc:3060] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.113055  3704 raft_consensus.cc:2749] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:19.122602  3805 raft_consensus.cc:2749] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:19.123435  3440 tablet_bootstrap.cc:492] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:19.127061  3438 tablet_bootstrap.cc:654] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.130625  3214 leader_election.cc:304] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:19.141988  3757 raft_consensus.cc:2804] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.142494  3757 raft_consensus.cc:493] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.142891  3757 raft_consensus.cc:3060] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.144554  3439 tablet_bootstrap.cc:492] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:19.149230  3762 raft_consensus.cc:493] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.149879  3762 raft_consensus.cc:515] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.152112  3762 leader_election.cc:290] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:19.158712  3393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3015bb6b71f444dcb17062ac96b48cf5" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:19.164149  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3015bb6b71f444dcb17062ac96b48cf5" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:19.165251  3320 raft_consensus.cc:2393] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cdaa0234c64450eb4ce4500b51b8a81 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:19.168110  3215 leader_election.cc:304] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:19.170825  3144 catalog_manager.cc:5697] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:19.183476  3763 raft_consensus.cc:3060] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.185719  3757 raft_consensus.cc:515] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.187995  3757 leader_election.cc:290] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:19.189131  3393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:19.201593  3440 tablet_bootstrap.cc:654] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.201992  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
W20260709 12:23:19.208355  3215 leader_election.cc:343] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.234115  3462 raft_consensus.cc:515] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.235780  3393 raft_consensus.cc:2393] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7cdaa0234c64450eb4ce4500b51b8a81 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:19.238669  3214 leader_election.cc:304] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:19.240309  3462 leader_election.cc:290] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:19.239969  3766 raft_consensus.cc:2749] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:19.242707  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
W20260709 12:23:19.244196  3364 leader_election.cc:343] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.247270  3438 tablet_bootstrap.cc:492] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:19.247813  3438 ts_tablet_manager.cc:1403] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.241s	user 0.015s	sys 0.005s
I20260709 12:23:19.248188  3439 tablet_bootstrap.cc:654] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.252703  3243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:19.253755  3243 raft_consensus.cc:2393] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c93989d3dd634ef585d77b03354121a4 in current term 1: Already voted for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in this term.
I20260709 12:23:19.256916  3438 raft_consensus.cc:359] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.259305  3438 raft_consensus.cc:385] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.259709  3438 raft_consensus.cc:740] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.260435  3438 consensus_queue.cc:260] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.263275  3362 leader_election.cc:304] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:19.264115  3704 raft_consensus.cc:2749] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:19.252108  3763 raft_consensus.cc:2749] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:19.271807  3438 ts_tablet_manager.cc:1434] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.003s
I20260709 12:23:19.274024  3440 tablet_bootstrap.cc:492] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:19.274748  3440 ts_tablet_manager.cc:1403] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.152s	user 0.023s	sys 0.000s
I20260709 12:23:19.284138  3438 tablet_bootstrap.cc:492] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:19.296468  3440 raft_consensus.cc:359] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.297116  3440 raft_consensus.cc:385] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.297389  3440 raft_consensus.cc:740] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.298050  3440 consensus_queue.cc:260] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.300087  3440 ts_tablet_manager.cc:1434] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.000s
I20260709 12:23:19.304850  3438 tablet_bootstrap.cc:654] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.307574  3439 tablet_bootstrap.cc:492] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:19.307753  3440 tablet_bootstrap.cc:492] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:19.308207  3439 ts_tablet_manager.cc:1403] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.199s	user 0.008s	sys 0.011s
I20260709 12:23:19.311554  3439 raft_consensus.cc:359] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.312399  3439 raft_consensus.cc:385] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.312815  3439 raft_consensus.cc:740] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.313588  3439 consensus_queue.cc:260] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.318396  3439 ts_tablet_manager.cc:1434] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:23:19.319762  3440 tablet_bootstrap.cc:654] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.320379  3439 tablet_bootstrap.cc:492] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:19.324391  3438 tablet_bootstrap.cc:492] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:19.325030  3438 ts_tablet_manager.cc:1403] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.049s	user 0.011s	sys 0.003s
I20260709 12:23:19.328061  3438 raft_consensus.cc:359] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.328835  3438 raft_consensus.cc:385] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.329149  3438 raft_consensus.cc:740] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.329885  3438 consensus_queue.cc:260] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.331938  3439 tablet_bootstrap.cc:654] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.332338  3438 ts_tablet_manager.cc:1434] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:23:19.337265  3438 tablet_bootstrap.cc:492] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:19.342993  3440 tablet_bootstrap.cc:492] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:19.343667  3440 ts_tablet_manager.cc:1403] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.001s
I20260709 12:23:19.345412  3438 tablet_bootstrap.cc:654] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.348697  3439 tablet_bootstrap.cc:492] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:19.349203  3439 ts_tablet_manager.cc:1403] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.006s
I20260709 12:23:19.349112  3440 raft_consensus.cc:359] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.350046  3440 raft_consensus.cc:385] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.350481  3440 raft_consensus.cc:740] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.351181  3440 consensus_queue.cc:260] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.352074  3439 raft_consensus.cc:359] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.352720  3439 raft_consensus.cc:385] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.353000  3439 raft_consensus.cc:740] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.353991  3440 ts_tablet_manager.cc:1434] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:23:19.355154  3440 tablet_bootstrap.cc:492] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:19.356653  3439 consensus_queue.cc:260] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.358942  3439 ts_tablet_manager.cc:1434] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:23:19.362442  3440 tablet_bootstrap.cc:654] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.365374  3439 tablet_bootstrap.cc:492] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:19.374640  3766 consensus_queue.cc:1048] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:19.396742  3438 tablet_bootstrap.cc:492] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:19.397454  3438 ts_tablet_manager.cc:1403] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.061s	user 0.010s	sys 0.003s
I20260709 12:23:19.398653  3439 tablet_bootstrap.cc:654] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.403352  3757 consensus_queue.cc:1048] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:19.405651  3462 consensus_queue.cc:1048] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:19.415817  3440 tablet_bootstrap.cc:492] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:19.418445  3440 ts_tablet_manager.cc:1403] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.063s	user 0.021s	sys 0.013s
I20260709 12:23:19.422572  3440 raft_consensus.cc:359] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.418543  3438 raft_consensus.cc:359] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.426095  3438 raft_consensus.cc:385] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.426414  3438 raft_consensus.cc:740] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.430691  3440 raft_consensus.cc:385] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.431063  3440 raft_consensus.cc:740] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.431780  3440 consensus_queue.cc:260] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.427234  3438 consensus_queue.cc:260] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.434939  3438 ts_tablet_manager.cc:1434] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.003s
I20260709 12:23:19.434983  3440 ts_tablet_manager.cc:1434] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 12:23:19.436744  3440 tablet_bootstrap.cc:492] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:19.442831  3750 raft_consensus.cc:493] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.443423  3750 raft_consensus.cc:515] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.446540  3750 leader_election.cc:290] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:19.447295  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:19.449690  3249 raft_consensus.cc:2468] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:19.448351  3393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:19.451007  3288 leader_election.cc:304] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:19.451231  3393 raft_consensus.cc:2468] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:19.452687  3462 raft_consensus.cc:493] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.453212  3462 raft_consensus.cc:515] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.453840  3749 raft_consensus.cc:2804] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.454267  3749 raft_consensus.cc:493] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.454628  3749 raft_consensus.cc:3060] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.456777  3243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f76214f21f642889fe174df15bb7047" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:19.457520  3243 raft_consensus.cc:2468] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:19.459223  3362 leader_election.cc:304] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:19.460148  3768 raft_consensus.cc:2804] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.460575  3768 raft_consensus.cc:493] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.461071  3768 raft_consensus.cc:3060] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.464480  3462 leader_election.cc:290] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:19.467823  3438 tablet_bootstrap.cc:492] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:19.474581  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f76214f21f642889fe174df15bb7047" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:19.478387  3705 consensus_queue.cc:1048] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:19.480535  3805 raft_consensus.cc:493] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.481033  3439 tablet_bootstrap.cc:492] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:19.481613  3439 ts_tablet_manager.cc:1403] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.118s	user 0.016s	sys 0.015s
I20260709 12:23:19.480926  3440 tablet_bootstrap.cc:654] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.481374  3805 raft_consensus.cc:515] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.488989  3319 raft_consensus.cc:2468] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:19.493175  3438 tablet_bootstrap.cc:654] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.505378  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14193bb0c64048cf8ad561f477695970" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:19.507300  3805 leader_election.cc:290] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:23:19.521896  3288 leader_election.cc:343] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.498013  3749 raft_consensus.cc:515] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.541579  3749 leader_election.cc:290] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:19.545962  3768 raft_consensus.cc:515] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:19.562695  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 76354.2us
I20260709 12:23:19.568341  3768 leader_election.cc:290] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:19.570294  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:19.570380  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f76214f21f642889fe174df15bb7047" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:19.571131  3248 raft_consensus.cc:3060] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.571027  3249 raft_consensus.cc:3060] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.576048  3439 raft_consensus.cc:359] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.576848  3439 raft_consensus.cc:385] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.577149  3439 raft_consensus.cc:740] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.577883  3439 consensus_queue.cc:260] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.588186  3249 raft_consensus.cc:2468] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:19.589721  3288 leader_election.cc:304] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:19.590834  3838 raft_consensus.cc:2804] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.593912  3838 raft_consensus.cc:697] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:19.598448  3248 raft_consensus.cc:2468] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:19.600266  3439 ts_tablet_manager.cc:1434] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.118s	user 0.008s	sys 0.000s
I20260709 12:23:19.602984  3705 raft_consensus.cc:493] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.603526  3705 raft_consensus.cc:515] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.605834  3362 leader_election.cc:304] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:19.606740  3838 consensus_queue.cc:237] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.608472  3438 tablet_bootstrap.cc:492] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:19.609083  3438 ts_tablet_manager.cc:1403] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.142s	user 0.014s	sys 0.005s
I20260709 12:23:19.611713  3440 tablet_bootstrap.cc:492] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:19.612346  3440 ts_tablet_manager.cc:1403] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.176s	user 0.020s	sys 0.007s
I20260709 12:23:19.611734  3438 raft_consensus.cc:359] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.613003  3438 raft_consensus.cc:385] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.613384  3768 raft_consensus.cc:2804] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.613548  3438 raft_consensus.cc:740] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.614049  3768 raft_consensus.cc:697] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:19.614598  3438 consensus_queue.cc:260] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.615010  3768 consensus_queue.cc:237] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.615217  3440 raft_consensus.cc:359] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.616058  3440 raft_consensus.cc:385] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.616341  3440 raft_consensus.cc:740] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.617151  3440 consensus_queue.cc:260] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.622586  3438 ts_tablet_manager.cc:1434] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 12:23:19.612986  3705 leader_election.cc:290] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:19.623587  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11c9e1f3b048769b242da5c7ae7cd7" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:19.625311  3362 leader_election.cc:343] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.625452  3440 ts_tablet_manager.cc:1434] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:23:19.625226  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14193bb0c64048cf8ad561f477695970" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:19.637238  3439 tablet_bootstrap.cc:492] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:19.639734  3438 tablet_bootstrap.cc:492] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:19.636003  3763 raft_consensus.cc:493] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.642598  3763 raft_consensus.cc:515] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.649753  3393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:19.650486  3393 raft_consensus.cc:3060] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.646014  3317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f76214f21f642889fe174df15bb7047" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:19.654353  3317 raft_consensus.cc:3060] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.664537  3757 raft_consensus.cc:493] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.665256  3757 raft_consensus.cc:515] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.667977  3763 leader_election.cc:290] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:19.670445  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcfe15edc4947b988fd204a12c79094" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:19.671768  3440 tablet_bootstrap.cc:492] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
W20260709 12:23:19.675340  3214 leader_election.cc:343] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.683997  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff1130a2f274bccbc5b15389deacdc1" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:19.675343  3757 leader_election.cc:290] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:23:19.686204  3214 leader_election.cc:343] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.682646  3705 raft_consensus.cc:493] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.691613  3705 raft_consensus.cc:515] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.694631  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73563dc7466d4636a64ebd4d363935b1" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:19.696307  3362 leader_election.cc:343] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.701123  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11c9e1f3b048769b242da5c7ae7cd7" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:19.701927  3324 raft_consensus.cc:2468] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
W20260709 12:23:19.709039  3289 leader_election.cc:343] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.710160  3705 leader_election.cc:290] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:19.712070  3439 tablet_bootstrap.cc:654] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.705926  3840 raft_consensus.cc:493] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.718906  3840 raft_consensus.cc:515] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.717598  3289 leader_election.cc:304] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:19.706403  3393 raft_consensus.cc:2468] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:19.722150  3721 raft_consensus.cc:2749] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:19.722209  3840 leader_election.cc:290] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:19.733695  3147 catalog_manager.cc:5697] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:19.736697  3438 tablet_bootstrap.cc:654] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.746105  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff1130a2f274bccbc5b15389deacdc1" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:19.746771  3324 raft_consensus.cc:2468] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:19.747898  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcfe15edc4947b988fd204a12c79094" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:19.748528  3324 raft_consensus.cc:2468] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:19.749763  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73563dc7466d4636a64ebd4d363935b1" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:19.761140  3364 leader_election.cc:304] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:19.762529  3704 raft_consensus.cc:2804] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.762969  3704 raft_consensus.cc:493] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.763293  3704 raft_consensus.cc:3060] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.764138  3440 tablet_bootstrap.cc:654] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.765357  3393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff1130a2f274bccbc5b15389deacdc1" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:19.766564  3317 raft_consensus.cc:2468] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:19.774953  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff1130a2f274bccbc5b15389deacdc1" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:19.775682  3249 raft_consensus.cc:2468] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:19.778291  3215 leader_election.cc:304] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: c93989d3dd634ef585d77b03354121a4
W20260709 12:23:19.778959  3289 leader_election.cc:343] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.779666  3288 leader_election.cc:304] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: c93989d3dd634ef585d77b03354121a4
I20260709 12:23:19.780165  3215 leader_election.cc:304] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: c93989d3dd634ef585d77b03354121a4
W20260709 12:23:19.781306  3364 leader_election.cc:343] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.781809  3364 leader_election.cc:304] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:19.782692  3462 raft_consensus.cc:2749] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:19.784901  3763 raft_consensus.cc:2804] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.785357  3763 raft_consensus.cc:493] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.785701  3763 raft_consensus.cc:3060] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:19.795626  3195 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:19.788021  3762 raft_consensus.cc:2804] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.787184  3721 raft_consensus.cc:2804] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.803789  3144 catalog_manager.cc:5697] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.825536  3721 raft_consensus.cc:493] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.820122  3762 raft_consensus.cc:493] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.836681  3439 tablet_bootstrap.cc:492] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:19.826745  3838 raft_consensus.cc:493] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.838965  3439 ts_tablet_manager.cc:1403] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.203s	user 0.027s	sys 0.015s
I20260709 12:23:19.844959  3439 raft_consensus.cc:359] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.845575  3439 raft_consensus.cc:385] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.846547  3439 raft_consensus.cc:740] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.838325  3721 raft_consensus.cc:3060] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.840376  3762 raft_consensus.cc:3060] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.847508  3439 consensus_queue.cc:260] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.832829  3704 raft_consensus.cc:515] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.846190  3838 raft_consensus.cc:515] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:19.872879  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 44822.3us
I20260709 12:23:19.883163  3704 leader_election.cc:290] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:19.883404  3838 leader_election.cc:290] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:19.886348  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11c9e1f3b048769b242da5c7ae7cd7" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:19.886529  3393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:19.886905  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11c9e1f3b048769b242da5c7ae7cd7" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:19.887595  3320 raft_consensus.cc:3060] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.891134  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:19.891505  3289 leader_election.cc:343] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.879758  3763 raft_consensus.cc:515] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:19.894058  3288 leader_election.cc:343] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.894596  3288 leader_election.cc:304] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:19.895511  3438 tablet_bootstrap.cc:492] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:19.896128  3763 leader_election.cc:290] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:19.896122  3438 ts_tablet_manager.cc:1403] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.259s	user 0.011s	sys 0.022s
I20260709 12:23:19.896687  3838 raft_consensus.cc:2749] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:19.898545  3362 leader_election.cc:343] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.899909  3438 raft_consensus.cc:359] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.903512  3438 raft_consensus.cc:385] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.905089  3438 raft_consensus.cc:740] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.869131  3762 raft_consensus.cc:515] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.904500  3320 raft_consensus.cc:2468] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:19.906276  3393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff1130a2f274bccbc5b15389deacdc1" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:19.908798  3440 tablet_bootstrap.cc:492] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:19.909127  3393 raft_consensus.cc:3060] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.909390  3440 ts_tablet_manager.cc:1403] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.264s	user 0.023s	sys 0.037s
I20260709 12:23:19.902614  3721 raft_consensus.cc:515] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.904731  3439 ts_tablet_manager.cc:1434] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.063s	user 0.006s	sys 0.001s
I20260709 12:23:19.912767  3440 raft_consensus.cc:359] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.912792  3439 tablet_bootstrap.cc:492] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:19.915419  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff1130a2f274bccbc5b15389deacdc1" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:19.915694  3393 raft_consensus.cc:2468] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:19.916067  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff1130a2f274bccbc5b15389deacdc1" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:19.916831  3324 raft_consensus.cc:2393] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7cdaa0234c64450eb4ce4500b51b8a81 in current term 1: Already voted for candidate 8525f77f43714a58bf158d872c1db3e4 in this term.
I20260709 12:23:19.913350  3721 leader_election.cc:290] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:19.915740  3364 leader_election.cc:304] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:19.918445  3774 raft_consensus.cc:2804] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.916260  3249 raft_consensus.cc:2393] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8525f77f43714a58bf158d872c1db3e4 in current term 1: Already voted for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in this term.
I20260709 12:23:19.919147  3774 raft_consensus.cc:697] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:19.919483  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff1130a2f274bccbc5b15389deacdc1" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:19.918567  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcfe15edc4947b988fd204a12c79094" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:19.913614  3440 raft_consensus.cc:385] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.920295  3396 raft_consensus.cc:2393] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8525f77f43714a58bf158d872c1db3e4 in current term 1: Already voted for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in this term.
I20260709 12:23:19.920514  3440 raft_consensus.cc:740] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.922752  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcfe15edc4947b988fd204a12c79094" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:19.920446  3214 leader_election.cc:304] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:19.921576  3440 consensus_queue.cc:260] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.925249  3763 raft_consensus.cc:2804] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.921001  3774 consensus_queue.cc:237] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.926882  3288 leader_election.cc:304] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:19.928221  3838 raft_consensus.cc:2749] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:19.926158  3763 raft_consensus.cc:697] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:19.906554  3438 consensus_queue.cc:260] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.929850  3763 consensus_queue.cc:237] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.920264  3324 raft_consensus.cc:3060] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.936241  3438 ts_tablet_manager.cc:1434] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.002s
I20260709 12:23:19.937403  3438 tablet_bootstrap.cc:492] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
W20260709 12:23:19.939302  3214 leader_election.cc:343] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.941927  3757 raft_consensus.cc:493] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.942476  3757 raft_consensus.cc:515] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:19.917891  3762 leader_election.cc:290] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:19.944566  3757 leader_election.cc:290] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:19.945892  3440 ts_tablet_manager.cc:1434] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:23:19.950870  3440 tablet_bootstrap.cc:492] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:19.951265  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73563dc7466d4636a64ebd4d363935b1" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:19.953872  3393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73563dc7466d4636a64ebd4d363935b1" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:19.954578  3393 raft_consensus.cc:2468] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
W20260709 12:23:19.957516  3215 leader_election.cc:343] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.962287  3440 tablet_bootstrap.cc:654] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.966506  3214 leader_election.cc:304] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:19.968046  3849 raft_consensus.cc:2804] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.968549  3849 raft_consensus.cc:493] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.968916  3849 raft_consensus.cc:3060] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.982702  3324 raft_consensus.cc:2468] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:19.949298  3439 tablet_bootstrap.cc:654] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.984423  3147 catalog_manager.cc:5697] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:19.992565  3215 leader_election.cc:304] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: c93989d3dd634ef585d77b03354121a4
I20260709 12:23:19.995537  3762 raft_consensus.cc:2804] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.996456  3762 raft_consensus.cc:697] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:19.997378  3762 consensus_queue.cc:237] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.010567  3838 consensus_queue.cc:1048] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:20.016033  3849 raft_consensus.cc:515] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.033243  3438 tablet_bootstrap.cc:654] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.037844  3440 tablet_bootstrap.cc:492] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:20.039072  3440 ts_tablet_manager.cc:1403] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.088s	user 0.028s	sys 0.018s
I20260709 12:23:20.042557  3440 raft_consensus.cc:359] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.043479  3440 raft_consensus.cc:385] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.043853  3440 raft_consensus.cc:740] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.044711  3440 consensus_queue.cc:260] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.047744  3440 ts_tablet_manager.cc:1434] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:20.049072  3440 tablet_bootstrap.cc:492] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:20.056790  3440 tablet_bootstrap.cc:654] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.059221  3393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73563dc7466d4636a64ebd4d363935b1" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:20.059950  3393 raft_consensus.cc:3060] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.060987  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73563dc7466d4636a64ebd4d363935b1" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
W20260709 12:23:20.065490  3215 leader_election.cc:343] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.066933  3750 consensus_queue.cc:1048] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:20.099821  3439 tablet_bootstrap.cc:492] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:20.100457  3439 ts_tablet_manager.cc:1403] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.188s	user 0.008s	sys 0.008s
I20260709 12:23:20.103657  3439 raft_consensus.cc:359] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.104496  3439 raft_consensus.cc:385] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.104542  3393 raft_consensus.cc:2468] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:20.104877  3439 raft_consensus.cc:740] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.105825  3214 leader_election.cc:304] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:20.107065  3766 raft_consensus.cc:2804] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:20.107698  3766 raft_consensus.cc:697] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:20.108795  3766 consensus_queue.cc:237] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.115471  3439 consensus_queue.cc:260] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.118462  3439 ts_tablet_manager.cc:1434] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.005s
I20260709 12:23:20.122397  3439 tablet_bootstrap.cc:492] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:20.145578  3774 consensus_queue.cc:1048] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:20.154088  3439 tablet_bootstrap.cc:654] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.167375  3439 tablet_bootstrap.cc:492] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:20.169766  3439 ts_tablet_manager.cc:1403] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.002s
I20260709 12:23:20.176056  3439 raft_consensus.cc:359] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.180563  3439 raft_consensus.cc:385] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.181371  3439 raft_consensus.cc:740] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.182631  3439 consensus_queue.cc:260] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.187142  3439 ts_tablet_manager.cc:1434] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:23:20.190477  3439 tablet_bootstrap.cc:492] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:20.199323  3439 tablet_bootstrap.cc:654] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.199467  3849 leader_election.cc:290] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:20.213893  3705 consensus_queue.cc:1048] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:20.162377  3147 catalog_manager.cc:5697] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:20.230194  3438 tablet_bootstrap.cc:492] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:20.230775  3438 ts_tablet_manager.cc:1403] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.294s	user 0.009s	sys 0.002s
I20260709 12:23:20.268018  3438 raft_consensus.cc:359] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.269315  3438 raft_consensus.cc:385] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.269727  3438 raft_consensus.cc:740] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.273398  3144 catalog_manager.cc:5697] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:20.272410  3438 consensus_queue.cc:260] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.276578  3144 catalog_manager.cc:5697] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:20.272234  3840 raft_consensus.cc:493] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.284297  3840 raft_consensus.cc:515] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.288883  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:20.289762  3397 raft_consensus.cc:2393] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8525f77f43714a58bf158d872c1db3e4 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:20.288364  3840 leader_election.cc:290] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:20.291567  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:20.291968  3438 ts_tablet_manager.cc:1434] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.061s	user 0.009s	sys 0.000s
I20260709 12:23:20.292323  3249 raft_consensus.cc:2393] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8525f77f43714a58bf158d872c1db3e4 in current term 1: Already voted for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in this term.
I20260709 12:23:20.296041  3288 leader_election.cc:304] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:20.299124  3440 tablet_bootstrap.cc:492] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:20.299649  3440 ts_tablet_manager.cc:1403] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.251s	user 0.041s	sys 0.055s
I20260709 12:23:20.302585  3721 raft_consensus.cc:3060] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.309232  3438 tablet_bootstrap.cc:492] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:20.311859  3440 raft_consensus.cc:359] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.312700  3440 raft_consensus.cc:385] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.313020  3440 raft_consensus.cc:740] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.313854  3440 consensus_queue.cc:260] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.317154  3439 tablet_bootstrap.cc:492] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:20.317268  3440 ts_tablet_manager.cc:1434] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 12:23:20.345816  3439 ts_tablet_manager.cc:1403] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.156s	user 0.034s	sys 0.017s
I20260709 12:23:20.353966  3439 raft_consensus.cc:359] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.354938  3439 raft_consensus.cc:385] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.355296  3439 raft_consensus.cc:740] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.356096  3439 consensus_queue.cc:260] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.358044  3440 tablet_bootstrap.cc:492] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
W20260709 12:23:20.358930  3362 consensus_peers.cc:597] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:20.363173  3439 ts_tablet_manager.cc:1434] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:23:20.367515  3721 raft_consensus.cc:2749] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.357455  3438 tablet_bootstrap.cc:654] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.369962  3762 raft_consensus.cc:493] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.370865  3439 tablet_bootstrap.cc:492] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:20.370689  3762 raft_consensus.cc:515] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.374240  3762 leader_election.cc:290] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:20.390033  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:20.390882  3395 raft_consensus.cc:2468] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:20.392158  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:20.394519  3214 leader_election.cc:304] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:20.395643  3766 raft_consensus.cc:2804] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:20.396059  3766 raft_consensus.cc:493] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.396421  3766 raft_consensus.cc:3060] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:20.397475  3215 leader_election.cc:343] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.400581  3438 tablet_bootstrap.cc:492] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:20.401154  3438 ts_tablet_manager.cc:1403] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.094s	user 0.011s	sys 0.009s
I20260709 12:23:20.401213  3395 raft_consensus.cc:3060] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.402002  3440 tablet_bootstrap.cc:654] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.422516  3438 raft_consensus.cc:359] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.422813  3766 raft_consensus.cc:515] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.423507  3438 raft_consensus.cc:385] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.423929  3438 raft_consensus.cc:740] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.425995  3439 tablet_bootstrap.cc:654] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.428941  3438 consensus_queue.cc:260] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.431448  3721 raft_consensus.cc:493] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.439476  3438 ts_tablet_manager.cc:1434] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.001s
I20260709 12:23:20.445834  3438 tablet_bootstrap.cc:492] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4: Bootstrap starting.
I20260709 12:23:20.432019  3721 raft_consensus.cc:515] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.437422  3849 consensus_queue.cc:1048] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:23:20.442764  3766 leader_election.cc:290] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:20.440572  3317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:20.454461  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73563dc7466d4636a64ebd4d363935b1" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:20.456233  3750 raft_consensus.cc:493] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.456816  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73563dc7466d4636a64ebd4d363935b1" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:20.457856  3396 raft_consensus.cc:2393] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8525f77f43714a58bf158d872c1db3e4 in current term 1: Already voted for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in this term.
I20260709 12:23:20.450799  3439 tablet_bootstrap.cc:492] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:20.452971  3721 leader_election.cc:290] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:20.442828  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:20.456717  3750 raft_consensus.cc:515] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.463193  3395 raft_consensus.cc:3060] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.461525  3439 ts_tablet_manager.cc:1403] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.097s	user 0.010s	sys 0.009s
I20260709 12:23:20.465421  3289 leader_election.cc:304] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
W20260709 12:23:20.468164  3215 leader_election.cc:343] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.471691  3849 raft_consensus.cc:493] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.471309  3840 raft_consensus.cc:3060] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.472194  3849 raft_consensus.cc:515] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.476637  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a494da53f44c1d9780c046a155769a" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:20.477708  3766 consensus_queue.cc:1048] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:20.482218  3857 raft_consensus.cc:493] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.482823  3857 raft_consensus.cc:515] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:20.480715  3288 leader_election.cc:343] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.488466  3439 raft_consensus.cc:359] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.490945  3439 raft_consensus.cc:385] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.491205  3439 raft_consensus.cc:740] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.491842  3439 consensus_queue.cc:260] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.482998  3849 leader_election.cc:290] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:20.494272  3805 raft_consensus.cc:493] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.498451  3805 raft_consensus.cc:515] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.502833  3440 tablet_bootstrap.cc:492] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:20.503435  3440 ts_tablet_manager.cc:1403] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.180s	user 0.006s	sys 0.006s
I20260709 12:23:20.503829  3805 leader_election.cc:290] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:20.482841  3317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14193bb0c64048cf8ad561f477695970" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:20.507920  3317 raft_consensus.cc:2468] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 0.
I20260709 12:23:20.508347  3750 leader_election.cc:290] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:20.494526  3774 consensus_queue.cc:1048] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:23:20.509361  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14193bb0c64048cf8ad561f477695970" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:20.510141  3440 raft_consensus.cc:359] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.511564  3440 raft_consensus.cc:385] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.511871  3440 raft_consensus.cc:740] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.512646  3857 leader_election.cc:290] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:20.512602  3440 consensus_queue.cc:260] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.515308  3438 tablet_bootstrap.cc:654] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.517092  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3795979cf042008b3447b47c75de09" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:20.518276  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3795979cf042008b3447b47c75de09" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:20.518721  3215 leader_election.cc:304] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:20.520015  3762 raft_consensus.cc:2804] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:20.520454  3762 raft_consensus.cc:493] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.520576  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a494da53f44c1d9780c046a155769a" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:20.520808  3762 raft_consensus.cc:3060] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.521354  3394 raft_consensus.cc:2468] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:20.521673  3440 ts_tablet_manager.cc:1434] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 12:23:20.502763  3840 raft_consensus.cc:2749] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.518985  3323 raft_consensus.cc:2468] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:20.522382  3398 raft_consensus.cc:2468] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:20.523908  3364 leader_election.cc:304] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:20.524091  3289 leader_election.cc:304] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
W20260709 12:23:20.525276  3214 leader_election.cc:343] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.526639  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3795979cf042008b3447b47c75de09" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:20.527361  3248 raft_consensus.cc:2468] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:20.529143  3288 leader_election.cc:304] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:20.532042  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3795979cf042008b3447b47c75de09" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:20.532685  3248 raft_consensus.cc:2468] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:20.533008  3768 raft_consensus.cc:2804] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:20.533102  3750 raft_consensus.cc:2804] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:20.535241  3439 ts_tablet_manager.cc:1434] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.000s
I20260709 12:23:20.536175  3750 raft_consensus.cc:493] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.537109  3750 raft_consensus.cc:3060] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.541227  3440 tablet_bootstrap.cc:492] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:20.542089  3840 raft_consensus.cc:2804] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:20.554400  3439 tablet_bootstrap.cc:492] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:20.556349  3768 raft_consensus.cc:493] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.556887  3768 raft_consensus.cc:3060] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.550853  3840 raft_consensus.cc:493] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.559641  3840 raft_consensus.cc:3060] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.564296  3774 raft_consensus.cc:493] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.572067  3774 raft_consensus.cc:515] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.574241  3774 leader_election.cc:290] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:20.564599  3766 consensus_queue.cc:1048] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:23:20.576026  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:23:20.586158  3362 leader_election.cc:343] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.587743  3318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:20.588544  3318 raft_consensus.cc:2468] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 0.
I20260709 12:23:20.574137  3750 raft_consensus.cc:515] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.587791  3849 consensus_queue.cc:1048] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:23:20.583313  3395 raft_consensus.cc:2468] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:20.592151  3768 raft_consensus.cc:515] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.598488  3364 leader_election.cc:304] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:20.601591  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3795979cf042008b3447b47c75de09" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:20.602313  3249 raft_consensus.cc:3060] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.594421  3768 leader_election.cc:290] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:20.613318  3214 leader_election.cc:304] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:20.606650  3774 raft_consensus.cc:2804] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:20.622255  3750 leader_election.cc:290] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:20.622450  3774 raft_consensus.cc:493] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.622949  3774 raft_consensus.cc:3060] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.630425  3849 raft_consensus.cc:2804] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:20.631103  3849 raft_consensus.cc:697] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:20.615428  3318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3795979cf042008b3447b47c75de09" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:20.637017  3392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a494da53f44c1d9780c046a155769a" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:20.638757  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a494da53f44c1d9780c046a155769a" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:20.641981  3249 raft_consensus.cc:2468] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:20.648566  3362 leader_election.cc:304] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
W20260709 12:23:20.642756  3288 leader_election.cc:343] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.656960  3438 tablet_bootstrap.cc:492] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4: No bootstrap required, opened a new log
I20260709 12:23:20.657522  3438 ts_tablet_manager.cc:1403] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4: Time spent bootstrapping tablet: real 0.212s	user 0.012s	sys 0.012s
I20260709 12:23:20.657678  3440 tablet_bootstrap.cc:654] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.660948  3438 raft_consensus.cc:359] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.661684  3438 raft_consensus.cc:385] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.661936  3438 raft_consensus.cc:740] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.662637  3438 consensus_queue.cc:260] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.651649  3705 raft_consensus.cc:2804] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:20.637733  3392 raft_consensus.cc:3060] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.640640  3762 raft_consensus.cc:515] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.691129  3438 ts_tablet_manager.cc:1434] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.001s
I20260709 12:23:20.706226  3439 tablet_bootstrap.cc:654] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.706074  3762 leader_election.cc:290] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:20.664177  3849 consensus_queue.cc:237] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.725124  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14193bb0c64048cf8ad561f477695970" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:20.721405  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14193bb0c64048cf8ad561f477695970" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:20.720136  3757 consensus_queue.cc:1048] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 12:23:20.720729  3705 raft_consensus.cc:697] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:20.741959  3721 raft_consensus.cc:493] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.722990  3840 raft_consensus.cc:515] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.750494  3320 raft_consensus.cc:3060] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.738242  3774 raft_consensus.cc:515] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.783594  3318 raft_consensus.cc:2393] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c93989d3dd634ef585d77b03354121a4 in current term 1: Already voted for candidate 8525f77f43714a58bf158d872c1db3e4 in this term.
I20260709 12:23:20.784387  3840 leader_election.cc:290] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:20.785074  3774 leader_election.cc:290] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:20.757892  3721 raft_consensus.cc:515] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.787380  3721 leader_election.cc:290] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:23:20.840003  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 85709.9us
W20260709 12:23:20.844192  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 88593.5us
W20260709 12:23:20.845089  3214 leader_election.cc:343] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.853922  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:20.854650  3324 raft_consensus.cc:3060] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.875243  3440 tablet_bootstrap.cc:492] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:20.875713  3439 tablet_bootstrap.cc:492] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:20.875834  3440 ts_tablet_manager.cc:1403] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.335s	user 0.019s	sys 0.022s
I20260709 12:23:20.876359  3439 ts_tablet_manager.cc:1403] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.326s	user 0.013s	sys 0.005s
I20260709 12:23:20.878048  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:20.851115  3721 raft_consensus.cc:493] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.881423  3324 raft_consensus.cc:2468] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:23:20.830129  3705 consensus_queue.cc:237] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:20.890240  3195 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:20.887320  3721 raft_consensus.cc:515] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.897563  3721 leader_election.cc:290] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:20.900933  3320 raft_consensus.cc:2468] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
W20260709 12:23:20.922626  3362 leader_election.cc:343] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:20.932721  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 160927us
I20260709 12:23:20.935606  3440 raft_consensus.cc:359] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.936460  3440 raft_consensus.cc:385] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.936749  3440 raft_consensus.cc:740] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.937376  3440 consensus_queue.cc:260] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.942675  3852 raft_consensus.cc:493] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.943427  3852 raft_consensus.cc:515] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.991027  3440 ts_tablet_manager.cc:1434] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.115s	user 0.009s	sys 0.001s
I20260709 12:23:20.991551  3392 raft_consensus.cc:2468] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:20.993216  3439 raft_consensus.cc:359] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:20.994010  3439 raft_consensus.cc:385] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.994400  3439 raft_consensus.cc:740] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.008199  3852 leader_election.cc:290] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:21.014096  3905 raft_consensus.cc:493] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:21.014261  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 57466.9us
I20260709 12:23:21.015489  3215 leader_election.cc:304] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: c93989d3dd634ef585d77b03354121a4
I20260709 12:23:21.014890  3905 raft_consensus.cc:515] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.019291  3849 raft_consensus.cc:2804] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:21.020808  3849 raft_consensus.cc:697] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:21.024165  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:21.024922  3396 raft_consensus.cc:2468] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:21.018702  3906 consensus_queue.cc:1048] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:23:21.034230  3324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:21.035737  3324 raft_consensus.cc:2468] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 1.
I20260709 12:23:21.035015  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3795979cf042008b3447b47c75de09" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:21.036686  3248 raft_consensus.cc:2393] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8525f77f43714a58bf158d872c1db3e4 in current term 1: Already voted for candidate c93989d3dd634ef585d77b03354121a4 in this term.
I20260709 12:23:21.023095  3849 consensus_queue.cc:237] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.047395  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4a63fc24af42ca862894414f658194" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:21.048288  3249 raft_consensus.cc:2468] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:21.049734  3214 leader_election.cc:304] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:21.054116  3439 consensus_queue.cc:260] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.056809  3439 ts_tablet_manager.cc:1434] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.180s	user 0.012s	sys 0.000s
I20260709 12:23:21.057878  3899 raft_consensus.cc:2804] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.058343  3899 raft_consensus.cc:493] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.058565  3439 tablet_bootstrap.cc:492] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:21.058770  3899 raft_consensus.cc:3060] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.062721  3440 tablet_bootstrap.cc:492] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:21.062523  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04a8b7530d5486a880a89a4ba0faac8" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:21.071676  3147 catalog_manager.cc:5697] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:21.086611  3750 raft_consensus.cc:493] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.087288  3750 raft_consensus.cc:515] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.100174  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4a63fc24af42ca862894414f658194" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:21.100970  3289 leader_election.cc:304] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:21.122330  3840 raft_consensus.cc:2804] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:21.123164  3840 raft_consensus.cc:697] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:21.131856  3840 consensus_queue.cc:237] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.138460  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04a8b7530d5486a880a89a4ba0faac8" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:21.139284  3396 raft_consensus.cc:2468] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 0.
I20260709 12:23:21.142578  3288 leader_election.cc:304] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
W20260709 12:23:21.149217  3288 leader_election.cc:343] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.133037  3750 leader_election.cc:290] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:21.155108  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:21.151394  3838 raft_consensus.cc:2804] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:21.159808  3838 raft_consensus.cc:493] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.169168  3440 tablet_bootstrap.cc:654] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.175386  3838 raft_consensus.cc:3060] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.144163  3393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f3795979cf042008b3447b47c75de09" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:21.168843  3905 leader_election.cc:290] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:21.189471  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4a63fc24af42ca862894414f658194" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
W20260709 12:23:21.210613  3214 leader_election.cc:343] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.219677  3364 leader_election.cc:304] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:21.225231  3888 raft_consensus.cc:2804] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:21.217397  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:21.239151  3888 raft_consensus.cc:697] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:21.229939  3317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4a63fc24af42ca862894414f658194" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:21.244777  3439 tablet_bootstrap.cc:654] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.250666  3395 raft_consensus.cc:2393] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8525f77f43714a58bf158d872c1db3e4 in current term 1: Already voted for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in this term.
I20260709 12:23:21.254272  3888 consensus_queue.cc:237] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:21.283205  3289 leader_election.cc:343] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.285262  3289 leader_election.cc:304] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:21.253377  3838 raft_consensus.cc:515] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.269840  3899 raft_consensus.cc:515] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.289691  3840 raft_consensus.cc:2804] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:21.291251  3840 raft_consensus.cc:493] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.294691  3317 raft_consensus.cc:2393] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cdaa0234c64450eb4ce4500b51b8a81 in current term 1: Already voted for candidate 8525f77f43714a58bf158d872c1db3e4 in this term.
I20260709 12:23:21.299108  3899 leader_election.cc:290] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:21.302930  3840 raft_consensus.cc:3060] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.305836  3762 consensus_queue.cc:1048] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:21.309522  3838 leader_election.cc:290] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:21.329291  3246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4a63fc24af42ca862894414f658194" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:21.329839  3440 tablet_bootstrap.cc:492] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:21.330039  3246 raft_consensus.cc:3060] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.330760  3440 ts_tablet_manager.cc:1403] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.268s	user 0.018s	sys 0.014s
I20260709 12:23:21.338195  3439 tablet_bootstrap.cc:492] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:21.338896  3439 ts_tablet_manager.cc:1403] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.281s	user 0.014s	sys 0.006s
I20260709 12:23:21.341800  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:21.343843  3391 raft_consensus.cc:3060] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:21.352125  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 60109.6us
I20260709 12:23:21.357154  3289 leader_election.cc:304] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:21.358090  3897 raft_consensus.cc:2749] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:21.359288  3289 leader_election.cc:304] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:21.361536  3750 raft_consensus.cc:3060] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.369669  3215 leader_election.cc:304] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:21.389997  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4a63fc24af42ca862894414f658194" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:21.407763  3317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:21.408601  3317 raft_consensus.cc:3060] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:21.414909  3144 catalog_manager.cc:5697] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:21.416666  3840 raft_consensus.cc:515] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.421643  3391 raft_consensus.cc:2468] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:21.424678  3245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04a8b7530d5486a880a89a4ba0faac8" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:21.425649  3214 leader_election.cc:304] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
W20260709 12:23:21.428416  3288 leader_election.cc:343] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.427587  3849 raft_consensus.cc:2804] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:21.434486  3849 raft_consensus.cc:697] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:21.435355  3849 consensus_queue.cc:237] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.437392  3317 raft_consensus.cc:2468] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:21.438665  3750 raft_consensus.cc:2749] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:21.443781  3840 leader_election.cc:290] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:21.473186  3439 raft_consensus.cc:359] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.474030  3439 raft_consensus.cc:385] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.474284  3439 raft_consensus.cc:740] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.474967  3439 consensus_queue.cc:260] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.477061  3439 ts_tablet_manager.cc:1434] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.138s	user 0.006s	sys 0.000s
I20260709 12:23:21.482630  3147 catalog_manager.cc:5697] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:21.486083  3439 tablet_bootstrap.cc:492] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4: Bootstrap starting.
I20260709 12:23:21.473094  3440 raft_consensus.cc:359] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.491048  3440 raft_consensus.cc:385] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.491420  3440 raft_consensus.cc:740] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.497097  3246 raft_consensus.cc:2468] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:23:21.497824  3762 raft_consensus.cc:2749] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:21.500125  3288 leader_election.cc:304] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:21.500985  3840 raft_consensus.cc:2804] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:21.501770  3840 raft_consensus.cc:697] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:21.503137  3439 tablet_bootstrap.cc:654] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.505936  3440 consensus_queue.cc:260] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.502617  3840 consensus_queue.cc:237] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.503963  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04a8b7530d5486a880a89a4ba0faac8" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:21.507476  3391 raft_consensus.cc:3060] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.514158  3889 consensus_queue.cc:1048] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:21.516304  3391 raft_consensus.cc:2468] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
W20260709 12:23:21.519692  3289 leader_election.cc:343] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.525209  3289 leader_election.cc:304] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:21.528447  3440 ts_tablet_manager.cc:1434] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.197s	user 0.009s	sys 0.004s
I20260709 12:23:21.529814  3849 consensus_queue.cc:1048] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:21.530541  3750 raft_consensus.cc:2804] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:21.531124  3750 raft_consensus.cc:697] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:21.531620  3440 tablet_bootstrap.cc:492] T f8a494da53f44c1d9780c046a155769a P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:21.531975  3750 consensus_queue.cc:237] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.558552  3145 catalog_manager.cc:5697] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:21.575269  3768 raft_consensus.cc:493] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.583107  3439 tablet_bootstrap.cc:492] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4: No bootstrap required, opened a new log
I20260709 12:23:21.583777  3439 ts_tablet_manager.cc:1403] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4: Time spent bootstrapping tablet: real 0.098s	user 0.008s	sys 0.005s
I20260709 12:23:21.582394  3768 raft_consensus.cc:515] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.599229  3440 tablet_bootstrap.cc:654] T f8a494da53f44c1d9780c046a155769a P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.602766  3144 catalog_manager.cc:5697] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:21.600437  3896 consensus_queue.cc:1048] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:23:21.608615  3768 leader_election.cc:290] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:21.610929  3245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4a63fc24af42ca862894414f658194" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:21.612310  3245 raft_consensus.cc:2393] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c93989d3dd634ef585d77b03354121a4 in current term 1: Already voted for candidate 8525f77f43714a58bf158d872c1db3e4 in this term.
I20260709 12:23:21.613991  3144 catalog_manager.cc:5697] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 0 to 2, leader changed from <none> to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:21.619381  3439 raft_consensus.cc:359] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.620839  3439 raft_consensus.cc:385] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.625227  3439 raft_consensus.cc:740] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.626122  3439 consensus_queue.cc:260] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.628536  3439 ts_tablet_manager.cc:1434] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 12:23:21.632711  3440 tablet_bootstrap.cc:492] T f8a494da53f44c1d9780c046a155769a P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:21.633364  3440 ts_tablet_manager.cc:1403] T f8a494da53f44c1d9780c046a155769a P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.103s	user 0.014s	sys 0.000s
W20260709 12:23:21.637949  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 47463.5us
I20260709 12:23:21.637744  3440 raft_consensus.cc:359] T f8a494da53f44c1d9780c046a155769a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.638682  3440 raft_consensus.cc:385] T f8a494da53f44c1d9780c046a155769a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.639001  3440 raft_consensus.cc:740] T f8a494da53f44c1d9780c046a155769a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.639700  3440 consensus_queue.cc:260] T f8a494da53f44c1d9780c046a155769a P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.644197  3440 ts_tablet_manager.cc:1434] T f8a494da53f44c1d9780c046a155769a P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:23:21.645094  3242 raft_consensus.cc:3060] T f8a494da53f44c1d9780c046a155769a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.645906  3440 tablet_bootstrap.cc:492] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:21.660269  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4a63fc24af42ca862894414f658194" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:21.660981  3838 consensus_queue.cc:1048] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:21.682343  3440 tablet_bootstrap.cc:654] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.684973  3246 raft_consensus.cc:3060] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.690100  3364 leader_election.cc:304] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:21.691473  3145 catalog_manager.cc:5697] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:21.695271  3145 catalog_manager.cc:5697] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:21.699259  3899 consensus_queue.cc:1048] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:21.703761  3887 raft_consensus.cc:3060] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.727900  3889 consensus_queue.cc:1048] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:23:21.748533  3887 raft_consensus.cc:2749] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:21.765233  3440 tablet_bootstrap.cc:492] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:21.765789  3440 ts_tablet_manager.cc:1403] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.120s	user 0.020s	sys 0.014s
I20260709 12:23:21.774166  3440 raft_consensus.cc:359] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.774924  3440 raft_consensus.cc:385] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.775198  3440 raft_consensus.cc:740] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.778162  3440 consensus_queue.cc:260] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.779649  3391 raft_consensus.cc:3060] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.781977  3440 ts_tablet_manager.cc:1434] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.004s
I20260709 12:23:21.783581  3440 tablet_bootstrap.cc:492] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81: Bootstrap starting.
I20260709 12:23:21.796401  3840 consensus_queue.cc:1048] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:21.802086  3440 tablet_bootstrap.cc:654] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.807576  3762 consensus_queue.cc:1048] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:21.816800  3440 tablet_bootstrap.cc:492] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81: No bootstrap required, opened a new log
I20260709 12:23:21.817408  3440 ts_tablet_manager.cc:1403] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.000s
I20260709 12:23:21.820981  3440 raft_consensus.cc:359] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.821805  3440 raft_consensus.cc:385] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.822063  3440 raft_consensus.cc:740] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.822727  3440 consensus_queue.cc:260] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.825263  3440 ts_tablet_manager.cc:1434] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:21.830482  3887 consensus_queue.cc:1048] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:21.855158  3242 raft_consensus.cc:3060] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.900566  3899 raft_consensus.cc:493] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.901116  3899 raft_consensus.cc:515] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:21.901763  3323 raft_consensus.cc:1275] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:23:21.903192  3899 leader_election.cc:290] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:21.906042  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04a8b7530d5486a880a89a4ba0faac8" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:23:21.914240  3903 consensus_queue.cc:1048] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:21.916733  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04a8b7530d5486a880a89a4ba0faac8" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:21.921012  3395 raft_consensus.cc:2393] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cdaa0234c64450eb4ce4500b51b8a81 in current term 1: Already voted for candidate 8525f77f43714a58bf158d872c1db3e4 in this term.
I20260709 12:23:21.922463  3214 leader_election.cc:304] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:21.926213  3903 raft_consensus.cc:3060] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.932044  3391 raft_consensus.cc:1275] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:23:21.933807  3762 consensus_queue.cc:1048] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:21.954219  3903 raft_consensus.cc:2749] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:22.010313  3840 consensus_queue.cc:1048] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:22.027063  3840 consensus_queue.cc:1048] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:22.070930  3840 consensus_queue.cc:1048] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:22.120961  3838 consensus_queue.cc:1048] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:23.369513  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:23.379382  3950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:23.384945  3951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:23.390760  2067 server_base.cc:1061] running on GCE node
W20260709 12:23:23.394515  3953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:23.398286  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:23.398785  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:23.399121  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599803399098 us; error 0 us; skew 500 ppm
I20260709 12:23:23.400061  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:23.432335  2067 webserver.cc:533] Webserver started at http://127.2.4.196:34421/ using document root <none> and password file <none>
I20260709 12:23:23.432992  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:23.433262  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:23.433740  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:23.436035  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-3-root/instance:
uuid: "e38cc90af94c47f8a86be67ad610cbd4"
format_stamp: "Formatted at 2026-07-09 12:23:23 on dist-test-slave-4qvp"
I20260709 12:23:23.459908  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.023s	user 0.001s	sys 0.019s
I20260709 12:23:23.470571  3959 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:23.471700  2067 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 12:23:23.472186  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-3-root
uuid: "e38cc90af94c47f8a86be67ad610cbd4"
format_stamp: "Formatted at 2026-07-09 12:23:23 on dist-test-slave-4qvp"
I20260709 12:23:23.472658  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:23.506583  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:23.508562  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:23.511199  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:23.514659  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:23.514940  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:23.515167  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:23.515323  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:23.666046  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.196:45287
I20260709 12:23:23.666213  4023 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.196:45287 every 8 connection(s)
I20260709 12:23:23.684990  2067 auto_leader_rebalancer-test.cc:439] add a tserver: e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:23.708081  4024 heartbeater.cc:344] Connected to a master server at 127.2.4.254:39037
I20260709 12:23:23.708717  4024 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:23.709913  4024 heartbeater.cc:507] Master 127.2.4.254:39037 requested a full tablet report, sending...
I20260709 12:23:23.712801  3145 ts_manager.cc:194] Registered new tserver with Master: e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:23:23.715238  3145 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58104
I20260709 12:23:24.315136  3395 consensus_queue.cc:237] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:24.327358  3323 raft_consensus.cc:1275] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:24.331274  4028 consensus_queue.cc:1048] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:24.341780  3245 raft_consensus.cc:1275] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:24.344389  4028 consensus_queue.cc:1048] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:24.366923  3853 raft_consensus.cc:2955] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:24.372720  3323 raft_consensus.cc:2955] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
W20260709 12:23:24.375667  3362 consensus_peers.cc:597] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f76214f21f642889fe174df15bb7047. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:24.390337  3245 raft_consensus.cc:2955] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:24.404292  3395 consensus_queue.cc:237] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:24.445930  3145 catalog_manager.cc:5697] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:24.448755  3323 raft_consensus.cc:1275] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:24.450412  4028 consensus_queue.cc:1048] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:24.452036  3362 consensus_peers.cc:597] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c9145effc86e4e0db3de0ff579cd939d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:24.456930  3242 raft_consensus.cc:1275] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:24.459014  3853 consensus_queue.cc:1048] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:24.498064  4028 raft_consensus.cc:2955] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:24.500138  3323 raft_consensus.cc:2955] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:24.523404  3242 raft_consensus.cc:2955] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:24.582203  3145 catalog_manager.cc:5697] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:24.601620  3242 consensus_queue.cc:237] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:24.617018  3323 raft_consensus.cc:1275] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:24.618760  3905 consensus_queue.cc:1048] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:24.634092  3395 raft_consensus.cc:1275] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:24.637719  3905 consensus_queue.cc:1048] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:24.651084  3213 consensus_peers.cc:597] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a5d938460364390a17cada19308b314. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:24.653218  4049 raft_consensus.cc:2955] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:24.655548  3323 raft_consensus.cc:2955] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:24.663199  3395 raft_consensus.cc:2955] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:24.672713  3392 consensus_queue.cc:237] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:24.685181  3146 catalog_manager.cc:5697] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:24.753348  4024 heartbeater.cc:499] Master 127.2.4.254:39037 was elected leader, sending a full tablet report...
W20260709 12:23:24.753130  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 45789.2us
W20260709 12:23:24.774145  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 70375.4us
W20260709 12:23:24.786293  3362 consensus_peers.cc:597] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b8f62d0c16794e8c941849d55bb4bb0b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:24.786463  3246 raft_consensus.cc:1275] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:24.802254  4054 consensus_queue.cc:1048] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:24.820600  4057 raft_consensus.cc:2955] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:24.827044  3316 raft_consensus.cc:1275] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:24.830778  3246 raft_consensus.cc:2955] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:24.837553  4054 consensus_queue.cc:1048] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:24.848726  3146 catalog_manager.cc:5697] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:24.890094  3323 raft_consensus.cc:2955] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:24.914299  4061 ts_tablet_manager.cc:933] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:24.934789  4061 tablet_copy_client.cc:323] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:23:24.982661  4065 ts_tablet_manager.cc:933] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:24.992903  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet 1f76214f21f642889fe174df15bb7047 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:23:24.993773  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet 1f76214f21f642889fe174df15bb7047 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-1f76214f21f642889fe174df15bb7047
I20260709 12:23:25.004136  3408 tablet_copy_source_session.cc:215] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:25.034955  4061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f76214f21f642889fe174df15bb7047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:25.040745  4065 tablet_copy_client.cc:323] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:23:25.049922  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet c9145effc86e4e0db3de0ff579cd939d from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:23:25.050637  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet c9145effc86e4e0db3de0ff579cd939d from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-c9145effc86e4e0db3de0ff579cd939d
I20260709 12:23:25.054857  4061 tablet_copy_client.cc:806] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:25.055680  4061 tablet_copy_client.cc:670] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:25.071758  4061 tablet_copy_client.cc:538] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:25.077507  3408 tablet_copy_source_session.cc:215] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:25.085265  4061 tablet_bootstrap.cc:492] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:25.088724  4065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9145effc86e4e0db3de0ff579cd939d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:25.125444  4068 ts_tablet_manager.cc:933] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:25.137451  4068 tablet_copy_client.cc:323] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:23:25.139449  4061 tablet_bootstrap.cc:492] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:25.160926  4061 tablet_bootstrap.cc:492] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:25.163650  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet 0a5d938460364390a17cada19308b314 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:23:25.164409  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet 0a5d938460364390a17cada19308b314 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-0a5d938460364390a17cada19308b314
I20260709 12:23:25.165378  4061 ts_tablet_manager.cc:1403] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.080s	user 0.029s	sys 0.004s
I20260709 12:23:25.174772  4065 tablet_copy_client.cc:806] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:25.178519  3259 tablet_copy_source_session.cc:215] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:25.181670  4061 raft_consensus.cc:359] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:25.182415  4068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a5d938460364390a17cada19308b314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:25.187915  4061 raft_consensus.cc:740] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:25.189006  4061 consensus_queue.cc:260] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:25.184660  4065 tablet_copy_client.cc:670] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:25.197739  4065 tablet_copy_client.cc:538] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:25.198012  4061 ts_tablet_manager.cc:1434] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.002s
I20260709 12:23:25.203763  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-1f76214f21f642889fe174df15bb7047 received from {username='slave'} at 127.0.0.1:34216
I20260709 12:23:25.204310  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-1f76214f21f642889fe174df15bb7047 on tablet 1f76214f21f642889fe174df15bb7047 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:25.210409  4065 tablet_bootstrap.cc:492] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:25.222506  4068 tablet_copy_client.cc:806] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:25.223182  4068 tablet_copy_client.cc:670] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:25.268005  4065 tablet_bootstrap.cc:492] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:25.269110  4065 tablet_bootstrap.cc:492] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:25.269850  4065 ts_tablet_manager.cc:1403] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.060s	user 0.020s	sys 0.012s
I20260709 12:23:25.275471  4065 raft_consensus.cc:359] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:25.276502  4065 raft_consensus.cc:740] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:25.277174  4065 consensus_queue.cc:260] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:25.281996  4068 tablet_copy_client.cc:538] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:25.290920  4065 ts_tablet_manager.cc:1434] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:23:25.293741  4068 tablet_bootstrap.cc:492] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:25.304328  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-c9145effc86e4e0db3de0ff579cd939d received from {username='slave'} at 127.0.0.1:34216
I20260709 12:23:25.304822  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-c9145effc86e4e0db3de0ff579cd939d on tablet c9145effc86e4e0db3de0ff579cd939d with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:25.327816  4068 tablet_bootstrap.cc:492] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:25.330518  4068 tablet_bootstrap.cc:492] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:25.331305  4068 ts_tablet_manager.cc:1403] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.011s
I20260709 12:23:25.334758  4068 raft_consensus.cc:359] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:25.335716  4068 raft_consensus.cc:740] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:25.336428  4068 consensus_queue.cc:260] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:25.339080  4068 ts_tablet_manager.cc:1434] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:25.342779  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-0a5d938460364390a17cada19308b314 received from {username='slave'} at 127.0.0.1:57122
I20260709 12:23:25.343331  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-0a5d938460364390a17cada19308b314 on tablet 0a5d938460364390a17cada19308b314 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:25.382131  4068 ts_tablet_manager.cc:933] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:25.390487  4068 tablet_copy_client.cc:323] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:23:25.392364  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet b8f62d0c16794e8c941849d55bb4bb0b from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:23:25.392983  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet b8f62d0c16794e8c941849d55bb4bb0b from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-b8f62d0c16794e8c941849d55bb4bb0b
I20260709 12:23:25.401176  3408 tablet_copy_source_session.cc:215] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:25.404565  4068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f62d0c16794e8c941849d55bb4bb0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:25.423077  4068 tablet_copy_client.cc:806] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:25.423805  4068 tablet_copy_client.cc:670] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:25.428856  4068 tablet_copy_client.cc:538] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:25.441267  4068 tablet_bootstrap.cc:492] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:25.493014  4068 tablet_bootstrap.cc:492] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:25.493080  3998 raft_consensus.cc:1217] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:25.494266  4068 tablet_bootstrap.cc:492] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:25.495081  4068 ts_tablet_manager.cc:1403] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.054s	user 0.025s	sys 0.005s
I20260709 12:23:25.499778  4068 raft_consensus.cc:359] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:25.500723  4068 raft_consensus.cc:740] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:25.501497  4068 consensus_queue.cc:260] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:25.503952  4068 ts_tablet_manager.cc:1434] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:23:25.506170  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-b8f62d0c16794e8c941849d55bb4bb0b received from {username='slave'} at 127.0.0.1:34216
I20260709 12:23:25.506711  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-b8f62d0c16794e8c941849d55bb4bb0b on tablet b8f62d0c16794e8c941849d55bb4bb0b with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:25.525156  3998 raft_consensus.cc:1217] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:25.541873  3998 raft_consensus.cc:1217] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:23:25.579505  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 52584.6us
I20260709 12:23:25.767181  3998 raft_consensus.cc:1217] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:26.079131  4057 raft_consensus.cc:1064] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:26.084487  4057 consensus_queue.cc:237] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:26.085851  3967 raft_consensus.cc:1064] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:26.088622  3967 consensus_queue.cc:237] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:26.105661  3317 raft_consensus.cc:1275] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:26.107604  4054 consensus_queue.cc:1048] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:26.108165  3998 raft_consensus.cc:1275] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:26.109827  3905 consensus_queue.cc:1048] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:26.117094  3997 raft_consensus.cc:1275] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:26.119220  3245 raft_consensus.cc:1275] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:26.124015  4058 consensus_queue.cc:1048] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:26.125183  4054 consensus_queue.cc:1048] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:26.156839  4054 raft_consensus.cc:1064] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:26.159760  3323 raft_consensus.cc:1275] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:26.166993  3967 consensus_queue.cc:1048] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:26.182734  3393 raft_consensus.cc:1275] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:26.186863  4049 consensus_queue.cc:1048] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:23:26.239321  4078 log.cc:927] Time spent T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:23:26.189848  4054 consensus_queue.cc:237] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:23:26.266357  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 75281.9us
I20260709 12:23:26.289379  4057 raft_consensus.cc:1064] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:26.296787  4057 consensus_queue.cc:237] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:26.306862  4049 raft_consensus.cc:2955] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.317972  3245 raft_consensus.cc:1275] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:26.320729  3998 raft_consensus.cc:1275] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:26.324227  3323 raft_consensus.cc:2955] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.339679  3998 raft_consensus.cc:2955] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.341255  4086 raft_consensus.cc:2955] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.365445  3317 raft_consensus.cc:1275] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:26.370174  3144 catalog_manager.cc:5697] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:26.375640  4057 consensus_queue.cc:1048] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:26.382494  4057 consensus_queue.cc:1048] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:26.382972  3997 raft_consensus.cc:1275] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:26.384842  3996 raft_consensus.cc:2955] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:26.386663  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 53777.8us
I20260709 12:23:26.387682  3246 raft_consensus.cc:1275] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:26.390618  4086 consensus_queue.cc:1048] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:26.389671  3246 raft_consensus.cc:2955] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.401064  4057 consensus_queue.cc:1048] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:26.427314  3316 raft_consensus.cc:1275] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:26.429879  4089 consensus_queue.cc:1048] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:26.440502  4057 consensus_queue.cc:1048] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:26.442252  3316 raft_consensus.cc:2955] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.518524  3145 catalog_manager.cc:5697] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:23:26.560562  4096 log.cc:927] Time spent T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 12:23:26.566983  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 106162us
W20260709 12:23:26.601894  4095 log.cc:927] Time spent T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4: Append to log took a long time: real 0.075s	user 0.002s	sys 0.000s
I20260709 12:23:26.703519  3394 raft_consensus.cc:2955] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.743224  4104 raft_consensus.cc:2955] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.747973  3242 raft_consensus.cc:2955] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.782752  3316 raft_consensus.cc:2955] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.796916  4113 raft_consensus.cc:2955] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.807128  3249 raft_consensus.cc:2955] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.804548  3997 raft_consensus.cc:2955] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.870344  3997 raft_consensus.cc:2955] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.889294  3317 raft_consensus.cc:2955] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.903054  3242 consensus_queue.cc:237] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:26.923595  3145 catalog_manager.cc:5697] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.928021  3997 raft_consensus.cc:1275] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:26.929981  4108 consensus_queue.cc:1048] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:26.931938  3323 raft_consensus.cc:1275] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:26.935525  4108 consensus_queue.cc:1048] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:26.940053  3146 catalog_manager.cc:5697] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:26.942163  4105 raft_consensus.cc:2955] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.964151  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0a5d938460364390a17cada19308b314 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:26.965966  3997 raft_consensus.cc:2955] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.982208  3394 consensus_queue.cc:237] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:26.991801  3995 raft_consensus.cc:1275] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:26.991009  3147 catalog_manager.cc:5697] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:26.993572  4088 consensus_queue.cc:1048] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:23:26.997077  3132 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 0a5d938460364390a17cada19308b314 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20260709 12:23:26.998229  3317 raft_consensus.cc:2955] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:27.006376  3323 raft_consensus.cc:1275] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:27.009199  3394 consensus_queue.cc:237] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:27.010048  4104 raft_consensus.cc:2955] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:27.010389  3997 raft_consensus.cc:2955] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:27.019584  3995 raft_consensus.cc:1275] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:27.021325  4054 consensus_queue.cc:1048] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:27.026829  3379 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a5d938460364390a17cada19308b314 with delete_type TABLET_DATA_TOMBSTONED (TS c93989d3dd634ef585d77b03354121a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54706
I20260709 12:23:27.040913  4086 consensus_queue.cc:1048] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:27.046218  3317 raft_consensus.cc:1275] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:27.049183  4107 raft_consensus.cc:2955] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:27.048614  3323 raft_consensus.cc:2955] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:27.052754  3147 catalog_manager.cc:5697] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:27.054934  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1f76214f21f642889fe174df15bb7047 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:27.065541  3995 raft_consensus.cc:2955] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:27.067061  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b8f62d0c16794e8c941849d55bb4bb0b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:27.034202  3394 consensus_queue.cc:237] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:27.075971  3239 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f76214f21f642889fe174df15bb7047 with delete_type TABLET_DATA_TOMBSTONED (TS 7cdaa0234c64450eb4ce4500b51b8a81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53720
I20260709 12:23:27.085937  3997 raft_consensus.cc:1275] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:27.085708  4086 consensus_queue.cc:1048] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:27.087935  4054 consensus_queue.cc:1048] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:27.088637  3242 raft_consensus.cc:1275] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:27.093261  3316 raft_consensus.cc:2955] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:27.094180  4086 consensus_queue.cc:1048] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:27.099054  4120 tablet_replica.cc:333] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:23:27.105377  4088 raft_consensus.cc:2955] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:27.093190  4119 tablet_replica.cc:333] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:23:27.120930  4119 raft_consensus.cc:2243] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:27.122002  3995 raft_consensus.cc:2955] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:27.150558  4120 raft_consensus.cc:2243] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:27.151178  4119 raft_consensus.cc:2272] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:27.151337  4120 raft_consensus.cc:2272] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:27.155846  4120 ts_tablet_manager.cc:1916] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:27.119167  3242 raft_consensus.cc:2955] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:27.133453  3144 catalog_manager.cc:5697] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:27.142448  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c9145effc86e4e0db3de0ff579cd939d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:27.161909  4119 ts_tablet_manager.cc:1916] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:27.170023  3239 tablet_service.cc:1558] Processing DeleteTablet for tablet b8f62d0c16794e8c941849d55bb4bb0b with delete_type TABLET_DATA_TOMBSTONED (TS 7cdaa0234c64450eb4ce4500b51b8a81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53720
I20260709 12:23:27.190093  4120 ts_tablet_manager.cc:1929] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:27.190785  4120 log.cc:1199] T 1f76214f21f642889fe174df15bb7047 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/wals/1f76214f21f642889fe174df15bb7047
I20260709 12:23:27.192528  3132 catalog_manager.cc:5028] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): tablet 1f76214f21f642889fe174df15bb7047 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:27.193063  4120 tablet_replica.cc:333] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:23:27.194309  4120 raft_consensus.cc:2243] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:27.195245  4120 raft_consensus.cc:2272] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:27.199400  4120 ts_tablet_manager.cc:1916] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:27.204666  4119 ts_tablet_manager.cc:1929] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:27.205286  4119 log.cc:1199] T 0a5d938460364390a17cada19308b314 P c93989d3dd634ef585d77b03354121a4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/wals/0a5d938460364390a17cada19308b314
I20260709 12:23:27.207232  3133 catalog_manager.cc:5028] TS c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): tablet 0a5d938460364390a17cada19308b314 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:27.211318  3147 catalog_manager.cc:5697] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:27.219375  4120 ts_tablet_manager.cc:1929] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:27.219957  4120 log.cc:1199] T b8f62d0c16794e8c941849d55bb4bb0b P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/wals/b8f62d0c16794e8c941849d55bb4bb0b
I20260709 12:23:27.221781  3132 catalog_manager.cc:5028] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): tablet b8f62d0c16794e8c941849d55bb4bb0b (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:27.225523  3311 tablet_service.cc:1558] Processing DeleteTablet for tablet c9145effc86e4e0db3de0ff579cd939d with delete_type TABLET_DATA_TOMBSTONED (TS 8525f77f43714a58bf158d872c1db3e4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40236
I20260709 12:23:27.258080  4121 tablet_replica.cc:333] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:23:27.260461  4121 raft_consensus.cc:2243] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:27.264993  4121 raft_consensus.cc:2272] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:27.270001  4121 ts_tablet_manager.cc:1916] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:27.297983  4121 ts_tablet_manager.cc:1929] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:27.298597  4121 log.cc:1199] T c9145effc86e4e0db3de0ff579cd939d P 8525f77f43714a58bf158d872c1db3e4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/wals/c9145effc86e4e0db3de0ff579cd939d
I20260709 12:23:27.300575  3134 catalog_manager.cc:5028] TS 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): tablet c9145effc86e4e0db3de0ff579cd939d (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
W20260709 12:23:27.970573  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 70083.8us
I20260709 12:23:28.341176  3249 consensus_queue.cc:237] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:28.349704  3393 raft_consensus.cc:1275] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:28.350239  3317 raft_consensus.cc:1275] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:28.350581  3213 consensus_peers.cc:597] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3af545b01cb24f1cacb0056f804a7692. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:28.351799  4129 consensus_queue.cc:1048] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:28.352964  4105 consensus_queue.cc:1048] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:28.362867  4105 raft_consensus.cc:2955] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:28.364837  3323 raft_consensus.cc:2955] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:28.366294  3393 raft_consensus.cc:2955] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:28.389021  3249 consensus_queue.cc:237] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:28.389038  3146 catalog_manager.cc:5697] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
W20260709 12:23:28.429966  3213 consensus_peers.cc:597] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7ff1130a2f274bccbc5b15389deacdc1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:28.431687  3392 raft_consensus.cc:1275] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:28.435686  4105 consensus_queue.cc:1048] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:28.440472  3323 raft_consensus.cc:1275] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:28.444738  4130 consensus_queue.cc:1048] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:28.472151  4129 raft_consensus.cc:2955] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:28.473057  3392 raft_consensus.cc:2955] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:28.495379  3316 raft_consensus.cc:2955] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:28.508705  3249 consensus_queue.cc:237] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:28.513150  3144 catalog_manager.cc:5697] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:28.518245  3392 raft_consensus.cc:1275] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:28.518522  3317 raft_consensus.cc:1275] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:28.518854  3213 consensus_peers.cc:597] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 73563dc7466d4636a64ebd4d363935b1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:28.521166  4141 consensus_queue.cc:1048] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:28.523329  4129 consensus_queue.cc:1048] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:28.536423  4141 raft_consensus.cc:2955] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:28.538945  3316 raft_consensus.cc:2955] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:28.565254  3393 consensus_queue.cc:237] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:28.570452  3146 catalog_manager.cc:5697] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:28.580188  3316 raft_consensus.cc:1275] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:23:28.581146  3362 consensus_peers.cc:597] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 34c56ebe809a4c09b75ee18faf980d8c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:28.582190  4113 consensus_queue.cc:1048] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:28.583240  3249 raft_consensus.cc:1275] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:28.584834  4113 consensus_queue.cc:1048] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:28.587098  3393 raft_consensus.cc:2955] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:28.609333  3317 raft_consensus.cc:2955] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:28.607650  4086 raft_consensus.cc:2955] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:28.613613  3249 raft_consensus.cc:2955] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:28.639138  3146 catalog_manager.cc:5697] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 2 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:28.817687  4149 ts_tablet_manager.cc:933] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:28.820941  4149 tablet_copy_client.cc:323] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:23:28.823424  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet 3af545b01cb24f1cacb0056f804a7692 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:23:28.824122  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet 3af545b01cb24f1cacb0056f804a7692 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-3af545b01cb24f1cacb0056f804a7692
I20260709 12:23:28.846925  4151 ts_tablet_manager.cc:933] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:28.847492  3259 tablet_copy_source_session.cc:215] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:28.850771  4151 tablet_copy_client.cc:323] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:23:28.853004  4149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af545b01cb24f1cacb0056f804a7692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.857306  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet 7ff1130a2f274bccbc5b15389deacdc1 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:23:28.858215  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet 7ff1130a2f274bccbc5b15389deacdc1 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-7ff1130a2f274bccbc5b15389deacdc1
I20260709 12:23:28.868224  3259 tablet_copy_source_session.cc:215] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:28.873274  4151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff1130a2f274bccbc5b15389deacdc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.893126  4149 tablet_copy_client.cc:806] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:28.894191  4151 tablet_copy_client.cc:806] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:28.894286  4149 tablet_copy_client.cc:670] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:28.895038  4151 tablet_copy_client.cc:670] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:28.899245  4149 tablet_copy_client.cc:538] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:28.900888  4151 tablet_copy_client.cc:538] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:28.910758  4151 tablet_bootstrap.cc:492] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:28.910758  4149 tablet_bootstrap.cc:492] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:28.947005  4151 tablet_bootstrap.cc:492] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:28.948035  4151 tablet_bootstrap.cc:492] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:28.948745  4151 ts_tablet_manager.cc:1403] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.016s
I20260709 12:23:28.952193  4151 raft_consensus.cc:359] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:28.953063  4151 raft_consensus.cc:740] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:28.954473  4151 consensus_queue.cc:260] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:28.962806  4149 tablet_bootstrap.cc:492] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:28.963812  4149 tablet_bootstrap.cc:492] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:28.964579  4149 ts_tablet_manager.cc:1403] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.054s	user 0.018s	sys 0.018s
I20260709 12:23:28.969553  4149 raft_consensus.cc:359] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:28.970597  4149 raft_consensus.cc:740] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:28.971319  4149 consensus_queue.cc:260] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:28.973670  4151 ts_tablet_manager.cc:1434] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.025s	user 0.010s	sys 0.009s
I20260709 12:23:28.976617  4149 ts_tablet_manager.cc:1434] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.004s
I20260709 12:23:28.983428  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-7ff1130a2f274bccbc5b15389deacdc1 received from {username='slave'} at 127.0.0.1:57122
I20260709 12:23:28.984002  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-7ff1130a2f274bccbc5b15389deacdc1 on tablet 7ff1130a2f274bccbc5b15389deacdc1 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:28.987846  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-3af545b01cb24f1cacb0056f804a7692 received from {username='slave'} at 127.0.0.1:57122
I20260709 12:23:28.988312  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-3af545b01cb24f1cacb0056f804a7692 on tablet 3af545b01cb24f1cacb0056f804a7692 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:29.052407  4149 ts_tablet_manager.cc:933] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:29.054586  4149 tablet_copy_client.cc:323] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:23:29.057106  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet 34c56ebe809a4c09b75ee18faf980d8c from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:23:29.057911  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet 34c56ebe809a4c09b75ee18faf980d8c from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-34c56ebe809a4c09b75ee18faf980d8c
I20260709 12:23:29.067654  3408 tablet_copy_source_session.cc:215] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:29.071708  4151 ts_tablet_manager.cc:933] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:29.073302  4149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c56ebe809a4c09b75ee18faf980d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:29.077837  4151 tablet_copy_client.cc:323] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:23:29.079942  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet 73563dc7466d4636a64ebd4d363935b1 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:23:29.080576  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet 73563dc7466d4636a64ebd4d363935b1 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-73563dc7466d4636a64ebd4d363935b1
I20260709 12:23:29.094345  3259 tablet_copy_source_session.cc:215] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:29.094766  4149 tablet_copy_client.cc:806] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:29.097394  4149 tablet_copy_client.cc:670] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:29.105077  4151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73563dc7466d4636a64ebd4d363935b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:29.112386  4149 tablet_copy_client.cc:538] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:29.143015  4151 tablet_copy_client.cc:806] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:29.143702  4151 tablet_copy_client.cc:670] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:29.146394  4149 tablet_bootstrap.cc:492] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:29.148233  4151 tablet_copy_client.cc:538] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:29.165313  4151 tablet_bootstrap.cc:492] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:29.188138  4149 tablet_bootstrap.cc:492] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:29.189543  4149 tablet_bootstrap.cc:492] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:29.190467  4149 ts_tablet_manager.cc:1403] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.048s	user 0.027s	sys 0.003s
I20260709 12:23:29.195242  4149 raft_consensus.cc:359] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:29.196264  4149 raft_consensus.cc:740] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:29.196960  4151 tablet_bootstrap.cc:492] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:29.197018  4149 consensus_queue.cc:260] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:29.199621  4151 tablet_bootstrap.cc:492] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:29.200482  4151 ts_tablet_manager.cc:1403] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.036s	user 0.030s	sys 0.000s
I20260709 12:23:29.202553  4149 ts_tablet_manager.cc:1434] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:23:29.205499  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-34c56ebe809a4c09b75ee18faf980d8c received from {username='slave'} at 127.0.0.1:34216
I20260709 12:23:29.206035  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-34c56ebe809a4c09b75ee18faf980d8c on tablet 34c56ebe809a4c09b75ee18faf980d8c with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:29.213497  4151 raft_consensus.cc:359] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:29.214591  4151 raft_consensus.cc:740] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:29.215340  4151 consensus_queue.cc:260] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:29.218129  4151 ts_tablet_manager.cc:1434] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:23:29.220924  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-73563dc7466d4636a64ebd4d363935b1 received from {username='slave'} at 127.0.0.1:57122
I20260709 12:23:29.221462  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-73563dc7466d4636a64ebd4d363935b1 on tablet 73563dc7466d4636a64ebd4d363935b1 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:29.398625  3995 raft_consensus.cc:1217] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:29.404994  3995 raft_consensus.cc:1217] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:29.550433  3995 raft_consensus.cc:1217] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:29.565887  3995 raft_consensus.cc:1217] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:23:29.786944  4159 raft_consensus.cc:1064] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:29.789222  4159 consensus_queue.cc:237] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:29.797215  3995 raft_consensus.cc:1275] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.798436  3393 raft_consensus.cc:1275] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.800005  3322 raft_consensus.cc:1275] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.801424  4129 consensus_queue.cc:1048] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:29.802784  4162 consensus_queue.cc:1048] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:29.803752  4141 consensus_queue.cc:1048] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:29.843578  4141 raft_consensus.cc:2955] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:29.851517  3997 raft_consensus.cc:2955] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:29.863327  4129 raft_consensus.cc:1064] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:29.867329  3322 raft_consensus.cc:2955] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:29.870797  3393 raft_consensus.cc:2955] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:29.872097  4129 consensus_queue.cc:237] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:29.875020  3147 catalog_manager.cc:5697] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:29.894292  3995 raft_consensus.cc:1275] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.895861  3322 raft_consensus.cc:1275] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.896600  4159 consensus_queue.cc:1048] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:29.899122  4159 consensus_queue.cc:1048] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:29.913982  3392 raft_consensus.cc:1275] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:29.925199  4162 consensus_queue.cc:1048] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:30.004431  4162 raft_consensus.cc:1064] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:30.007864  4162 consensus_queue.cc:237] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:30.010663  4141 raft_consensus.cc:2955] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.015753  3397 raft_consensus.cc:2955] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.028988  4125 raft_consensus.cc:1064] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:30.032405  4125 consensus_queue.cc:237] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:30.044189  3322 raft_consensus.cc:2955] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.047178  3996 raft_consensus.cc:2955] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.052230  3997 raft_consensus.cc:1275] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:30.053586  3318 raft_consensus.cc:1275] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:30.056285  4159 consensus_queue.cc:1048] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:23:30.063704  4162 consensus_queue.cc:1048] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:30.067545  3397 raft_consensus.cc:1275] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:30.071887  4159 consensus_queue.cc:1048] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:30.097848  3995 raft_consensus.cc:1275] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:30.099968  3323 raft_consensus.cc:1275] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:30.100718  4113 consensus_queue.cc:1048] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:30.102366  4086 consensus_queue.cc:1048] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:30.113461  3244 raft_consensus.cc:1275] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:30.073570  3249 consensus_queue.cc:237] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:30.118522  3146 catalog_manager.cc:5697] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.128993  4162 raft_consensus.cc:2955] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.135703  3997 raft_consensus.cc:1275] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:30.137876  4141 consensus_queue.cc:1048] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:30.144726  4113 consensus_queue.cc:1048] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:23:30.148741  3288 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.2.4.254:39037, user_credentials={real_user=slave}} blocked reactor thread for 61153.9us
I20260709 12:23:30.152449  3244 consensus_queue.cc:237] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:30.157781  3322 raft_consensus.cc:2955] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:30.171583  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 51380.3us
W20260709 12:23:30.177848  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 61107us
I20260709 12:23:30.183678  3322 raft_consensus.cc:1275] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:30.204990  4188 consensus_queue.cc:1048] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:30.167219  4159 raft_consensus.cc:2955] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.252498  3391 raft_consensus.cc:2955] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.262879  3145 catalog_manager.cc:5697] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.304389  3316 raft_consensus.cc:2955] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.310421  3995 raft_consensus.cc:1275] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:30.315104  3996 raft_consensus.cc:2955] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.321952  3997 raft_consensus.cc:2955] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.329931  4129 consensus_queue.cc:1048] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:30.331302  3243 consensus_queue.cc:237] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:30.336524  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3af545b01cb24f1cacb0056f804a7692 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:30.352125  3995 raft_consensus.cc:1275] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:30.351135  3145 catalog_manager.cc:5697] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.347421  4159 raft_consensus.cc:2955] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.359671  4129 consensus_queue.cc:1048] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:23:30.374958  3379 tablet_service.cc:1558] Processing DeleteTablet for tablet 3af545b01cb24f1cacb0056f804a7692 with delete_type TABLET_DATA_TOMBSTONED (TS c93989d3dd634ef585d77b03354121a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54706
I20260709 12:23:30.376667  4141 raft_consensus.cc:2955] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.378217  3995 raft_consensus.cc:2955] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.379132  3996 raft_consensus.cc:2955] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.385111  4125 raft_consensus.cc:2955] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.393298  3391 raft_consensus.cc:1275] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:30.396196  3391 raft_consensus.cc:1275] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:30.403633  4129 consensus_queue.cc:1048] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:30.407740  4141 consensus_queue.cc:1048] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:30.407972  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 73563dc7466d4636a64ebd4d363935b1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:30.423698  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7ff1130a2f274bccbc5b15389deacdc1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:30.439657  4196 tablet_replica.cc:333] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:23:30.444095  3145 catalog_manager.cc:5697] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.443681  3996 raft_consensus.cc:2955] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.446238  3144 catalog_manager.cc:5697] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:30.448448  3318 raft_consensus.cc:2955] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.464351  3146 catalog_manager.cc:5697] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:30.460325  3392 raft_consensus.cc:2955] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.458858  3397 raft_consensus.cc:2955] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.468192  3249 raft_consensus.cc:2955] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.478099  3311 tablet_service.cc:1558] Processing DeleteTablet for tablet 7ff1130a2f274bccbc5b15389deacdc1 with delete_type TABLET_DATA_TOMBSTONED (TS 8525f77f43714a58bf158d872c1db3e4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40236
I20260709 12:23:30.499557  4196 raft_consensus.cc:2243] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:30.500569  4196 raft_consensus.cc:2272] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:30.507858  4197 tablet_replica.cc:333] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:23:30.519630  3311 tablet_service.cc:1558] Processing DeleteTablet for tablet 73563dc7466d4636a64ebd4d363935b1 with delete_type TABLET_DATA_TOMBSTONED (TS 8525f77f43714a58bf158d872c1db3e4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40236
I20260709 12:23:30.526096  4196 ts_tablet_manager.cc:1916] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:30.536418  3397 consensus_queue.cc:237] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:30.545048  4197 raft_consensus.cc:2243] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:30.546087  4197 raft_consensus.cc:2272] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:30.550454  3316 raft_consensus.cc:1275] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:30.549286  3996 raft_consensus.cc:1275] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:30.553225  4194 consensus_queue.cc:1048] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:30.554569  4197 ts_tablet_manager.cc:1916] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:30.555959  4196 ts_tablet_manager.cc:1929] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:30.556583  4196 log.cc:1199] T 3af545b01cb24f1cacb0056f804a7692 P c93989d3dd634ef585d77b03354121a4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/wals/3af545b01cb24f1cacb0056f804a7692
I20260709 12:23:30.558800  3133 catalog_manager.cc:5028] TS c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): tablet 3af545b01cb24f1cacb0056f804a7692 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:30.561731  4194 consensus_queue.cc:1048] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:30.566794  4125 raft_consensus.cc:2955] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.570619  3996 raft_consensus.cc:2955] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.578452  3316 raft_consensus.cc:2955] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:30.582036  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 34c56ebe809a4c09b75ee18faf980d8c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:30.608925  3147 catalog_manager.cc:5697] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New cstate: current_term: 2 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:30.615729  4197 ts_tablet_manager.cc:1929] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:30.616647  4197 log.cc:1199] T 7ff1130a2f274bccbc5b15389deacdc1 P 8525f77f43714a58bf158d872c1db3e4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/wals/7ff1130a2f274bccbc5b15389deacdc1
I20260709 12:23:30.618884  3134 catalog_manager.cc:5028] TS 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): tablet 7ff1130a2f274bccbc5b15389deacdc1 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:30.620203  4197 tablet_replica.cc:333] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:23:30.621524  4197 raft_consensus.cc:2243] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:30.622398  4197 raft_consensus.cc:2272] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:30.628960  4197 ts_tablet_manager.cc:1916] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:30.631306  3239 tablet_service.cc:1558] Processing DeleteTablet for tablet 34c56ebe809a4c09b75ee18faf980d8c with delete_type TABLET_DATA_TOMBSTONED (TS 7cdaa0234c64450eb4ce4500b51b8a81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53720
I20260709 12:23:30.633455  4198 tablet_replica.cc:333] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:23:30.634910  4198 raft_consensus.cc:2243] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:30.635761  4198 raft_consensus.cc:2272] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:30.641052  4198 ts_tablet_manager.cc:1916] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:30.647428  4197 ts_tablet_manager.cc:1929] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:30.647940  4197 log.cc:1199] T 73563dc7466d4636a64ebd4d363935b1 P 8525f77f43714a58bf158d872c1db3e4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/wals/73563dc7466d4636a64ebd4d363935b1
I20260709 12:23:30.649914  3134 catalog_manager.cc:5028] TS 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): tablet 73563dc7466d4636a64ebd4d363935b1 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:30.656373  4198 ts_tablet_manager.cc:1929] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 12:23:30.656893  4198 log.cc:1199] T 34c56ebe809a4c09b75ee18faf980d8c P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/wals/34c56ebe809a4c09b75ee18faf980d8c
I20260709 12:23:30.658882  3132 catalog_manager.cc:5028] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): tablet 34c56ebe809a4c09b75ee18faf980d8c (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:31.709728  3249 consensus_queue.cc:237] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
W20260709 12:23:31.716792  3213 consensus_peers.cc:597] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 572d97394c304e81affa5ee75b52de28. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:31.717308  3316 raft_consensus.cc:1275] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:31.718042  3397 raft_consensus.cc:1275] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:31.719034  4129 consensus_queue.cc:1048] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:31.720288  4201 consensus_queue.cc:1048] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:31.731122  4193 raft_consensus.cc:2955] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:31.733595  3397 raft_consensus.cc:2955] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:31.736727  3316 raft_consensus.cc:2955] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:31.762710  3147 catalog_manager.cc:5697] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:31.799679  3318 consensus_queue.cc:237] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:31.811398  3397 raft_consensus.cc:1275] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:31.814329  4177 consensus_queue.cc:1048] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:31.817317  3249 raft_consensus.cc:1275] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:31.820017  4177 consensus_queue.cc:1048] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:31.837181  4124 raft_consensus.cc:2955] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:31.841018  3391 raft_consensus.cc:2955] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
W20260709 12:23:31.842224  3288 consensus_peers.cc:597] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c04a8b7530d5486a880a89a4ba0faac8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:31.842698  3249 raft_consensus.cc:2955] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:31.862661  3249 consensus_queue.cc:237] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:31.865685  3144 catalog_manager.cc:5697] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:31.874022  3213 consensus_peers.cc:597] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9f37527cd3a34753b502172f81b808e0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:31.876603  3322 raft_consensus.cc:1275] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:31.880607  4201 consensus_queue.cc:1048] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:31.883863  3391 raft_consensus.cc:1275] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:31.888799  4201 consensus_queue.cc:1048] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:31.907420  4129 raft_consensus.cc:2955] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:31.909329  3391 raft_consensus.cc:2955] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:31.914569  3316 raft_consensus.cc:2955] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:31.936096  3145 catalog_manager.cc:5697] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:31.940707  3391 consensus_queue.cc:237] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:31.949536  3316 raft_consensus.cc:1275] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:31.951146  3362 consensus_peers.cc:597] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1d167141acd1450192e10b3c44347860. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:31.952013  3249 raft_consensus.cc:1275] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:31.961730  4125 consensus_queue.cc:1048] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:31.962656  4194 consensus_queue.cc:1048] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:31.977841  4194 raft_consensus.cc:2955] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:31.982630  3322 raft_consensus.cc:2955] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:31.985906  3244 raft_consensus.cc:2955] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:31.998468  3145 catalog_manager.cc:5697] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:32.196054  4222 ts_tablet_manager.cc:933] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:32.198144  4222 tablet_copy_client.cc:323] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:23:32.199827  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet 572d97394c304e81affa5ee75b52de28 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:23:32.200413  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet 572d97394c304e81affa5ee75b52de28 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-572d97394c304e81affa5ee75b52de28
I20260709 12:23:32.208045  3259 tablet_copy_source_session.cc:215] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:32.219379  4222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 572d97394c304e81affa5ee75b52de28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:32.246723  4222 tablet_copy_client.cc:806] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:32.247484  4222 tablet_copy_client.cc:670] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:32.251801  4222 tablet_copy_client.cc:538] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:32.261124  4222 tablet_bootstrap.cc:492] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:32.283425  4222 tablet_bootstrap.cc:492] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:32.284298  4222 tablet_bootstrap.cc:492] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:32.284921  4222 ts_tablet_manager.cc:1403] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.004s
I20260709 12:23:32.287330  4222 raft_consensus.cc:359] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:32.288165  4222 raft_consensus.cc:740] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:32.288823  4222 consensus_queue.cc:260] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:32.296783  4222 ts_tablet_manager.cc:1434] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.012s	user 0.012s	sys 0.000s
I20260709 12:23:32.298900  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-572d97394c304e81affa5ee75b52de28 received from {username='slave'} at 127.0.0.1:57122
I20260709 12:23:32.299469  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-572d97394c304e81affa5ee75b52de28 on tablet 572d97394c304e81affa5ee75b52de28 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:32.311465  4222 ts_tablet_manager.cc:933] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:32.313184  4222 tablet_copy_client.cc:323] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.194:33649
I20260709 12:23:32.351919  3334 tablet_copy_service.cc:140] P 8525f77f43714a58bf158d872c1db3e4: Received BeginTabletCopySession request for tablet c04a8b7530d5486a880a89a4ba0faac8 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:33282)
I20260709 12:23:32.352507  3334 tablet_copy_service.cc:161] P 8525f77f43714a58bf158d872c1db3e4: Beginning new tablet copy session on tablet c04a8b7530d5486a880a89a4ba0faac8 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:33282: session id = e38cc90af94c47f8a86be67ad610cbd4-c04a8b7530d5486a880a89a4ba0faac8
I20260709 12:23:32.354439  4229 ts_tablet_manager.cc:933] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:32.358475  4229 tablet_copy_client.cc:323] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:23:32.360913  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet 1d167141acd1450192e10b3c44347860 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:23:32.361481  3334 tablet_copy_source_session.cc:215] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:32.361548  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet 1d167141acd1450192e10b3c44347860 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-1d167141acd1450192e10b3c44347860
I20260709 12:23:32.365367  4222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c04a8b7530d5486a880a89a4ba0faac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:32.371320  3408 tablet_copy_source_session.cc:215] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:32.379568  4229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d167141acd1450192e10b3c44347860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:32.387382  4222 tablet_copy_client.cc:806] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:32.388126  4222 tablet_copy_client.cc:670] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:32.393353  4222 tablet_copy_client.cc:538] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:32.404366  4229 tablet_copy_client.cc:806] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:32.405154  4229 tablet_copy_client.cc:670] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:32.406392  4222 tablet_bootstrap.cc:492] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:32.411056  4229 tablet_copy_client.cc:538] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:32.439450  4229 tablet_bootstrap.cc:492] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:32.455452  4222 tablet_bootstrap.cc:492] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:32.456566  4222 tablet_bootstrap.cc:492] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:32.457314  4222 ts_tablet_manager.cc:1403] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.051s	user 0.020s	sys 0.018s
I20260709 12:23:32.463284  4222 raft_consensus.cc:359] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:32.475651  4222 raft_consensus.cc:740] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:32.478612  4222 consensus_queue.cc:260] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:32.480903  4232 ts_tablet_manager.cc:933] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:32.494596  4222 ts_tablet_manager.cc:1434] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.001s
I20260709 12:23:32.496619  3334 tablet_copy_service.cc:342] P 8525f77f43714a58bf158d872c1db3e4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-c04a8b7530d5486a880a89a4ba0faac8 received from {username='slave'} at 127.0.0.1:33282
I20260709 12:23:32.497102  3334 tablet_copy_service.cc:434] P 8525f77f43714a58bf158d872c1db3e4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-c04a8b7530d5486a880a89a4ba0faac8 on tablet c04a8b7530d5486a880a89a4ba0faac8 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:32.505980  4232 tablet_copy_client.cc:323] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:23:32.508710  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet 9f37527cd3a34753b502172f81b808e0 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:23:32.509335  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet 9f37527cd3a34753b502172f81b808e0 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-9f37527cd3a34753b502172f81b808e0
I20260709 12:23:32.519742  3259 tablet_copy_source_session.cc:215] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:32.521667  4229 tablet_bootstrap.cc:492] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:32.523352  4229 tablet_bootstrap.cc:492] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:32.524112  4229 ts_tablet_manager.cc:1403] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.085s	user 0.014s	sys 0.021s
I20260709 12:23:32.528118  4232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f37527cd3a34753b502172f81b808e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:32.530354  4229 raft_consensus.cc:359] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:32.531270  4229 raft_consensus.cc:740] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:32.531950  4229 consensus_queue.cc:260] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:32.535007  4229 ts_tablet_manager.cc:1434] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:23:32.537091  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-1d167141acd1450192e10b3c44347860 received from {username='slave'} at 127.0.0.1:34216
I20260709 12:23:32.537607  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-1d167141acd1450192e10b3c44347860 on tablet 1d167141acd1450192e10b3c44347860 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:32.548908  4232 tablet_copy_client.cc:806] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:32.549494  4232 tablet_copy_client.cc:670] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:32.553815  4232 tablet_copy_client.cc:538] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:32.564967  4232 tablet_bootstrap.cc:492] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:32.585529  4232 tablet_bootstrap.cc:492] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:32.586619  4232 tablet_bootstrap.cc:492] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:32.587338  4232 ts_tablet_manager.cc:1403] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.023s	user 0.020s	sys 0.000s
I20260709 12:23:32.590396  4232 raft_consensus.cc:359] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:32.591250  4232 raft_consensus.cc:740] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:32.591877  4232 consensus_queue.cc:260] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:32.594364  4232 ts_tablet_manager.cc:1434] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:23:32.598410  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-9f37527cd3a34753b502172f81b808e0 received from {username='slave'} at 127.0.0.1:57122
I20260709 12:23:32.598940  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-9f37527cd3a34753b502172f81b808e0 on tablet 9f37527cd3a34753b502172f81b808e0 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:32.763469  3996 raft_consensus.cc:1217] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:32.901901  3996 raft_consensus.cc:1217] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:32.935611  3996 raft_consensus.cc:1217] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:32.963366  3996 raft_consensus.cc:1217] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:33.165803  4193 raft_consensus.cc:1064] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:33.168712  4193 consensus_queue.cc:237] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:33.186501  3391 raft_consensus.cc:1275] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:33.187700  3996 raft_consensus.cc:1275] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:33.190693  4201 consensus_queue.cc:1048] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:33.192309  3322 raft_consensus.cc:1275] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:33.201040  4129 consensus_queue.cc:1048] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:33.210301  4129 consensus_queue.cc:1048] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:33.223407  4129 raft_consensus.cc:2955] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.227982  3391 raft_consensus.cc:2955] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.239744  3322 raft_consensus.cc:2955] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.249272  3996 raft_consensus.cc:2955] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.341161  3147 catalog_manager.cc:5697] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:33.366214  4238 raft_consensus.cc:1064] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:33.368925  4238 consensus_queue.cc:237] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:33.377002  4237 raft_consensus.cc:1064] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:33.384799  4243 raft_consensus.cc:1064] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:33.383224  4237 consensus_queue.cc:237] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:33.392410  4243 consensus_queue.cc:237] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:33.412806  3996 raft_consensus.cc:1275] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:33.413254  3243 raft_consensus.cc:1275] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:33.414446  3322 raft_consensus.cc:1275] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:33.414492  3397 raft_consensus.cc:1275] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:33.414908  4237 consensus_queue.cc:1048] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:23:33.417902  4215 consensus_queue.cc:1048] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:33.422420  4184 consensus_queue.cc:1048] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:33.421988  4215 consensus_queue.cc:1048] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:23:33.434727  3243 raft_consensus.cc:1275] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:33.421988  3996 raft_consensus.cc:1275] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:33.436882  4238 consensus_queue.cc:1048] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:33.439236  4184 consensus_queue.cc:1048] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:33.483322  3995 raft_consensus.cc:1275] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:33.484865  4245 consensus_queue.cc:1048] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:33.490274  3397 raft_consensus.cc:1275] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:33.496065  4243 consensus_queue.cc:1048] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:33.511400  4237 raft_consensus.cc:2955] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.519642  3320 raft_consensus.cc:1275] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:33.523484  4125 raft_consensus.cc:2955] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.537324  3241 raft_consensus.cc:2955] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.540508  4245 raft_consensus.cc:2955] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.542557  3392 raft_consensus.cc:2955] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.547062  4243 consensus_queue.cc:1048] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:33.549506  3320 raft_consensus.cc:2955] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.565143  3316 raft_consensus.cc:2955] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.563242  3995 raft_consensus.cc:2955] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:33.651225  4258 log.cc:927] Time spent T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4: Append to log took a long time: real 0.085s	user 0.008s	sys 0.000s
W20260709 12:23:33.652412  4260 log.cc:927] Time spent T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4: Append to log took a long time: real 0.087s	user 0.000s	sys 0.008s
I20260709 12:23:33.655920  3996 raft_consensus.cc:2955] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.801074  3143 catalog_manager.cc:5697] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:33.814441  3241 raft_consensus.cc:2955] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.822628  3994 raft_consensus.cc:2955] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.827421  3144 catalog_manager.cc:5697] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.804694  3143 catalog_manager.cc:5697] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.839871  3392 raft_consensus.cc:2955] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.864861  3318 consensus_queue.cc:237] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:33.872305  3994 raft_consensus.cc:1275] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.872622  3245 raft_consensus.cc:1275] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.876062  4237 consensus_queue.cc:1048] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:33.879626  4215 consensus_queue.cc:1048] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:33.897755  4236 raft_consensus.cc:2955] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.900002  3995 raft_consensus.cc:2955] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.919165  3243 raft_consensus.cc:2955] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.942806  3392 consensus_queue.cc:237] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:33.948463  3134 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c04a8b7530d5486a880a89a4ba0faac8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:33.947372  3245 consensus_queue.cc:237] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:33.952374  3994 raft_consensus.cc:1275] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.962647  3147 catalog_manager.cc:5697] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:33.966825  3316 raft_consensus.cc:1275] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.962762  4271 consensus_queue.cc:1048] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:23:33.944168  3241 consensus_queue.cc:237] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:33.974463  4266 consensus_queue.cc:1048] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:33.978483  3995 raft_consensus.cc:1275] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.979589  3392 raft_consensus.cc:1275] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.979660  3379 tablet_service.cc:1558] Processing DeleteTablet for tablet c04a8b7530d5486a880a89a4ba0faac8 with delete_type TABLET_DATA_TOMBSTONED (TS c93989d3dd634ef585d77b03354121a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54706
I20260709 12:23:33.983379  4245 consensus_queue.cc:1048] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:33.985323  4243 consensus_queue.cc:1048] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:33.989187  4272 raft_consensus.cc:2955] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:34.004767  4243 raft_consensus.cc:2955] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:34.006114  3995 raft_consensus.cc:2955] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:34.008739  3997 raft_consensus.cc:2955] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:34.015038  3392 raft_consensus.cc:2955] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:34.028765  3994 raft_consensus.cc:1275] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:34.028676  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9f37527cd3a34753b502172f81b808e0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:34.031715  3323 raft_consensus.cc:1275] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:34.030397  3316 raft_consensus.cc:2955] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:34.033253  4245 consensus_queue.cc:1048] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:34.025446  4278 tablet_replica.cc:333] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:23:34.041051  4278 raft_consensus.cc:2243] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:34.041818  4278 raft_consensus.cc:2272] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:34.045025  4129 raft_consensus.cc:2955] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:34.056511  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1d167141acd1450192e10b3c44347860 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:34.057392  4193 consensus_queue.cc:1048] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:34.070856  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 572d97394c304e81affa5ee75b52de28 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:34.058998  3323 raft_consensus.cc:2955] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:34.112212  4278 ts_tablet_manager.cc:1916] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:34.132654  3143 catalog_manager.cc:5697] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:34.140638  3146 catalog_manager.cc:5697] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:34.132977  3995 raft_consensus.cc:2955] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:34.158093  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 87864us
I20260709 12:23:34.166826  4278 ts_tablet_manager.cc:1929] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:34.167393  4278 log.cc:1199] T c04a8b7530d5486a880a89a4ba0faac8 P c93989d3dd634ef585d77b03354121a4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/wals/c04a8b7530d5486a880a89a4ba0faac8
I20260709 12:23:34.169880  3133 catalog_manager.cc:5028] TS c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): tablet c04a8b7530d5486a880a89a4ba0faac8 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:34.170094  3239 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d167141acd1450192e10b3c44347860 with delete_type TABLET_DATA_TOMBSTONED (TS 7cdaa0234c64450eb4ce4500b51b8a81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53720
I20260709 12:23:34.172640  4281 tablet_replica.cc:333] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:23:34.173834  4281 raft_consensus.cc:2243] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:34.174558  4281 raft_consensus.cc:2272] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:34.175362  3311 tablet_service.cc:1558] Processing DeleteTablet for tablet 9f37527cd3a34753b502172f81b808e0 with delete_type TABLET_DATA_TOMBSTONED (TS 8525f77f43714a58bf158d872c1db3e4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40236
I20260709 12:23:34.183239  4281 ts_tablet_manager.cc:1916] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:34.190770  4282 tablet_replica.cc:333] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:23:34.192508  4282 raft_consensus.cc:2243] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:34.193509  4282 raft_consensus.cc:2272] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:34.197762  4282 ts_tablet_manager.cc:1916] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:34.204028  3143 catalog_manager.cc:5697] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:34.211385  4281 ts_tablet_manager.cc:1929] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:34.211915  4281 log.cc:1199] T 1d167141acd1450192e10b3c44347860 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/wals/1d167141acd1450192e10b3c44347860
I20260709 12:23:34.218472  4282 ts_tablet_manager.cc:1929] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:34.219008  4282 log.cc:1199] T 9f37527cd3a34753b502172f81b808e0 P 8525f77f43714a58bf158d872c1db3e4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/wals/9f37527cd3a34753b502172f81b808e0
I20260709 12:23:34.221026  3134 catalog_manager.cc:5028] TS 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): tablet 9f37527cd3a34753b502172f81b808e0 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:34.227474  3132 catalog_manager.cc:5028] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): tablet 1d167141acd1450192e10b3c44347860 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:34.237167  3379 tablet_service.cc:1558] Processing DeleteTablet for tablet 572d97394c304e81affa5ee75b52de28 with delete_type TABLET_DATA_TOMBSTONED (TS c93989d3dd634ef585d77b03354121a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54706
I20260709 12:23:34.238545  4278 tablet_replica.cc:333] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:23:34.239795  4278 raft_consensus.cc:2243] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:34.240513  4278 raft_consensus.cc:2272] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:34.244236  4278 ts_tablet_manager.cc:1916] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:34.260164  4278 ts_tablet_manager.cc:1929] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:34.260592  4278 log.cc:1199] T 572d97394c304e81affa5ee75b52de28 P c93989d3dd634ef585d77b03354121a4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/wals/572d97394c304e81affa5ee75b52de28
I20260709 12:23:34.262187  3133 catalog_manager.cc:5028] TS c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): tablet 572d97394c304e81affa5ee75b52de28 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:35.315893  3241 consensus_queue.cc:237] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:35.328097  3323 raft_consensus.cc:1275] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:35.328644  3391 raft_consensus.cc:1275] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:35.332172  4280 consensus_queue.cc:1048] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:35.333204  4193 consensus_queue.cc:1048] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:35.333804  3213 consensus_peers.cc:597] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: af3368aec5b1412797545abe458a95fe. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:35.346328  4280 raft_consensus.cc:2955] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:35.349007  3323 raft_consensus.cc:2955] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:35.351104  3392 raft_consensus.cc:2955] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:35.366536  3391 consensus_queue.cc:237] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:35.371773  3143 catalog_manager.cc:5697] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:35.391613  3323 raft_consensus.cc:1275] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:35.393342  4272 consensus_queue.cc:1048] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:35.395920  3245 raft_consensus.cc:1275] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:35.407680  4271 raft_consensus.cc:2955] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:35.409281  3320 raft_consensus.cc:2955] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:35.435271  4272 consensus_queue.cc:1048] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:35.439183  3362 consensus_peers.cc:597] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3015bb6b71f444dcb17062ac96b48cf5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:35.461460  3245 raft_consensus.cc:2955] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:35.500703  3241 consensus_queue.cc:237] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:35.515548  3144 catalog_manager.cc:5697] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:35.525424  3213 consensus_peers.cc:597] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dbb481337ada4c83945e213fb117ee7f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:35.528950  3397 raft_consensus.cc:1275] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:35.528999  3320 raft_consensus.cc:1275] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:35.530881  4298 consensus_queue.cc:1048] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:35.532370  4284 consensus_queue.cc:1048] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:35.549525  4243 raft_consensus.cc:2955] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:35.551355  3397 raft_consensus.cc:2955] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:35.551831  3320 raft_consensus.cc:2955] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:35.573477  3397 consensus_queue.cc:237] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:35.573414  3147 catalog_manager.cc:5697] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:35.587527  3320 raft_consensus.cc:1275] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:35.589645  3362 consensus_peers.cc:597] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a7494971ced1402da04932982df157fd. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:35.591207  3245 raft_consensus.cc:1275] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:35.591526  4293 consensus_queue.cc:1048] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:35.592986  4271 consensus_queue.cc:1048] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:35.611210  4293 raft_consensus.cc:2955] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:35.616268  3320 raft_consensus.cc:2955] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:35.626228  3241 raft_consensus.cc:2955] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:35.643520  3144 catalog_manager.cc:5697] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:35.751595  4306 ts_tablet_manager.cc:933] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:35.759089  4306 tablet_copy_client.cc:323] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:23:35.763504  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet af3368aec5b1412797545abe458a95fe from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:23:35.764143  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet af3368aec5b1412797545abe458a95fe from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-af3368aec5b1412797545abe458a95fe
I20260709 12:23:35.776072  3259 tablet_copy_source_session.cc:215] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:35.779605  4306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af3368aec5b1412797545abe458a95fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:35.797137  4306 tablet_copy_client.cc:806] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:35.797770  4306 tablet_copy_client.cc:670] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:35.802644  4306 tablet_copy_client.cc:538] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:35.815455  4306 tablet_bootstrap.cc:492] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:35.836459  4308 ts_tablet_manager.cc:933] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:35.850418  4308 tablet_copy_client.cc:323] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:23:35.852691  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet 3015bb6b71f444dcb17062ac96b48cf5 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:23:35.853329  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet 3015bb6b71f444dcb17062ac96b48cf5 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-3015bb6b71f444dcb17062ac96b48cf5
I20260709 12:23:35.858448  4306 tablet_bootstrap.cc:492] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:35.859651  4306 tablet_bootstrap.cc:492] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:35.860502  4306 ts_tablet_manager.cc:1403] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.027s
I20260709 12:23:35.862629  3408 tablet_copy_source_session.cc:215] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:35.864136  4306 raft_consensus.cc:359] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:35.865042  4306 raft_consensus.cc:740] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:35.865901  4306 consensus_queue.cc:260] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:35.866650  4308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3015bb6b71f444dcb17062ac96b48cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:35.872196  4306 ts_tablet_manager.cc:1434] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 12:23:35.877238  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-af3368aec5b1412797545abe458a95fe received from {username='slave'} at 127.0.0.1:57122
I20260709 12:23:35.877770  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-af3368aec5b1412797545abe458a95fe on tablet af3368aec5b1412797545abe458a95fe with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:35.887858  4308 tablet_copy_client.cc:806] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:35.888573  4308 tablet_copy_client.cc:670] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:35.892302  4308 tablet_copy_client.cc:538] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:35.901109  4308 tablet_bootstrap.cc:492] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:35.920230  4306 ts_tablet_manager.cc:933] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:35.922899  4306 tablet_copy_client.cc:323] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:23:35.924469  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet dbb481337ada4c83945e213fb117ee7f from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:23:35.924961  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet dbb481337ada4c83945e213fb117ee7f from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-dbb481337ada4c83945e213fb117ee7f
I20260709 12:23:35.927039  4308 tablet_bootstrap.cc:492] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:35.928274  4308 tablet_bootstrap.cc:492] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:35.929036  4308 ts_tablet_manager.cc:1403] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.008s
I20260709 12:23:35.933507  3259 tablet_copy_source_session.cc:215] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:35.933987  4308 raft_consensus.cc:359] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:35.935026  4308 raft_consensus.cc:740] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:35.935765  4308 consensus_queue.cc:260] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:35.938890  4308 ts_tablet_manager.cc:1434] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 12:23:35.941038  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-3015bb6b71f444dcb17062ac96b48cf5 received from {username='slave'} at 127.0.0.1:34216
I20260709 12:23:35.941505  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-3015bb6b71f444dcb17062ac96b48cf5 on tablet 3015bb6b71f444dcb17062ac96b48cf5 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:35.968103  4306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbb481337ada4c83945e213fb117ee7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:35.988692  4306 tablet_copy_client.cc:806] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:35.989488  4306 tablet_copy_client.cc:670] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:35.993273  4306 tablet_copy_client.cc:538] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:36.007926  4306 tablet_bootstrap.cc:492] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:36.027195  4306 tablet_bootstrap.cc:492] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:36.028715  4306 tablet_bootstrap.cc:492] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:36.042184  4306 ts_tablet_manager.cc:1403] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.006s
I20260709 12:23:36.045781  4306 raft_consensus.cc:359] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:36.046742  4306 raft_consensus.cc:740] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:36.047434  4306 consensus_queue.cc:260] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:36.049988  4306 ts_tablet_manager.cc:1434] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 12:23:36.052511  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-dbb481337ada4c83945e213fb117ee7f received from {username='slave'} at 127.0.0.1:57122
I20260709 12:23:36.053028  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-dbb481337ada4c83945e213fb117ee7f on tablet dbb481337ada4c83945e213fb117ee7f with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:36.204481  4306 ts_tablet_manager.cc:933] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:36.206389  4306 tablet_copy_client.cc:323] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:23:36.208093  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet a7494971ced1402da04932982df157fd from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:23:36.208772  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet a7494971ced1402da04932982df157fd from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-a7494971ced1402da04932982df157fd
I20260709 12:23:36.220162  3408 tablet_copy_source_session.cc:215] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:36.225493  4306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7494971ced1402da04932982df157fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:36.248421  3995 raft_consensus.cc:1217] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:36.252240  4306 tablet_copy_client.cc:806] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:36.252905  4306 tablet_copy_client.cc:670] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:36.276057  4306 tablet_copy_client.cc:538] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:36.287168  4306 tablet_bootstrap.cc:492] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:36.316223  4306 tablet_bootstrap.cc:492] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:36.317517  4306 tablet_bootstrap.cc:492] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:36.318418  4306 ts_tablet_manager.cc:1403] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.031s	user 0.025s	sys 0.004s
I20260709 12:23:36.333045  4306 raft_consensus.cc:359] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:36.334017  4306 raft_consensus.cc:740] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:36.334676  4306 consensus_queue.cc:260] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:36.337605  4306 ts_tablet_manager.cc:1434] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.002s
I20260709 12:23:36.339453  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-a7494971ced1402da04932982df157fd received from {username='slave'} at 127.0.0.1:34216
I20260709 12:23:36.339890  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-a7494971ced1402da04932982df157fd on tablet a7494971ced1402da04932982df157fd with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:36.350205  3995 raft_consensus.cc:1217] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:36.392364  3995 raft_consensus.cc:1217] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:36.726395  4272 raft_consensus.cc:1064] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:36.731462  4272 consensus_queue.cc:237] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:36.741268  3995 raft_consensus.cc:1275] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.741314  3316 raft_consensus.cc:1275] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.742142  3241 raft_consensus.cc:1275] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.743723  4272 consensus_queue.cc:1048] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:36.744745  4296 consensus_queue.cc:1048] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:36.746088  4271 consensus_queue.cc:1048] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:36.778946  4280 raft_consensus.cc:1064] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:36.781881  4280 consensus_queue.cc:237] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:36.797783  4272 raft_consensus.cc:2955] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:36.799608  3316 raft_consensus.cc:2955] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:36.812597  3243 raft_consensus.cc:2955] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:36.819949  3994 raft_consensus.cc:1275] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.822124  4298 consensus_queue.cc:1048] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:36.831794  3147 catalog_manager.cc:5697] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:36.835322  3994 raft_consensus.cc:1217] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:36.853258  3397 raft_consensus.cc:1275] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.873288  4325 consensus_queue.cc:1048] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:36.902616  3995 raft_consensus.cc:2955] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:36.910745  3316 raft_consensus.cc:1275] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.913523  4243 consensus_queue.cc:1048] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:23:36.923053  3213 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 63958.9us
I20260709 12:23:36.936149  4298 raft_consensus.cc:2955] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:36.944290  3397 raft_consensus.cc:2955] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:36.972309  3397 consensus_queue.cc:237] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:36.991137  3316 raft_consensus.cc:2955] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.008653  3316 raft_consensus.cc:1275] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:37.011895  4329 consensus_queue.cc:1048] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:37.012850  3995 raft_consensus.cc:1275] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:37.014901  4329 consensus_queue.cc:1048] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:37.017776  3144 catalog_manager.cc:5697] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.026285  4329 raft_consensus.cc:2955] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.032438  3994 raft_consensus.cc:2955] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.039749  3997 raft_consensus.cc:2955] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.045830  3316 raft_consensus.cc:2955] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.048105  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3015bb6b71f444dcb17062ac96b48cf5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:37.058350  4243 raft_consensus.cc:1064] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:37.061183  4243 consensus_queue.cc:237] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:37.074074  3397 raft_consensus.cc:1275] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:37.075870  3316 raft_consensus.cc:1275] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:37.078969  3995 raft_consensus.cc:1275] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:37.083211  3241 consensus_queue.cc:237] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:37.077605  4328 consensus_queue.cc:1048] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:23:37.092501  3147 catalog_manager.cc:5697] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.094308  4298 consensus_queue.cc:1048] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:37.096740  3320 raft_consensus.cc:1275] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:37.098996  4336 consensus_queue.cc:1048] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:37.109683  4298 consensus_queue.cc:1048] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:37.115042  3995 raft_consensus.cc:1275] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:37.118590  4328 consensus_queue.cc:1048] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:37.126753  3239 tablet_service.cc:1558] Processing DeleteTablet for tablet 3015bb6b71f444dcb17062ac96b48cf5 with delete_type TABLET_DATA_TOMBSTONED (TS 7cdaa0234c64450eb4ce4500b51b8a81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53720
I20260709 12:23:37.138336  4325 raft_consensus.cc:2955] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.156658  3997 raft_consensus.cc:2955] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.167469  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet af3368aec5b1412797545abe458a95fe with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:37.179538  3147 catalog_manager.cc:5697] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.194732  3379 tablet_service.cc:1558] Processing DeleteTablet for tablet af3368aec5b1412797545abe458a95fe with delete_type TABLET_DATA_TOMBSTONED (TS c93989d3dd634ef585d77b03354121a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54706
I20260709 12:23:37.199095  3320 raft_consensus.cc:2955] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.203912  4325 raft_consensus.cc:2955] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.210256  4340 tablet_replica.cc:333] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:23:37.211869  4340 raft_consensus.cc:2243] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:37.212167  4342 tablet_replica.cc:333] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:23:37.212816  4340 raft_consensus.cc:2272] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:37.221575  4342 raft_consensus.cc:2243] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:37.222677  4342 raft_consensus.cc:2272] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:37.215303  3994 raft_consensus.cc:2955] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.237761  4342 ts_tablet_manager.cc:1916] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:37.239753  3147 catalog_manager.cc:5697] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.258570  4340 ts_tablet_manager.cc:1916] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:37.276448  3241 consensus_queue.cc:237] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:37.276151  3320 raft_consensus.cc:2955] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.271258  4342 ts_tablet_manager.cc:1929] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:37.278833  4342 log.cc:1199] T af3368aec5b1412797545abe458a95fe P c93989d3dd634ef585d77b03354121a4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/wals/af3368aec5b1412797545abe458a95fe
I20260709 12:23:37.248679  3397 raft_consensus.cc:2955] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.280934  3133 catalog_manager.cc:5028] TS c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): tablet af3368aec5b1412797545abe458a95fe (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:37.307955  3997 raft_consensus.cc:1275] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:37.311178  4336 consensus_queue.cc:1048] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:37.316740  3392 raft_consensus.cc:1275] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:37.319029  4340 ts_tablet_manager.cc:1929] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:37.319572  4340 log.cc:1199] T 3015bb6b71f444dcb17062ac96b48cf5 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/wals/3015bb6b71f444dcb17062ac96b48cf5
I20260709 12:23:37.319584  4336 consensus_queue.cc:1048] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:37.322355  3132 catalog_manager.cc:5028] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): tablet 3015bb6b71f444dcb17062ac96b48cf5 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:37.324848  4298 raft_consensus.cc:2955] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.327889  3997 raft_consensus.cc:2955] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.337534  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dbb481337ada4c83945e213fb117ee7f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:37.334244  3397 raft_consensus.cc:2955] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.345983  3147 catalog_manager.cc:5697] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:37.360759  3311 tablet_service.cc:1558] Processing DeleteTablet for tablet dbb481337ada4c83945e213fb117ee7f with delete_type TABLET_DATA_TOMBSTONED (TS 8525f77f43714a58bf158d872c1db3e4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40236
I20260709 12:23:37.362753  4343 tablet_replica.cc:333] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:23:37.364013  4343 raft_consensus.cc:2243] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:37.366058  4343 raft_consensus.cc:2272] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:37.376803  4343 ts_tablet_manager.cc:1916] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:37.392902  4343 ts_tablet_manager.cc:1929] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:37.393275  4343 log.cc:1199] T dbb481337ada4c83945e213fb117ee7f P 8525f77f43714a58bf158d872c1db3e4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/wals/dbb481337ada4c83945e213fb117ee7f
I20260709 12:23:37.394964  3134 catalog_manager.cc:5028] TS 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): tablet dbb481337ada4c83945e213fb117ee7f (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:37.409869  4296 raft_consensus.cc:1064] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:37.412932  4296 consensus_queue.cc:237] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:37.428807  3997 raft_consensus.cc:1275] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:37.428893  3320 raft_consensus.cc:1275] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:37.430998  4324 consensus_queue.cc:1048] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:37.433463  3241 raft_consensus.cc:1275] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:37.433895  4329 consensus_queue.cc:1048] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:37.439640  4271 consensus_queue.cc:1048] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:37.484066  4272 raft_consensus.cc:2955] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.485854  3997 raft_consensus.cc:2955] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.486233  3320 raft_consensus.cc:2955] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.508936  3144 catalog_manager.cc:5697] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.512568  3244 raft_consensus.cc:2955] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.535941  3397 consensus_queue.cc:237] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:37.542748  3994 raft_consensus.cc:1275] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:37.544948  3244 raft_consensus.cc:1275] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:37.545317  4272 consensus_queue.cc:1048] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:37.550024  4271 consensus_queue.cc:1048] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:37.561308  4272 raft_consensus.cc:2955] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.564301  3994 raft_consensus.cc:2955] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.577963  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a7494971ced1402da04932982df157fd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:37.597432  3144 catalog_manager.cc:5697] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.607360  3244 raft_consensus.cc:2955] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:37.609700  3311 tablet_service.cc:1558] Processing DeleteTablet for tablet a7494971ced1402da04932982df157fd with delete_type TABLET_DATA_TOMBSTONED (TS 8525f77f43714a58bf158d872c1db3e4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40236
I20260709 12:23:37.610733  4343 tablet_replica.cc:333] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:23:37.611838  4343 raft_consensus.cc:2243] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:37.612886  4343 raft_consensus.cc:2272] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:37.618824  4343 ts_tablet_manager.cc:1916] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:37.643247  4343 ts_tablet_manager.cc:1929] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:37.644237  4343 log.cc:1199] T a7494971ced1402da04932982df157fd P 8525f77f43714a58bf158d872c1db3e4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/wals/a7494971ced1402da04932982df157fd
I20260709 12:23:37.646731  3134 catalog_manager.cc:5028] TS 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): tablet a7494971ced1402da04932982df157fd (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:38.712242  3244 consensus_queue.cc:237] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
W20260709 12:23:38.723646  3213 consensus_peers.cc:597] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a6d2799afd04cbe9bfaa74d467e66cb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:38.723742  3318 raft_consensus.cc:1275] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:38.724342  3391 raft_consensus.cc:1275] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:38.726768  4336 consensus_queue.cc:1048] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:23:38.733085  4336 consensus_queue.cc:1048] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:38.742430  4336 raft_consensus.cc:2955] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:38.747116  3323 raft_consensus.cc:2955] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:38.752010  3391 raft_consensus.cc:2955] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:38.771914  3323 consensus_queue.cc:237] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:38.774036  3144 catalog_manager.cc:5697] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:38.785478  3391 raft_consensus.cc:1275] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:38.787215  3288 consensus_peers.cc:597] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9253ca0fa41f43a7ac13ff17879c2ed3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:38.788139  4320 consensus_queue.cc:1048] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:38.800307  3244 raft_consensus.cc:1275] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:38.802279  4320 consensus_queue.cc:1048] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:38.801124  4362 raft_consensus.cc:2955] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:38.804808  3391 raft_consensus.cc:2955] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:38.823225  3318 consensus_queue.cc:237] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:38.812629  3244 raft_consensus.cc:2955] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:38.835573  3392 raft_consensus.cc:1275] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:38.840461  4320 consensus_queue.cc:1048] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:38.841131  3147 catalog_manager.cc:5697] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:38.857430  3244 raft_consensus.cc:1275] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:38.858544  4362 raft_consensus.cc:2955] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
W20260709 12:23:38.866194  3288 consensus_peers.cc:597] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 87fddcaae9194bef90d1047a518e45a6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:38.869853  4349 consensus_queue.cc:1048] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:38.869462  3392 raft_consensus.cc:2955] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:38.901144  3245 raft_consensus.cc:2955] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:38.902079  3143 catalog_manager.cc:5697] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:38.909274  3392 consensus_queue.cc:237] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
W20260709 12:23:38.927960  3362 consensus_peers.cc:597] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0634624f456740b48b77096645ab6b0b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:38.926509  3245 raft_consensus.cc:1275] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:38.929572  3317 raft_consensus.cc:1275] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:38.933566  4371 consensus_queue.cc:1048] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:38.935075  4351 consensus_queue.cc:1048] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:38.960093  4351 raft_consensus.cc:2955] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:38.963949  3323 raft_consensus.cc:2955] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:38.976929  3245 raft_consensus.cc:2955] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:38.995067  3144 catalog_manager.cc:5697] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:39.123147  4377 ts_tablet_manager.cc:933] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:39.128769  4377 tablet_copy_client.cc:323] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:23:39.130870  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet 7a6d2799afd04cbe9bfaa74d467e66cb from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:23:39.131618  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet 7a6d2799afd04cbe9bfaa74d467e66cb from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-7a6d2799afd04cbe9bfaa74d467e66cb
I20260709 12:23:39.144837  3259 tablet_copy_source_session.cc:215] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:39.150403  4377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a6d2799afd04cbe9bfaa74d467e66cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:39.172448  4377 tablet_copy_client.cc:806] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:39.173239  4377 tablet_copy_client.cc:670] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:39.180732  4377 tablet_copy_client.cc:538] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:39.194190  4377 tablet_bootstrap.cc:492] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:39.217417  4377 tablet_bootstrap.cc:492] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:39.218645  4377 tablet_bootstrap.cc:492] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:39.219470  4377 ts_tablet_manager.cc:1403] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.012s
I20260709 12:23:39.223155  4377 raft_consensus.cc:359] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:39.224129  4377 raft_consensus.cc:740] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:39.224933  4377 consensus_queue.cc:260] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:39.355523  4381 ts_tablet_manager.cc:933] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:39.374661  4381 tablet_copy_client.cc:323] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.194:33649
I20260709 12:23:39.378039  3334 tablet_copy_service.cc:140] P 8525f77f43714a58bf158d872c1db3e4: Received BeginTabletCopySession request for tablet 9253ca0fa41f43a7ac13ff17879c2ed3 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:33282)
I20260709 12:23:39.378702  3334 tablet_copy_service.cc:161] P 8525f77f43714a58bf158d872c1db3e4: Beginning new tablet copy session on tablet 9253ca0fa41f43a7ac13ff17879c2ed3 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:33282: session id = e38cc90af94c47f8a86be67ad610cbd4-9253ca0fa41f43a7ac13ff17879c2ed3
I20260709 12:23:39.424749  4377 ts_tablet_manager.cc:1434] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.205s	user 0.048s	sys 0.112s
I20260709 12:23:39.426963  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-7a6d2799afd04cbe9bfaa74d467e66cb received from {username='slave'} at 127.0.0.1:57122
I20260709 12:23:39.427546  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-7a6d2799afd04cbe9bfaa74d467e66cb on tablet 7a6d2799afd04cbe9bfaa74d467e66cb with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:39.443507  3334 tablet_copy_source_session.cc:215] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:39.444504  4377 ts_tablet_manager.cc:933] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:39.457835  4381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9253ca0fa41f43a7ac13ff17879c2ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:39.473800  4377 tablet_copy_client.cc:323] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.194:33649
I20260709 12:23:39.476150  3334 tablet_copy_service.cc:140] P 8525f77f43714a58bf158d872c1db3e4: Received BeginTabletCopySession request for tablet 87fddcaae9194bef90d1047a518e45a6 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:33282)
I20260709 12:23:39.476754  3334 tablet_copy_service.cc:161] P 8525f77f43714a58bf158d872c1db3e4: Beginning new tablet copy session on tablet 87fddcaae9194bef90d1047a518e45a6 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:33282: session id = e38cc90af94c47f8a86be67ad610cbd4-87fddcaae9194bef90d1047a518e45a6
I20260709 12:23:39.490782  4384 ts_tablet_manager.cc:933] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:39.494786  3334 tablet_copy_source_session.cc:215] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:39.495743  4384 tablet_copy_client.cc:323] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:23:39.498354  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet 0634624f456740b48b77096645ab6b0b from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:23:39.498974  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet 0634624f456740b48b77096645ab6b0b from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-0634624f456740b48b77096645ab6b0b
I20260709 12:23:39.501005  4381 tablet_copy_client.cc:806] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:39.503523  4377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87fddcaae9194bef90d1047a518e45a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:39.505790  4381 tablet_copy_client.cc:670] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:39.515216  3408 tablet_copy_source_session.cc:215] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:39.520673  4384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0634624f456740b48b77096645ab6b0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:39.526304  4381 tablet_copy_client.cc:538] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:39.541260  4377 tablet_copy_client.cc:806] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:39.542129  4377 tablet_copy_client.cc:670] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:39.549788  4377 tablet_copy_client.cc:538] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:39.560640  4381 tablet_bootstrap.cc:492] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:39.569147  4377 tablet_bootstrap.cc:492] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:39.572532  4384 tablet_copy_client.cc:806] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:39.573405  4384 tablet_copy_client.cc:670] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:39.597262  4384 tablet_copy_client.cc:538] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:39.604348  4377 tablet_bootstrap.cc:492] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:39.605870  4377 tablet_bootstrap.cc:492] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:39.607038  4377 ts_tablet_manager.cc:1403] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.038s	user 0.027s	sys 0.006s
I20260709 12:23:39.611213  4381 tablet_bootstrap.cc:492] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:39.612344  4377 raft_consensus.cc:359] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:39.613433  4377 raft_consensus.cc:740] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:39.614265  4377 consensus_queue.cc:260] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:39.615723  4381 tablet_bootstrap.cc:492] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:39.616690  4381 ts_tablet_manager.cc:1403] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.056s	user 0.025s	sys 0.010s
I20260709 12:23:39.622992  4381 raft_consensus.cc:359] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:39.624010  4381 raft_consensus.cc:740] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:39.624760  4381 consensus_queue.cc:260] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:39.627681  4381 ts_tablet_manager.cc:1434] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.007s
I20260709 12:23:39.630864  3334 tablet_copy_service.cc:342] P 8525f77f43714a58bf158d872c1db3e4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-9253ca0fa41f43a7ac13ff17879c2ed3 received from {username='slave'} at 127.0.0.1:33282
I20260709 12:23:39.631409  3334 tablet_copy_service.cc:434] P 8525f77f43714a58bf158d872c1db3e4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-9253ca0fa41f43a7ac13ff17879c2ed3 on tablet 9253ca0fa41f43a7ac13ff17879c2ed3 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:39.640378  4377 ts_tablet_manager.cc:1434] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.001s
I20260709 12:23:39.647806  3334 tablet_copy_service.cc:342] P 8525f77f43714a58bf158d872c1db3e4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-87fddcaae9194bef90d1047a518e45a6 received from {username='slave'} at 127.0.0.1:33282
I20260709 12:23:39.648365  3334 tablet_copy_service.cc:434] P 8525f77f43714a58bf158d872c1db3e4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-87fddcaae9194bef90d1047a518e45a6 on tablet 87fddcaae9194bef90d1047a518e45a6 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:39.649775  4384 tablet_bootstrap.cc:492] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:39.658011  3994 raft_consensus.cc:1217] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:39.681170  4384 tablet_bootstrap.cc:492] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:39.682360  4384 tablet_bootstrap.cc:492] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:39.683207  4384 ts_tablet_manager.cc:1403] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.034s	user 0.029s	sys 0.004s
I20260709 12:23:39.687973  4384 raft_consensus.cc:359] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:39.688977  4384 raft_consensus.cc:740] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:39.691344  4384 consensus_queue.cc:260] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:39.696002  4384 ts_tablet_manager.cc:1434] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 12:23:39.698670  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-0634624f456740b48b77096645ab6b0b received from {username='slave'} at 127.0.0.1:34216
I20260709 12:23:39.699218  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-0634624f456740b48b77096645ab6b0b on tablet 0634624f456740b48b77096645ab6b0b with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:39.705411  3994 raft_consensus.cc:1217] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:23:39.896008  3994 raft_consensus.cc:1217] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:40.031239  3994 raft_consensus.cc:1217] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:40.156085  4362 raft_consensus.cc:1064] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:40.158725  4362 consensus_queue.cc:237] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:40.173018  3994 raft_consensus.cc:1275] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:40.173396  3244 raft_consensus.cc:1275] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:40.175021  4320 consensus_queue.cc:1048] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:40.176173  3391 raft_consensus.cc:1275] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:40.177932  4320 consensus_queue.cc:1048] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:40.178996  4368 consensus_queue.cc:1048] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:40.200384  4390 raft_consensus.cc:1064] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:40.203102  4390 consensus_queue.cc:237] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:40.221475  4368 raft_consensus.cc:2955] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.226763  3994 raft_consensus.cc:1275] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:40.228657  4389 consensus_queue.cc:1048] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:40.232694  3317 raft_consensus.cc:1275] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:40.235538  3391 raft_consensus.cc:1275] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:40.240144  4389 consensus_queue.cc:1048] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:40.241410  4325 consensus_queue.cc:1048] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:40.259361  3997 raft_consensus.cc:2955] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.298213  3144 catalog_manager.cc:5697] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.286083  4391 raft_consensus.cc:2955] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.307096  3392 raft_consensus.cc:2955] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.316792  3997 raft_consensus.cc:2955] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.330600  3394 raft_consensus.cc:2955] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.360858  3245 raft_consensus.cc:2955] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.390166  3317 raft_consensus.cc:2955] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.408023  4393 raft_consensus.cc:1064] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:40.411310  4393 consensus_queue.cc:237] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:23:40.428294  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 56187.4us
I20260709 12:23:40.432737  3394 raft_consensus.cc:1275] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:40.435078  3997 raft_consensus.cc:1275] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:40.436636  3245 raft_consensus.cc:1275] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:40.440301  4405 consensus_queue.cc:1048] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:40.445108  4362 consensus_queue.cc:1048] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:23:40.456552  3143 catalog_manager.cc:5697] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.458091  4405 consensus_queue.cc:1048] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:40.496678  3317 consensus_queue.cc:237] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:40.499357  4362 raft_consensus.cc:2955] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.514058  3997 raft_consensus.cc:2955] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.522789  3394 raft_consensus.cc:1275] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:40.525293  4393 consensus_queue.cc:1048] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:40.512673  3245 raft_consensus.cc:2955] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.525681  3998 raft_consensus.cc:1275] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:40.528970  4393 consensus_queue.cc:1048] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:40.535167  3244 consensus_queue.cc:237] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:40.550551  3998 raft_consensus.cc:1275] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:40.550769  4405 raft_consensus.cc:2955] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.552892  4390 consensus_queue.cc:1048] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:40.556408  3998 raft_consensus.cc:2955] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.561381  4391 raft_consensus.cc:2955] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.563175  3994 raft_consensus.cc:2955] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.574710  3317 raft_consensus.cc:1275] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:40.576987  3394 raft_consensus.cc:2955] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.590323  4359 consensus_queue.cc:1048] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:40.593168  3392 raft_consensus.cc:2955] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.596482  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7a6d2799afd04cbe9bfaa74d467e66cb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:40.564832  3143 catalog_manager.cc:5697] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.625517  3134 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9253ca0fa41f43a7ac13ff17879c2ed3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:40.636039  4329 raft_consensus.cc:1064] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:40.629240  3147 catalog_manager.cc:5697] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.647292  4329 consensus_queue.cc:237] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:40.636883  3317 raft_consensus.cc:2955] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.664022  3319 consensus_queue.cc:237] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:40.667429  3239 tablet_service.cc:1558] Processing DeleteTablet for tablet 9253ca0fa41f43a7ac13ff17879c2ed3 with delete_type TABLET_DATA_TOMBSTONED (TS 7cdaa0234c64450eb4ce4500b51b8a81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53720
I20260709 12:23:40.660909  3143 catalog_manager.cc:5697] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:40.671797  3995 raft_consensus.cc:1275] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:40.672329  3245 raft_consensus.cc:1275] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:40.674496  4403 consensus_queue.cc:1048] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:40.678697  4393 consensus_queue.cc:1048] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:40.684051  3379 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a6d2799afd04cbe9bfaa74d467e66cb with delete_type TABLET_DATA_TOMBSTONED (TS c93989d3dd634ef585d77b03354121a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54706
I20260709 12:23:40.689594  4405 raft_consensus.cc:2955] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.697295  3244 raft_consensus.cc:2955] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.719954  4416 tablet_replica.cc:333] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:23:40.721738  4416 raft_consensus.cc:2243] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:40.722793  4416 raft_consensus.cc:2272] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:40.725076  3995 raft_consensus.cc:1275] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:40.727918  4329 consensus_queue.cc:1048] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:40.715677  3998 raft_consensus.cc:2955] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.729818  3143 catalog_manager.cc:5697] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.735749  3319 raft_consensus.cc:1275] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:40.736256  3245 raft_consensus.cc:1275] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:40.739302  4413 consensus_queue.cc:1048] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:40.740435  4416 ts_tablet_manager.cc:1916] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:40.740722  4371 consensus_queue.cc:1048] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:40.762338  3134 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 87fddcaae9194bef90d1047a518e45a6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:40.767367  4415 tablet_replica.cc:333] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:23:40.768805  4415 raft_consensus.cc:2243] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:40.770190  4415 raft_consensus.cc:2272] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:40.798918  4415 ts_tablet_manager.cc:1916] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:40.815656  3379 tablet_service.cc:1558] Processing DeleteTablet for tablet 87fddcaae9194bef90d1047a518e45a6 with delete_type TABLET_DATA_TOMBSTONED (TS c93989d3dd634ef585d77b03354121a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54706
I20260709 12:23:40.821444  4416 ts_tablet_manager.cc:1929] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 12:23:40.824394  4416 log.cc:1199] T 7a6d2799afd04cbe9bfaa74d467e66cb P c93989d3dd634ef585d77b03354121a4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/wals/7a6d2799afd04cbe9bfaa74d467e66cb
I20260709 12:23:40.826704  3133 catalog_manager.cc:5028] TS c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): tablet 7a6d2799afd04cbe9bfaa74d467e66cb (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:40.827033  4416 tablet_replica.cc:333] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:23:40.828308  4416 raft_consensus.cc:2243] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:40.829151  4416 raft_consensus.cc:2272] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:40.833282  4329 raft_consensus.cc:2955] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.836346  4415 ts_tablet_manager.cc:1929] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:40.836957  4415 log.cc:1199] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/wals/9253ca0fa41f43a7ac13ff17879c2ed3
I20260709 12:23:40.840065  3132 catalog_manager.cc:5028] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): tablet 9253ca0fa41f43a7ac13ff17879c2ed3 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:40.842366  4416 ts_tablet_manager.cc:1916] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:40.846834  3995 raft_consensus.cc:2955] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.851733  3317 raft_consensus.cc:2955] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.867060  4416 ts_tablet_manager.cc:1929] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:40.867625  4416 log.cc:1199] T 87fddcaae9194bef90d1047a518e45a6 P c93989d3dd634ef585d77b03354121a4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/wals/87fddcaae9194bef90d1047a518e45a6
I20260709 12:23:40.869436  3133 catalog_manager.cc:5028] TS c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): tablet 87fddcaae9194bef90d1047a518e45a6 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:40.873123  3245 raft_consensus.cc:2955] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.881897  3144 catalog_manager.cc:5697] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.913750  3394 consensus_queue.cc:237] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:40.918841  3995 raft_consensus.cc:1275] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:40.919255  3244 raft_consensus.cc:1275] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:40.921190  4371 consensus_queue.cc:1048] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:40.924154  4398 consensus_queue.cc:1048] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:40.931818  4398 raft_consensus.cc:2955] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.933775  3995 raft_consensus.cc:2955] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.944868  3244 raft_consensus.cc:2955] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.958951  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0634624f456740b48b77096645ab6b0b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:40.964273  3144 catalog_manager.cc:5697] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:40.992548  3311 tablet_service.cc:1558] Processing DeleteTablet for tablet 0634624f456740b48b77096645ab6b0b with delete_type TABLET_DATA_TOMBSTONED (TS 8525f77f43714a58bf158d872c1db3e4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40236
I20260709 12:23:40.995306  4420 tablet_replica.cc:333] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:23:40.996423  4420 raft_consensus.cc:2243] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:40.997200  4420 raft_consensus.cc:2272] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:41.000814  4420 ts_tablet_manager.cc:1916] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:41.018640  4420 ts_tablet_manager.cc:1929] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:41.019037  4420 log.cc:1199] T 0634624f456740b48b77096645ab6b0b P 8525f77f43714a58bf158d872c1db3e4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/wals/0634624f456740b48b77096645ab6b0b
I20260709 12:23:41.020771  3134 catalog_manager.cc:5028] TS 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): tablet 0634624f456740b48b77096645ab6b0b (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:42.068542  3394 consensus_queue.cc:237] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:42.083225  3317 raft_consensus.cc:1275] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:42.084860  3245 raft_consensus.cc:1275] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:42.084919  3362 consensus_peers.cc:597] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4a1156be8e3418790f41cd380136b93. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:42.084910  4398 consensus_queue.cc:1048] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:42.086391  4422 consensus_queue.cc:1048] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:42.097856  4398 raft_consensus.cc:2955] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:42.101697  3244 raft_consensus.cc:2955] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:42.113935  3245 consensus_queue.cc:237] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:42.126062  3317 raft_consensus.cc:2955] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:42.148015  3144 catalog_manager.cc:5697] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:42.155181  3319 raft_consensus.cc:1275] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:42.155416  3394 raft_consensus.cc:1275] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:42.160585  4390 consensus_queue.cc:1048] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:42.161970  4391 consensus_queue.cc:1048] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:42.170800  3213 consensus_peers.cc:597] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e2884104b65f4a589e915fd4328b729c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:42.187253  3319 raft_consensus.cc:2955] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:42.188670  3392 raft_consensus.cc:2955] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:42.187278  4391 raft_consensus.cc:2955] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:42.216042  3391 consensus_queue.cc:237] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:42.224246  3144 catalog_manager.cc:5697] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:42.226816  3317 raft_consensus.cc:1275] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:42.229031  4428 consensus_queue.cc:1048] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:42.234607  3362 consensus_peers.cc:597] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fd6cee8d33de46388ec38d683a789e94. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:42.237571  3244 raft_consensus.cc:1275] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:42.243153  4423 consensus_queue.cc:1048] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:42.246344  4398 raft_consensus.cc:2955] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:42.251210  3319 raft_consensus.cc:2955] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:42.266788  3317 consensus_queue.cc:237] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:42.273322  3244 raft_consensus.cc:2955] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:42.279393  3245 raft_consensus.cc:1275] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:42.279312  3392 raft_consensus.cc:1275] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:42.282246  3288 consensus_peers.cc:597] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6a9849f17fd4313ab9ddbb06589c46f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:42.283335  4368 consensus_queue.cc:1048] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:42.277388  3143 catalog_manager.cc:5697] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:42.285553  4442 consensus_queue.cc:1048] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:23:42.309801  4368 raft_consensus.cc:2955] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:42.311990  3392 raft_consensus.cc:2955] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:42.322193  3244 raft_consensus.cc:2955] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:42.335021  3147 catalog_manager.cc:5697] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:42.564533  4445 ts_tablet_manager.cc:933] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:42.569471  4445 tablet_copy_client.cc:323] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:23:42.571219  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet e2884104b65f4a589e915fd4328b729c from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:23:42.571703  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet e2884104b65f4a589e915fd4328b729c from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-e2884104b65f4a589e915fd4328b729c
I20260709 12:23:42.577544  3259 tablet_copy_source_session.cc:215] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:42.580523  4445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2884104b65f4a589e915fd4328b729c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.597426  4445 tablet_copy_client.cc:806] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:42.600644  4445 tablet_copy_client.cc:670] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:42.608160  4445 tablet_copy_client.cc:538] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:42.620011  4445 tablet_bootstrap.cc:492] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:42.646476  4445 tablet_bootstrap.cc:492] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:42.647432  4445 tablet_bootstrap.cc:492] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:42.648109  4445 ts_tablet_manager.cc:1403] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.028s	user 0.019s	sys 0.006s
I20260709 12:23:42.648720  4448 ts_tablet_manager.cc:933] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:42.651636  4445 raft_consensus.cc:359] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:42.652796  4448 tablet_copy_client.cc:323] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:23:42.653066  4445 raft_consensus.cc:740] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:42.653872  4445 consensus_queue.cc:260] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:42.654935  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet c4a1156be8e3418790f41cd380136b93 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:23:42.655524  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet c4a1156be8e3418790f41cd380136b93 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-c4a1156be8e3418790f41cd380136b93
I20260709 12:23:42.664830  4445 ts_tablet_manager.cc:1434] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.016s	user 0.014s	sys 0.001s
I20260709 12:23:42.667092  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-e2884104b65f4a589e915fd4328b729c received from {username='slave'} at 127.0.0.1:57122
I20260709 12:23:42.667629  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-e2884104b65f4a589e915fd4328b729c on tablet e2884104b65f4a589e915fd4328b729c with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:42.673532  3408 tablet_copy_source_session.cc:215] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:42.678182  4448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4a1156be8e3418790f41cd380136b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.695849  4448 tablet_copy_client.cc:806] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:42.696517  4448 tablet_copy_client.cc:670] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:42.702803  4445 ts_tablet_manager.cc:933] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:42.703881  4448 tablet_copy_client.cc:538] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:42.708375  4445 tablet_copy_client.cc:323] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.194:33649
I20260709 12:23:42.710610  3334 tablet_copy_service.cc:140] P 8525f77f43714a58bf158d872c1db3e4: Received BeginTabletCopySession request for tablet d6a9849f17fd4313ab9ddbb06589c46f from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:33282)
I20260709 12:23:42.711298  3334 tablet_copy_service.cc:161] P 8525f77f43714a58bf158d872c1db3e4: Beginning new tablet copy session on tablet d6a9849f17fd4313ab9ddbb06589c46f from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:33282: session id = e38cc90af94c47f8a86be67ad610cbd4-d6a9849f17fd4313ab9ddbb06589c46f
I20260709 12:23:42.714609  4448 tablet_bootstrap.cc:492] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:42.720333  3334 tablet_copy_source_session.cc:215] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:42.739884  4445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6a9849f17fd4313ab9ddbb06589c46f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.756472  4448 tablet_bootstrap.cc:492] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:42.757565  4448 tablet_bootstrap.cc:492] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:42.758322  4448 ts_tablet_manager.cc:1403] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.044s	user 0.018s	sys 0.011s
I20260709 12:23:42.758347  4445 tablet_copy_client.cc:806] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:42.760068  4445 tablet_copy_client.cc:670] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:42.765364  4448 raft_consensus.cc:359] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:42.766841  4448 raft_consensus.cc:740] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:42.767661  4448 consensus_queue.cc:260] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:42.768431  4445 tablet_copy_client.cc:538] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:42.770000  4448 ts_tablet_manager.cc:1434] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:23:42.772225  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-c4a1156be8e3418790f41cd380136b93 received from {username='slave'} at 127.0.0.1:34216
I20260709 12:23:42.772691  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-c4a1156be8e3418790f41cd380136b93 on tablet c4a1156be8e3418790f41cd380136b93 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:42.780261  4445 tablet_bootstrap.cc:492] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:42.814565  4445 tablet_bootstrap.cc:492] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:42.815547  4445 tablet_bootstrap.cc:492] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:42.816290  4445 ts_tablet_manager.cc:1403] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.036s	user 0.024s	sys 0.004s
I20260709 12:23:42.820657  4445 raft_consensus.cc:359] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:42.821514  4445 raft_consensus.cc:740] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:42.822304  4445 consensus_queue.cc:260] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:42.826987  4445 ts_tablet_manager.cc:1434] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:23:42.828859  3334 tablet_copy_service.cc:342] P 8525f77f43714a58bf158d872c1db3e4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-d6a9849f17fd4313ab9ddbb06589c46f received from {username='slave'} at 127.0.0.1:33282
I20260709 12:23:42.829417  3334 tablet_copy_service.cc:434] P 8525f77f43714a58bf158d872c1db3e4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-d6a9849f17fd4313ab9ddbb06589c46f on tablet d6a9849f17fd4313ab9ddbb06589c46f with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:42.834667  4448 ts_tablet_manager.cc:933] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:42.836450  4448 tablet_copy_client.cc:323] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:23:42.840654  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet fd6cee8d33de46388ec38d683a789e94 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:23:42.841413  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet fd6cee8d33de46388ec38d683a789e94 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-fd6cee8d33de46388ec38d683a789e94
I20260709 12:23:42.853698  3408 tablet_copy_source_session.cc:215] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:42.862033  4448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd6cee8d33de46388ec38d683a789e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:42.884477  4448 tablet_copy_client.cc:806] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:42.885241  4448 tablet_copy_client.cc:670] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:42.891814  4448 tablet_copy_client.cc:538] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:42.900784  4448 tablet_bootstrap.cc:492] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:42.936885  4448 tablet_bootstrap.cc:492] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:42.937778  4448 tablet_bootstrap.cc:492] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:42.938406  4448 ts_tablet_manager.cc:1403] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.038s	user 0.023s	sys 0.014s
I20260709 12:23:42.941880  4448 raft_consensus.cc:359] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:42.942600  4448 raft_consensus.cc:740] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:42.943280  4448 consensus_queue.cc:260] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:42.945439  4448 ts_tablet_manager.cc:1434] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:23:42.947798  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-fd6cee8d33de46388ec38d683a789e94 received from {username='slave'} at 127.0.0.1:34216
I20260709 12:23:42.948244  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-fd6cee8d33de46388ec38d683a789e94 on tablet fd6cee8d33de46388ec38d683a789e94 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:43.020376  3994 raft_consensus.cc:1217] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:43.231526  3994 raft_consensus.cc:1217] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:43.303364  3994 raft_consensus.cc:1217] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:43.398438  3994 raft_consensus.cc:1217] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:43.632895  4391 raft_consensus.cc:1064] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:43.635854  4391 consensus_queue.cc:237] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:43.642638  3994 raft_consensus.cc:1275] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:43.643082  3392 raft_consensus.cc:1275] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:43.644546  3320 raft_consensus.cc:1275] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:43.644804  4391 consensus_queue.cc:1048] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:43.646061  4431 consensus_queue.cc:1048] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:43.647033  4390 consensus_queue.cc:1048] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:43.671240  4431 raft_consensus.cc:2955] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:43.673789  3994 raft_consensus.cc:2955] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:43.673544  3392 raft_consensus.cc:2955] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:43.686825  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:43.695695  3320 raft_consensus.cc:2955] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:43.705176  3143 catalog_manager.cc:5697] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:43.743956  4398 raft_consensus.cc:1064] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:43.750308  4398 consensus_queue.cc:237] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:43.761160  3244 consensus_queue.cc:237] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:43.768684  3994 raft_consensus.cc:1275] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.771126  3317 raft_consensus.cc:1275] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.770299  4390 consensus_queue.cc:1048] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:23:43.772931  4431 consensus_queue.cc:1048] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:43.784711  4391 raft_consensus.cc:2955] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:43.791891  3994 raft_consensus.cc:1275] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:43.793843  3317 raft_consensus.cc:1275] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:43.793928  3320 raft_consensus.cc:2955] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:43.795892  4398 consensus_queue.cc:1048] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:43.792198  3244 raft_consensus.cc:1275] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:43.807205  3994 raft_consensus.cc:2955] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:43.810858  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e2884104b65f4a589e915fd4328b729c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:43.819113  3147 catalog_manager.cc:5697] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:43.820304  4458 consensus_queue.cc:1048] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:43.821973  4429 consensus_queue.cc:1048] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:43.853933  3379 tablet_service.cc:1558] Processing DeleteTablet for tablet e2884104b65f4a589e915fd4328b729c with delete_type TABLET_DATA_TOMBSTONED (TS c93989d3dd634ef585d77b03354121a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54706
I20260709 12:23:43.888942  4429 raft_consensus.cc:2955] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:43.896318  4471 tablet_replica.cc:333] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:23:43.894781  3244 raft_consensus.cc:2955] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:43.897979  4471 raft_consensus.cc:2243] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:43.901047  4471 raft_consensus.cc:2272] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:43.901396  3317 raft_consensus.cc:2955] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:43.910749  4458 raft_consensus.cc:1064] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:43.914026  4471 ts_tablet_manager.cc:1916] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:43.913578  4458 consensus_queue.cc:237] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:43.928401  4368 raft_consensus.cc:1064] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:43.931277  3146 catalog_manager.cc:5697] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:43.944931  4471 ts_tablet_manager.cc:1929] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:43.945683  4471 log.cc:1199] T e2884104b65f4a589e915fd4328b729c P c93989d3dd634ef585d77b03354121a4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/wals/e2884104b65f4a589e915fd4328b729c
I20260709 12:23:43.947705  3133 catalog_manager.cc:5028] TS c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): tablet e2884104b65f4a589e915fd4328b729c (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:43.955215  3998 raft_consensus.cc:2955] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:43.958081  4368 consensus_queue.cc:237] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:43.986012  3245 raft_consensus.cc:1275] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:43.986205  3994 raft_consensus.cc:1275] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:43.991411  3998 raft_consensus.cc:1275] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:43.992076  4398 consensus_queue.cc:1048] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:43.993889  3245 raft_consensus.cc:1275] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:43.994822  4458 consensus_queue.cc:1048] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:43.996744  4475 consensus_queue.cc:1048] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:44.002761  3319 raft_consensus.cc:1275] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:44.014292  3392 consensus_queue.cc:237] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:44.036017  4458 consensus_queue.cc:1048] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:44.025506  3394 raft_consensus.cc:1275] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:44.047861  4393 consensus_queue.cc:1048] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:23:44.050158  4475 consensus_queue.cc:1048] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:44.051448  3995 raft_consensus.cc:1275] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:44.055150  3317 raft_consensus.cc:1275] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:44.073766  4423 consensus_queue.cc:1048] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:44.087612  4474 consensus_queue.cc:1048] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:44.147650  4475 raft_consensus.cc:2955] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.162771  4458 raft_consensus.cc:2955] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.169533  3997 raft_consensus.cc:2955] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:44.190582  4480 log.cc:927] Time spent T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:23:44.219870  3994 raft_consensus.cc:2955] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.230237  3147 catalog_manager.cc:5697] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.231088  4423 raft_consensus.cc:2955] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.236340  3243 raft_consensus.cc:2955] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.236804  3998 raft_consensus.cc:2955] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.240701  3244 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14193bb0c64048cf8ad561f477695970"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:23:44.241600  3244 raft_consensus.cc:606] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Received request to transfer leadership to TS 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:44.256248  3244 raft_consensus.cc:2955] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.261250  3144 catalog_manager.cc:5697] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:44.277395  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c4a1156be8e3418790f41cd380136b93 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:44.298072  3392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f76214f21f642889fe174df15bb7047"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:44.298764  3392 raft_consensus.cc:606] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:44.302897  3394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2bde98a54e9f47a0b9afe40221bcb336"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:44.303776  3394 raft_consensus.cc:606] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:44.304754  3146 catalog_manager.cc:5697] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.308432  3243 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73563dc7466d4636a64ebd4d363935b1"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:23:44.309187  3243 raft_consensus.cc:606] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:44.315234  3395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:44.315907  3395 raft_consensus.cc:606] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:44.317588  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:44.319775  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:44.324759  3239 tablet_service.cc:1558] Processing DeleteTablet for tablet c4a1156be8e3418790f41cd380136b93 with delete_type TABLET_DATA_TOMBSTONED (TS 7cdaa0234c64450eb4ce4500b51b8a81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53720
I20260709 12:23:44.329309  3395 consensus_queue.cc:237] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:44.331264  3317 raft_consensus.cc:2955] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.375313  3997 raft_consensus.cc:1275] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:44.379297  4490 tablet_replica.cc:333] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:23:44.383013  4490 raft_consensus.cc:2243] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:44.384112  4490 raft_consensus.cc:2272] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:44.388190  4490 ts_tablet_manager.cc:1916] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:44.390906  3244 raft_consensus.cc:1275] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:44.396890  4488 consensus_queue.cc:1048] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:44.400664  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:44.415745  4458 consensus_queue.cc:1048] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:44.416030  4490 ts_tablet_manager.cc:1929] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:44.416718  4490 log.cc:1199] T c4a1156be8e3418790f41cd380136b93 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/wals/c4a1156be8e3418790f41cd380136b93
I20260709 12:23:44.418749  3132 catalog_manager.cc:5028] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): tablet c4a1156be8e3418790f41cd380136b93 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:44.421327  4458 raft_consensus.cc:2955] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.431313  3392 raft_consensus.cc:2955] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.470489  3997 raft_consensus.cc:2955] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.593737  4487 raft_consensus.cc:993] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:44.594679  4487 raft_consensus.cc:1081] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:44.606277  3245 raft_consensus.cc:2955] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.642977  4493 raft_consensus.cc:993] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:44.643958  4492 raft_consensus.cc:1081] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Signalling peer 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:44.648156  3316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14193bb0c64048cf8ad561f477695970"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:51186
I20260709 12:23:44.649719  3316 raft_consensus.cc:493] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.650100  3316 raft_consensus.cc:3060] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.679883  3316 raft_consensus.cc:515] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:44.683224  3316 leader_election.cc:290] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:44.688650  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fd6cee8d33de46388ec38d683a789e94 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:44.689437  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14193bb0c64048cf8ad561f477695970" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:44.698379  3248 raft_consensus.cc:3055] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:44.698743  3248 raft_consensus.cc:740] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:44.699497  3248 consensus_queue.cc:260] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:44.700676  3248 raft_consensus.cc:3060] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.710541  3248 raft_consensus.cc:2468] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:44.722756  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14193bb0c64048cf8ad561f477695970" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:44.723712  3394 raft_consensus.cc:3060] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.725610  4486 raft_consensus.cc:993] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:44.726284  4486 raft_consensus.cc:1081] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:44.743371  3317 raft_consensus.cc:2955] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.750008  3244 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36494
I20260709 12:23:44.751003  3244 raft_consensus.cc:493] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.751379  3244 raft_consensus.cc:3060] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:44.766966  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 81202us
I20260709 12:23:44.780804  3394 raft_consensus.cc:2468] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:44.785142  3244 raft_consensus.cc:515] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:44.787786  3244 leader_election.cc:290] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:44.787957  3289 leader_election.cc:304] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:44.799576  4393 raft_consensus.cc:2804] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:44.800190  4393 raft_consensus.cc:697] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:44.797247  3994 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f76214f21f642889fe174df15bb7047"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:55592
I20260709 12:23:44.801117  4393 consensus_queue.cc:237] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:44.801158  3145 catalog_manager.cc:5697] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:44.801183  3994 raft_consensus.cc:493] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.809424  3994 raft_consensus.cc:3060] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.811904  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:44.812639  3394 raft_consensus.cc:3055] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:44.815171  4391 raft_consensus.cc:993] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:44.815863  4391 raft_consensus.cc:1081] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:44.812933  3394 raft_consensus.cc:740] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:44.817744  3318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:44.818725  3318 raft_consensus.cc:3060] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.819049  3394 consensus_queue.cc:260] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:44.820564  3394 raft_consensus.cc:3060] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.826047  3992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73563dc7466d4636a64ebd4d363935b1"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:55596
I20260709 12:23:44.826749  3992 raft_consensus.cc:493] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.827149  3992 raft_consensus.cc:3060] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.838557  3394 raft_consensus.cc:2468] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:44.839808  3994 raft_consensus.cc:515] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:44.844393  3317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f76214f21f642889fe174df15bb7047" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:44.844633  3992 raft_consensus.cc:515] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:44.845355  3317 raft_consensus.cc:3060] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.845474  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f76214f21f642889fe174df15bb7047" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:44.846151  3394 raft_consensus.cc:3055] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:44.846527  3394 raft_consensus.cc:740] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:44.853353  3992 leader_election.cc:290] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:44.850785  3394 consensus_queue.cc:260] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:44.855140  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73563dc7466d4636a64ebd4d363935b1" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:44.855506  3214 leader_election.cc:304] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:44.855821  3396 raft_consensus.cc:3060] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.856534  4390 raft_consensus.cc:2804] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:44.854640  3394 raft_consensus.cc:3060] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.853361  3318 raft_consensus.cc:2468] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:44.857209  4390 raft_consensus.cc:697] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:44.858501  4390 consensus_queue.cc:237] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:44.864506  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73563dc7466d4636a64ebd4d363935b1" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:44.865545  3994 leader_election.cc:290] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:44.866003  3244 raft_consensus.cc:3055] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:44.866530  3244 raft_consensus.cc:740] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:44.866751  3396 raft_consensus.cc:2468] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:44.867479  3392 raft_consensus.cc:1240] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Rejecting Update request from peer 7cdaa0234c64450eb4ce4500b51b8a81 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:44.867348  3244 consensus_queue.cc:260] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:44.869297  3244 raft_consensus.cc:3060] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.874123  3394 raft_consensus.cc:2468] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:44.875758  4486 raft_consensus.cc:993] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:44.877187  4429 raft_consensus.cc:1081] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:44.877254  3962 leader_election.cc:304] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:23:44.879344  4450 raft_consensus.cc:2804] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:44.882391  3317 raft_consensus.cc:2468] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:44.887691  4450 raft_consensus.cc:697] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:23:44.888442  3244 raft_consensus.cc:2468] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:44.887964  3962 leader_election.cc:304] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:23:44.889003  4450 consensus_queue.cc:237] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:44.889765  4499 raft_consensus.cc:2804] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:44.904017  3244 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2bde98a54e9f47a0b9afe40221bcb336"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36494
I20260709 12:23:44.904956  3244 raft_consensus.cc:493] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.905431  3244 raft_consensus.cc:3060] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.914803  3244 raft_consensus.cc:515] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:44.918035  3244 leader_election.cc:290] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:44.919950  3318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bde98a54e9f47a0b9afe40221bcb336" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:44.920650  3318 raft_consensus.cc:3060] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.922703  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bde98a54e9f47a0b9afe40221bcb336" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:44.923326  3396 raft_consensus.cc:3055] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:44.923599  3396 raft_consensus.cc:740] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:44.924366  3396 consensus_queue.cc:260] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:44.925555  3396 raft_consensus.cc:3060] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.928897  3318 raft_consensus.cc:2468] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:44.930154  4499 raft_consensus.cc:697] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:23:44.930542  3215 leader_election.cc:304] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:44.931380  4390 raft_consensus.cc:2804] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:44.931890  4390 raft_consensus.cc:697] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:44.932832  4390 consensus_queue.cc:237] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:44.938702  3311 tablet_service.cc:1558] Processing DeleteTablet for tablet fd6cee8d33de46388ec38d683a789e94 with delete_type TABLET_DATA_TOMBSTONED (TS 8525f77f43714a58bf158d872c1db3e4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40236
I20260709 12:23:44.937983  3141 catalog_manager.cc:5697] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:44.939618  3396 raft_consensus.cc:2468] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:44.942914  4504 tablet_replica.cc:333] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:23:44.959448  4499 consensus_queue.cc:237] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:44.979038  3318 consensus_queue.cc:237] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:44.984869  3994 raft_consensus.cc:1275] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:44.986769  4393 consensus_queue.cc:1048] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:44.988854  3396 raft_consensus.cc:1275] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:44.990780  4491 consensus_queue.cc:1048] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:44.999074  3141 catalog_manager.cc:5697] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:45.000700  4393 raft_consensus.cc:2955] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:45.001430  3141 catalog_manager.cc:5697] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:45.007426  3994 raft_consensus.cc:2955] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:45.008643  4504 raft_consensus.cc:2243] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.011250  3394 raft_consensus.cc:2955] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:45.019868  3134 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d6a9849f17fd4313ab9ddbb06589c46f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:45.029667  3141 catalog_manager.cc:5697] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:45.012245  4504 raft_consensus.cc:2272] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.035949  3140 catalog_manager.cc:5697] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:45.042333  4504 ts_tablet_manager.cc:1916] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:45.047665  3239 tablet_service.cc:1558] Processing DeleteTablet for tablet d6a9849f17fd4313ab9ddbb06589c46f with delete_type TABLET_DATA_TOMBSTONED (TS 7cdaa0234c64450eb4ce4500b51b8a81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53720
I20260709 12:23:45.038190  3140 catalog_manager.cc:5697] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:45.059854  4507 tablet_replica.cc:333] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:23:45.061611  4507 raft_consensus.cc:2243] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.062577  4507 raft_consensus.cc:2272] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.067471  4507 ts_tablet_manager.cc:1916] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:45.075631  4504 ts_tablet_manager.cc:1929] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:45.076205  4504 log.cc:1199] T fd6cee8d33de46388ec38d683a789e94 P 8525f77f43714a58bf158d872c1db3e4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/wals/fd6cee8d33de46388ec38d683a789e94
I20260709 12:23:45.077997  3134 catalog_manager.cc:5028] TS 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): tablet fd6cee8d33de46388ec38d683a789e94 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:45.092500  4507 ts_tablet_manager.cc:1929] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:45.093170  4507 log.cc:1199] T d6a9849f17fd4313ab9ddbb06589c46f P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/wals/d6a9849f17fd4313ab9ddbb06589c46f
I20260709 12:23:45.095206  3132 catalog_manager.cc:5028] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): tablet d6a9849f17fd4313ab9ddbb06589c46f (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:45.317370  3317 raft_consensus.cc:1275] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:45.319017  4499 consensus_queue.cc:1048] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:45.343644  3318 raft_consensus.cc:1275] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.346498  4485 consensus_queue.cc:1048] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:45.351661  3396 raft_consensus.cc:1275] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:45.353394  4500 consensus_queue.cc:1048] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:45.358978  3394 raft_consensus.cc:1275] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:45.360996  4450 consensus_queue.cc:1048] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:45.373052  3244 raft_consensus.cc:1275] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.375159  4393 consensus_queue.cc:1048] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:45.383862  3392 raft_consensus.cc:1275] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.384397  3391 raft_consensus.cc:1275] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.385613  4485 consensus_queue.cc:1048] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:45.389001  4390 consensus_queue.cc:1048] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:45.397536  3394 raft_consensus.cc:1275] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.402424  3247 raft_consensus.cc:1275] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:45.404801  4491 consensus_queue.cc:1048] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:45.414834  4500 consensus_queue.cc:1048] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:45.489514  3317 raft_consensus.cc:1275] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:45.508558  4524 consensus_queue.cc:1048] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:45.533564  3178 debug-util.cc:398] Leaking SignalData structure 0x7b080008a680 after lost signal to thread 2070
W20260709 12:23:45.535169  3178 debug-util.cc:398] Leaking SignalData structure 0x7b080008ad60 after lost signal to thread 3181
W20260709 12:23:45.536525  3178 debug-util.cc:398] Leaking SignalData structure 0x7b0800119c40 after lost signal to thread 3273
W20260709 12:23:45.537793  3178 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5260 after lost signal to thread 3348
W20260709 12:23:45.539016  3178 debug-util.cc:398] Leaking SignalData structure 0x7b0800323860 after lost signal to thread 3422
W20260709 12:23:45.540179  3178 debug-util.cc:398] Leaking SignalData structure 0x7b080033f540 after lost signal to thread 4023
I20260709 12:23:45.804370  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:45.867938  3398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "054906db16c94967bdef7c9dd5a47ea3"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:45.868651  3398 raft_consensus.cc:606] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:45.899956  3320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:48592
I20260709 12:23:45.900826  3320 raft_consensus.cc:606] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:45.904683  3247 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:23:45.905336  3247 raft_consensus.cc:606] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:45.908816  3247 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a2be382cd47e41839958b59bc8e3c40a"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:23:45.909377  3247 raft_consensus.cc:606] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Received request to transfer leadership to TS 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:45.913143  3398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb11c9e1f3b048769b242da5c7ae7cd7"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:45.914001  3398 raft_consensus.cc:606] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:45.915462  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:45.917744  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:45.919198  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:45.975803  4423 raft_consensus.cc:993] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4: : Instructing follower 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:45.976686  4423 raft_consensus.cc:1081] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:46.003394  3318 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb11c9e1f3b048769b242da5c7ae7cd7"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:51194
I20260709 12:23:46.004159  3318 raft_consensus.cc:493] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.004556  3318 raft_consensus.cc:3060] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.017999  3318 raft_consensus.cc:515] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:46.023849  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11c9e1f3b048769b242da5c7ae7cd7" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:46.024542  3247 raft_consensus.cc:3060] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.038129  4523 raft_consensus.cc:993] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:46.040277  4523 raft_consensus.cc:1081] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:46.042807  3994 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:55596
I20260709 12:23:46.044000  3994 raft_consensus.cc:493] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.044569  3994 raft_consensus.cc:3060] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.047396  3318 leader_election.cc:290] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:46.049489  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11c9e1f3b048769b242da5c7ae7cd7" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:46.050691  3398 raft_consensus.cc:3055] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:46.050768  3247 raft_consensus.cc:2468] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:46.051182  3398 raft_consensus.cc:740] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:46.052417  3288 leader_election.cc:304] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:46.052176  3398 consensus_queue.cc:260] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:46.053516  4491 raft_consensus.cc:2804] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:46.053928  3398 raft_consensus.cc:3060] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.053846  3994 raft_consensus.cc:515] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:46.054711  4491 raft_consensus.cc:697] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:46.055898  4491 consensus_queue.cc:237] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:46.057121  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:46.057870  3247 raft_consensus.cc:3055] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:46.058184  3247 raft_consensus.cc:740] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:46.059052  3247 consensus_queue.cc:260] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:46.060138  3247 raft_consensus.cc:3060] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.064707  3398 raft_consensus.cc:2468] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:46.066356  3994 leader_election.cc:290] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:46.068698  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:46.070824  3398 raft_consensus.cc:3060] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.076152  3247 raft_consensus.cc:2468] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:46.091944  3961 leader_election.cc:304] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:23:46.092983  4531 raft_consensus.cc:2804] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:46.093533  4531 raft_consensus.cc:697] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:23:46.094662  4531 consensus_queue.cc:237] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:46.100546  3398 raft_consensus.cc:2468] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:46.125025  3140 catalog_manager.cc:5697] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:46.131160  3144 catalog_manager.cc:5697] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:46.147544  4494 raft_consensus.cc:993] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:46.148130  4494 raft_consensus.cc:1081] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:46.150514  3244 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36508
I20260709 12:23:46.151124  3244 raft_consensus.cc:493] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.151477  3244 raft_consensus.cc:3060] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.163406  4423 raft_consensus.cc:993] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:46.164096  4423 raft_consensus.cc:1081] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:46.169214  3247 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "054906db16c94967bdef7c9dd5a47ea3"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36494
I20260709 12:23:46.169940  3247 raft_consensus.cc:493] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.170295  3247 raft_consensus.cc:3060] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.204707  3247 raft_consensus.cc:515] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:46.216399  3244 raft_consensus.cc:515] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:46.223654  3244 leader_election.cc:290] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:46.225764  3392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:46.225865  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054906db16c94967bdef7c9dd5a47ea3" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:46.226423  3392 raft_consensus.cc:3060] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.226672  3398 raft_consensus.cc:3055] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:46.227351  3398 raft_consensus.cc:740] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:46.228839  3398 consensus_queue.cc:260] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:46.230263  3398 raft_consensus.cc:3060] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.237998  3247 leader_election.cc:290] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:23:46.240702  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 55361us
I20260709 12:23:46.244038  3392 raft_consensus.cc:2468] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:46.252640  3318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:46.255647  3318 raft_consensus.cc:3055] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:46.256021  3318 raft_consensus.cc:740] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:46.256820  3318 consensus_queue.cc:260] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:46.257987  3318 raft_consensus.cc:3060] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.253033  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054906db16c94967bdef7c9dd5a47ea3" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:46.265935  3320 raft_consensus.cc:3060] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.267402  3398 raft_consensus.cc:2468] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:46.271482  3214 leader_election.cc:304] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:46.272445  4523 raft_consensus.cc:2804] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:46.272935  4523 raft_consensus.cc:697] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:46.273880  4523 consensus_queue.cc:237] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:46.282716  3320 raft_consensus.cc:2468] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:46.288733  3391 raft_consensus.cc:1240] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Rejecting Update request from peer 8525f77f43714a58bf158d872c1db3e4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:46.293473  3214 leader_election.cc:304] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:46.294668  4523 raft_consensus.cc:2804] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:46.295262  4523 raft_consensus.cc:697] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:46.296372  4523 consensus_queue.cc:237] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:46.301561  3318 raft_consensus.cc:2468] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:46.301420  3141 catalog_manager.cc:5697] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:46.303287  3244 consensus_queue.cc:237] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
W20260709 12:23:46.303946  4536 log.cc:927] Time spent T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:23:46.337311  3398 raft_consensus.cc:1275] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:46.338397  3318 raft_consensus.cc:1275] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:46.339270  4390 consensus_queue.cc:1048] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:46.341907  4524 consensus_queue.cc:1048] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:46.350082  3213 consensus_peers.cc:597] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3098f015fb154cf490e147a8e71f65b6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:46.383318  4523 raft_consensus.cc:2955] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:46.391633  3318 raft_consensus.cc:2955] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:46.429526  4525 raft_consensus.cc:993] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:46.430095  4525 raft_consensus.cc:1081] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Signalling peer 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:46.431085  3144 catalog_manager.cc:5697] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:46.436151  3320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a2be382cd47e41839958b59bc8e3c40a"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:51186
I20260709 12:23:46.436775  3320 raft_consensus.cc:493] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.437095  3320 raft_consensus.cc:3060] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.450026  3320 raft_consensus.cc:515] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:46.456310  3398 raft_consensus.cc:2955] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:46.458827  3318 consensus_queue.cc:237] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:46.482936  3320 leader_election.cc:290] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:46.488173  3141 catalog_manager.cc:5697] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:46.489295  3392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2be382cd47e41839958b59bc8e3c40a" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:46.493036  3392 raft_consensus.cc:3060] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.505419  3242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2be382cd47e41839958b59bc8e3c40a" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:46.506618  3242 raft_consensus.cc:3055] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:46.507002  3242 raft_consensus.cc:740] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:46.507889  3242 consensus_queue.cc:260] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:46.514087  3242 raft_consensus.cc:3060] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.522099  3392 raft_consensus.cc:2468] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:46.528208  3289 leader_election.cc:304] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:46.536150  3392 raft_consensus.cc:1275] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:46.541250  4538 raft_consensus.cc:2804] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:46.541975  4538 raft_consensus.cc:697] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:46.542972  4538 consensus_queue.cc:237] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:46.544193  4531 consensus_queue.cc:1048] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:46.553723  3242 raft_consensus.cc:2468] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:46.570574  3242 raft_consensus.cc:1275] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:23:46.578401  3288 consensus_peers.cc:597] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 14193bb0c64048cf8ad561f477695970. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:46.580173  4531 consensus_queue.cc:1048] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:46.581175  3391 raft_consensus.cc:1275] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:46.581892  3244 raft_consensus.cc:1275] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:46.629177  3391 raft_consensus.cc:1275] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:46.628481  3140 catalog_manager.cc:5697] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:46.631517  4393 consensus_queue.cc:1048] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:46.607488  4491 consensus_queue.cc:1048] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:23:46.643756  4537 consensus_queue.cc:1048] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:46.686285  3244 raft_consensus.cc:1275] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:46.688202  4537 consensus_queue.cc:1048] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:46.714408  4537 raft_consensus.cc:2955] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:46.724817  3391 raft_consensus.cc:2955] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:46.737491  3244 raft_consensus.cc:2955] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:46.750468  3398 raft_consensus.cc:1275] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:46.756258  3317 raft_consensus.cc:1275] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:46.758956  4485 consensus_queue.cc:1048] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:46.761111  3391 consensus_queue.cc:237] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:46.762121  4525 consensus_queue.cc:1048] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:46.770710  3141 catalog_manager.cc:5697] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:46.802390  3317 raft_consensus.cc:1275] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:46.820640  4488 consensus_queue.cc:1048] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:46.820952  3394 raft_consensus.cc:1275] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:23:46.831573  3362 consensus_peers.cc:597] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 00dbb3c83d794b5db75532d598235d9a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:46.840647  4523 consensus_queue.cc:1048] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:46.843644  3247 raft_consensus.cc:1275] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:46.850188  4423 consensus_queue.cc:1048] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:46.873778  3318 raft_consensus.cc:1275] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:46.895712  4549 raft_consensus.cc:2955] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:46.896589  4525 consensus_queue.cc:1048] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:46.898823  4567 ts_tablet_manager.cc:933] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:46.926924  3394 consensus_queue.cc:237] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:46.943311  3317 raft_consensus.cc:2955] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:46.950707  3141 catalog_manager.cc:5697] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:46.984620  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:46.985069  3244 raft_consensus.cc:2955] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
W20260709 12:23:46.998927  3362 consensus_peers.cc:597] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: caed93336fbb415c82acd602adce7969. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:47.005134  3247 raft_consensus.cc:1275] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:47.007227  4567 tablet_copy_client.cc:323] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:23:47.009482  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet 3098f015fb154cf490e147a8e71f65b6 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:23:47.010182  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet 3098f015fb154cf490e147a8e71f65b6 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-3098f015fb154cf490e147a8e71f65b6
I20260709 12:23:47.015232  4488 consensus_queue.cc:1048] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:47.025683  3259 tablet_copy_source_session.cc:215] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:47.027639  3316 raft_consensus.cc:1275] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:47.030022  4572 ts_tablet_manager.cc:933] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:47.031543  4567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3098f015fb154cf490e147a8e71f65b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:47.033290  4532 consensus_queue.cc:1048] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:47.057336  3244 raft_consensus.cc:1275] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:47.059526  4491 consensus_queue.cc:1048] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:47.067948  4567 tablet_copy_client.cc:806] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:47.068789  4567 tablet_copy_client.cc:670] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:47.072625  4572 tablet_copy_client.cc:323] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.194:33649
I20260709 12:23:47.075119  4567 tablet_copy_client.cc:538] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:47.076013  3334 tablet_copy_service.cc:140] P 8525f77f43714a58bf158d872c1db3e4: Received BeginTabletCopySession request for tablet 14193bb0c64048cf8ad561f477695970 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:33282)
I20260709 12:23:47.077360  3334 tablet_copy_service.cc:161] P 8525f77f43714a58bf158d872c1db3e4: Beginning new tablet copy session on tablet 14193bb0c64048cf8ad561f477695970 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:33282: session id = e38cc90af94c47f8a86be67ad610cbd4-14193bb0c64048cf8ad561f477695970
I20260709 12:23:47.084214  3394 raft_consensus.cc:1275] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:47.087723  4491 consensus_queue.cc:1048] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:47.094434  4567 tablet_bootstrap.cc:492] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:47.098261  3334 tablet_copy_source_session.cc:215] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:47.113511  4549 raft_consensus.cc:2955] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:47.131712  3247 raft_consensus.cc:2955] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:47.138132  4572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14193bb0c64048cf8ad561f477695970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:47.159281  3316 raft_consensus.cc:2955] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:47.204573  3140 catalog_manager.cc:5697] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:47.218643  4567 tablet_bootstrap.cc:492] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:47.216737  4572 tablet_copy_client.cc:806] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:47.224577  4572 tablet_copy_client.cc:670] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:47.227836  3242 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a5d938460364390a17cada19308b314"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:23:47.228466  3242 raft_consensus.cc:606] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:47.230397  4572 tablet_copy_client.cc:538] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:47.232040  4567 tablet_bootstrap.cc:492] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:47.232947  4567 ts_tablet_manager.cc:1403] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.139s	user 0.035s	sys 0.051s
I20260709 12:23:47.232827  3242 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "572d97394c304e81affa5ee75b52de28"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:23:47.235963  3242 raft_consensus.cc:606] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:47.240094  3316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9253ca0fa41f43a7ac13ff17879c2ed3"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:48592
I20260709 12:23:47.240768  3316 raft_consensus.cc:606] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:47.246217  3394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8f62d0c16794e8c941849d55bb4bb0b"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:47.247164  3394 raft_consensus.cc:606] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:47.251685  4567 raft_consensus.cc:359] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:47.253829  4572 tablet_bootstrap.cc:492] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:47.254894  4567 raft_consensus.cc:740] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:47.255941  4567 consensus_queue.cc:260] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:47.256935  3316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c04a8b7530d5486a880a89a4ba0faac8"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:48592
I20260709 12:23:47.257849  3316 raft_consensus.cc:606] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:47.261309  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:47.263043  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:47.263901  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:47.264206  4567 ts_tablet_manager.cc:1434] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.031s	user 0.001s	sys 0.007s
I20260709 12:23:47.267042  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-3098f015fb154cf490e147a8e71f65b6 received from {username='slave'} at 127.0.0.1:57122
I20260709 12:23:47.267544  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-3098f015fb154cf490e147a8e71f65b6 on tablet 3098f015fb154cf490e147a8e71f65b6 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:47.308980  4547 raft_consensus.cc:993] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:47.309777  4485 raft_consensus.cc:1081] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:47.312857  3992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "572d97394c304e81affa5ee75b52de28"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:55596
I20260709 12:23:47.313669  3992 raft_consensus.cc:493] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.314153  3992 raft_consensus.cc:3060] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.320247  3994 raft_consensus.cc:1217] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:47.334681  4567 ts_tablet_manager.cc:933] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:47.343254  4572 tablet_bootstrap.cc:492] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:47.350358  4572 tablet_bootstrap.cc:492] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:47.351444  4572 ts_tablet_manager.cc:1403] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.098s	user 0.018s	sys 0.009s
I20260709 12:23:47.381924  4572 raft_consensus.cc:359] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:47.383252  4572 raft_consensus.cc:740] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:47.384032  4572 consensus_queue.cc:260] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:47.385658  4567 tablet_copy_client.cc:323] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:23:47.389076  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet 00dbb3c83d794b5db75532d598235d9a from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:23:47.389873  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet 00dbb3c83d794b5db75532d598235d9a from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-00dbb3c83d794b5db75532d598235d9a
I20260709 12:23:47.369067  3992 raft_consensus.cc:515] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:47.396843  3242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d97394c304e81affa5ee75b52de28" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:47.397559  3242 raft_consensus.cc:3055] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:47.397714  4572 ts_tablet_manager.cc:1434] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 12:23:47.397966  3242 raft_consensus.cc:740] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:47.398662  3408 tablet_copy_source_session.cc:215] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:47.398886  3242 consensus_queue.cc:260] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:47.399948  3334 tablet_copy_service.cc:342] P 8525f77f43714a58bf158d872c1db3e4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-14193bb0c64048cf8ad561f477695970 received from {username='slave'} at 127.0.0.1:33282
I20260709 12:23:47.400290  3242 raft_consensus.cc:3060] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.400424  3334 tablet_copy_service.cc:434] P 8525f77f43714a58bf158d872c1db3e4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-14193bb0c64048cf8ad561f477695970 on tablet 14193bb0c64048cf8ad561f477695970 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:47.404613  3318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d97394c304e81affa5ee75b52de28" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:47.405257  3318 raft_consensus.cc:3060] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.408988  4567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00dbb3c83d794b5db75532d598235d9a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:47.413770  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 87760.7us
I20260709 12:23:47.415642  4580 raft_consensus.cc:993] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:47.417480  4491 raft_consensus.cc:1081] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:47.419481  3994 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c04a8b7530d5486a880a89a4ba0faac8"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:35074
I20260709 12:23:47.420189  3994 raft_consensus.cc:493] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.420836  3994 raft_consensus.cc:3060] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.421757  3992 leader_election.cc:290] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:47.461843  3992 raft_consensus.cc:1217] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:23:47.462821  3242 raft_consensus.cc:2468] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:47.474246  3961 leader_election.cc:304] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:23:47.475570  4531 raft_consensus.cc:2804] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:47.428884  3318 raft_consensus.cc:2468] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:47.485770  3994 raft_consensus.cc:515] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:47.498369  4531 raft_consensus.cc:697] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:23:47.499614  4531 consensus_queue.cc:237] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:47.506172  4580 raft_consensus.cc:993] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:47.506959  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04a8b7530d5486a880a89a4ba0faac8" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:47.508131  3316 raft_consensus.cc:3055] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:47.508535  3316 raft_consensus.cc:740] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:47.508623  3994 leader_election.cc:290] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:47.506994  4491 raft_consensus.cc:1081] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:47.510605  4532 raft_consensus.cc:993] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:47.510485  3316 consensus_queue.cc:260] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:47.511404  4532 raft_consensus.cc:1081] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:47.512041  3316 raft_consensus.cc:3060] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.515502  3996 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8f62d0c16794e8c941849d55bb4bb0b"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:55592
I20260709 12:23:47.516290  3996 raft_consensus.cc:493] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.516692  3996 raft_consensus.cc:3060] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.499460  3242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04a8b7530d5486a880a89a4ba0faac8" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:47.521435  3316 raft_consensus.cc:2468] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:47.526362  3242 raft_consensus.cc:3060] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.527158  3963 leader_election.cc:304] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:23:47.515364  3994 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9253ca0fa41f43a7ac13ff17879c2ed3"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:35074
I20260709 12:23:47.528148  3994 raft_consensus.cc:493] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.532066  3994 raft_consensus.cc:3060] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.537245  4567 tablet_copy_client.cc:806] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:47.538015  4567 tablet_copy_client.cc:670] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:47.531055  3996 raft_consensus.cc:515] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:47.528349  4582 raft_consensus.cc:2804] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:47.548094  4525 raft_consensus.cc:993] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:47.548695  4525 raft_consensus.cc:1081] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:47.551971  3992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a5d938460364390a17cada19308b314"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:55596
I20260709 12:23:47.552625  3992 raft_consensus.cc:493] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.553026  3992 raft_consensus.cc:3060] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.556351  3318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f62d0c16794e8c941849d55bb4bb0b" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:47.556965  3318 raft_consensus.cc:3060] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.567435  4572 ts_tablet_manager.cc:933] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:47.573133  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f62d0c16794e8c941849d55bb4bb0b" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:47.576217  4567 tablet_copy_client.cc:538] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:47.576743  3398 raft_consensus.cc:3055] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:47.570329  3996 leader_election.cc:290] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:47.577065  3398 raft_consensus.cc:740] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:47.578536  3398 consensus_queue.cc:260] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:47.579948  3398 raft_consensus.cc:3060] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.574764  3318 raft_consensus.cc:2468] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:47.582015  3963 leader_election.cc:304] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:23:47.582963  4583 raft_consensus.cc:2804] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:47.584797  4583 raft_consensus.cc:697] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:23:47.586838  4583 consensus_queue.cc:237] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:47.587536  3992 raft_consensus.cc:515] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:47.590565  4582 raft_consensus.cc:697] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:23:47.590943  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5d938460364390a17cada19308b314" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:47.591741  3249 raft_consensus.cc:3055] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:47.592211  3249 raft_consensus.cc:740] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:47.593152  3249 consensus_queue.cc:260] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:47.594455  3249 raft_consensus.cc:3060] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.598042  4572 tablet_copy_client.cc:323] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:23:47.599798  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet caed93336fbb415c82acd602adce7969 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:23:47.600327  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet caed93336fbb415c82acd602adce7969 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-caed93336fbb415c82acd602adce7969
I20260709 12:23:47.603507  3398 raft_consensus.cc:2468] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:47.608940  3994 raft_consensus.cc:515] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:47.610131  3249 raft_consensus.cc:2468] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:47.591939  4582 consensus_queue.cc:237] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:47.612475  3961 leader_election.cc:304] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:23:47.616067  3242 raft_consensus.cc:2468] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:47.629969  4585 raft_consensus.cc:2804] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:47.630731  4585 raft_consensus.cc:697] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:23:47.633401  3994 leader_election.cc:290] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:47.639016  3992 leader_election.cc:290] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:47.642100  3394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9253ca0fa41f43a7ac13ff17879c2ed3" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:47.646121  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5d938460364390a17cada19308b314" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:47.646395  3394 raft_consensus.cc:3060] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.646811  3316 raft_consensus.cc:3060] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.642423  3318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9253ca0fa41f43a7ac13ff17879c2ed3" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:47.650370  3318 raft_consensus.cc:3055] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:47.651602  3318 raft_consensus.cc:740] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:47.653777  3318 consensus_queue.cc:260] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:47.656041  3318 raft_consensus.cc:3060] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.660189  3316 raft_consensus.cc:2468] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:47.631788  4585 consensus_queue.cc:237] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:47.677408  3394 raft_consensus.cc:2468] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:47.680414  3962 leader_election.cc:304] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:23:47.681982  4583 raft_consensus.cc:2804] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:47.682730  3408 tablet_copy_source_session.cc:215] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:47.682833  4583 raft_consensus.cc:697] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:23:47.684203  4583 consensus_queue.cc:237] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:47.681528  4567 tablet_bootstrap.cc:492] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:47.689284  3318 raft_consensus.cc:2468] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:47.699923  4572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caed93336fbb415c82acd602adce7969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:47.739013  3140 catalog_manager.cc:5697] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.741403  3140 catalog_manager.cc:5697] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 1 to 2, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.763849  4524 raft_consensus.cc:1064] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:47.772512  4524 consensus_queue.cc:237] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:23:47.806208  4590 log.cc:927] Time spent T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:23:47.829473  3140 catalog_manager.cc:5697] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 1 to 2, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.833760  3140 catalog_manager.cc:5697] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.835577  3140 catalog_manager.cc:5697] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.841873  4567 tablet_bootstrap.cc:492] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:47.846491  4567 tablet_bootstrap.cc:492] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:47.848446  4567 ts_tablet_manager.cc:1403] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.167s	user 0.011s	sys 0.022s
I20260709 12:23:47.852015  3992 raft_consensus.cc:1275] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:47.863224  4524 consensus_queue.cc:1048] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:23:47.858338  4572 tablet_copy_client.cc:806] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:47.860749  3394 raft_consensus.cc:1275] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:47.867660  4572 tablet_copy_client.cc:670] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:47.873359  4572 tablet_copy_client.cc:538] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:47.874874  3318 raft_consensus.cc:1275] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:47.876449  4567 raft_consensus.cc:359] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:47.879474  4567 raft_consensus.cc:740] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:47.882059  4525 consensus_queue.cc:1048] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:47.887254  4572 tablet_bootstrap.cc:492] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:47.880716  4567 consensus_queue.cc:260] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:47.911222  4567 ts_tablet_manager.cc:1434] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.004s
I20260709 12:23:47.914492  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-00dbb3c83d794b5db75532d598235d9a received from {username='slave'} at 127.0.0.1:34216
I20260709 12:23:47.915066  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-00dbb3c83d794b5db75532d598235d9a on tablet 00dbb3c83d794b5db75532d598235d9a with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:47.914492  4523 consensus_queue.cc:1048] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:23:47.938616  4523 raft_consensus.cc:2955] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:47.957015  3316 raft_consensus.cc:2955] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:47.958940  4572 tablet_bootstrap.cc:492] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:47.960117  4572 tablet_bootstrap.cc:492] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:47.960929  4572 ts_tablet_manager.cc:1403] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.074s	user 0.029s	sys 0.009s
I20260709 12:23:47.964392  4572 raft_consensus.cc:359] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:47.965615  4572 raft_consensus.cc:740] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:47.982857  4393 raft_consensus.cc:1064] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:47.982697  3323 raft_consensus.cc:1275] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:47.986477  4572 consensus_queue.cc:260] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:47.989324  4572 ts_tablet_manager.cc:1434] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:23:47.990960  4585 consensus_queue.cc:1048] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:47.989240  4393 consensus_queue.cc:237] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:48.019085  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-caed93336fbb415c82acd602adce7969 received from {username='slave'} at 127.0.0.1:34216
I20260709 12:23:48.019673  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-caed93336fbb415c82acd602adce7969 on tablet caed93336fbb415c82acd602adce7969 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:48.038767  3318 raft_consensus.cc:1275] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.042757  4585 consensus_queue.cc:1048] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:48.070856  3398 raft_consensus.cc:1275] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 12:23:48.073122  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 58863.4us
I20260709 12:23:48.073408  4491 consensus_queue.cc:1048] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:48.080209  3323 raft_consensus.cc:1275] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.082271  4583 consensus_queue.cc:1048] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:48.119800  3994 raft_consensus.cc:1217] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:48.123710  3247 raft_consensus.cc:1275] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:48.141237  3391 raft_consensus.cc:1275] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.141889  3996 raft_consensus.cc:1275] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:48.143872  3391 raft_consensus.cc:2955] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:48.170413  4491 consensus_queue.cc:1048] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:48.174023  3247 raft_consensus.cc:1275] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.174417  3249 raft_consensus.cc:1275] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.175938  3996 raft_consensus.cc:2955] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:48.181311  4531 consensus_queue.cc:1048] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:48.181645  4582 consensus_queue.cc:1048] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:48.181547  4585 consensus_queue.cc:1048] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:48.200485  4580 consensus_queue.cc:1048] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:48.321864  3145 catalog_manager.cc:5697] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:48.388044  3242 raft_consensus.cc:1275] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.414151  3394 raft_consensus.cc:1275] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.418573  4491 raft_consensus.cc:2955] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:48.422694  3996 raft_consensus.cc:1217] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:48.446625  4620 consensus_queue.cc:1048] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:23:48.480384  4619 consensus_queue.cc:1048] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:23:48.495479  3323 raft_consensus.cc:1275] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.511852  3249 raft_consensus.cc:2955] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:48.525048  3323 raft_consensus.cc:1275] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.533998  4619 consensus_queue.cc:1048] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:48.578557  4582 consensus_queue.cc:1048] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:48.623579  3996 raft_consensus.cc:2955] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:48.853341  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 74928.9us
W20260709 12:23:48.901499  4617 log.cc:927] Time spent T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
W20260709 12:23:49.022275  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 112415us
W20260709 12:23:48.993276  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 159292us
W20260709 12:23:49.086066  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 118400us
I20260709 12:23:49.098019  4549 raft_consensus.cc:1064] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
W20260709 12:23:49.163542  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 94662us
I20260709 12:23:49.203020  3394 raft_consensus.cc:2955] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:49.231159  3244 consensus_queue.cc:237] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:23:49.254103  3364 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.268694  4549 consensus_queue.cc:237] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:49.201762  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:23:49.336696  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 88458.1us
W20260709 12:23:49.366473  3361 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.000s	sys 0.057s
W20260709 12:23:49.366931  3361 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.000s	sys 0.057s
I20260709 12:23:49.388676  4625 raft_consensus.cc:1064] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:49.389128  3998 raft_consensus.cc:1275] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:49.393289  4625 consensus_queue.cc:237] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:49.419157  3145 catalog_manager.cc:5697] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:49.431139  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 66110.1us
I20260709 12:23:49.431842  4655 consensus_queue.cc:1048] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:49.353400  3141 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54338 (request call id 240) took 1076 ms. Trace:
I20260709 12:23:49.434784  3141 rpcz_store.cc:276] 0709 12:23:48.277245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.277821 (+   576us) service_pool.cc:224] Handling call
0709 12:23:48.320009 (+ 42188us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b5400002210. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5f0b2 00007f39d8ad8759 00007f39d8a70a76 00007f39d8a41d58 00007f39d8b4004e 00007f39d70012a4 00007f39d7001232 00007f39d7001161 00007f39d70010dc
0709 12:23:49.353132 (+1033123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":354,"reported_tablets":4,"spinlock_wait_cycles":36726656}
I20260709 12:23:49.453358  3244 raft_consensus.cc:1275] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:49.455169  4632 consensus_queue.cc:1048] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:49.470814  3998 raft_consensus.cc:1275] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:49.478645  4632 consensus_queue.cc:1048] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:23:49.494592  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 72762.5us
I20260709 12:23:49.499806  3317 raft_consensus.cc:1275] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:49.505249  4648 raft_consensus.cc:2955] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:49.521250  4662 log.cc:927] Time spent T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:23:49.542263  3998 raft_consensus.cc:2955] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:49.560737  4632 raft_consensus.cc:2955] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:49.580140  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 60696.4us
I20260709 12:23:49.597090  4642 consensus_queue.cc:1048] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:23:49.613261  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 285328us
I20260709 12:23:49.615234  3992 raft_consensus.cc:2955] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:49.617460  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3098f015fb154cf490e147a8e71f65b6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:49.626787  3243 raft_consensus.cc:2955] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:49.637283  3998 raft_consensus.cc:1275] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:49.650229  3243 raft_consensus.cc:1275] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:49.650187  4660 consensus_queue.cc:1048] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:23:49.661008  3320 raft_consensus.cc:2955] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:49.666698  3323 raft_consensus.cc:1275] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:49.670616  3318 raft_consensus.cc:1275] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:49.723512  4660 consensus_queue.cc:1048] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:23:49.758560  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 57482.3us
W20260709 12:23:49.759379  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 54309.6us
I20260709 12:23:49.783242  4625 consensus_queue.cc:1048] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:23:49.788311  4632 consensus_queue.cc:1048] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
W20260709 12:23:49.880784  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 50927.9us
W20260709 12:23:49.882872  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 63291.8us
I20260709 12:23:49.881383  3320 consensus_queue.cc:237] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:49.898180  3145 catalog_manager.cc:5697] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:49.902467  3145 catalog_manager.cc:5697] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:49.904960  4635 raft_consensus.cc:2955] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:49.922480  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 49777.7us
I20260709 12:23:49.931670  3998 raft_consensus.cc:1275] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:49.933759  4636 consensus_queue.cc:1048] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:49.944355  3242 raft_consensus.cc:2955] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:49.962524  3397 raft_consensus.cc:1275] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:49.965276  3323 raft_consensus.cc:2955] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:49.966903  3397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d167141acd1450192e10b3c44347860"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:49.967864  3397 raft_consensus.cc:606] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:49.956961  3396 consensus_queue.cc:237] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:49.973829  3397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34c56ebe809a4c09b75ee18faf980d8c"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:49.974697  3397 raft_consensus.cc:606] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:49.979297  3396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "412da949752f49049ec6148896078f29"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:49.981015  3396 raft_consensus.cc:606] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:49.984346  3397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:49.989019  3998 raft_consensus.cc:2955] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:49.997601  3397 raft_consensus.cc:606] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:50.003546  4671 raft_consensus.cc:2955] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:50.005311  3992 raft_consensus.cc:2955] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:50.017725  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 61922.8us
I20260709 12:23:50.024152  3397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:50.025252  3397 raft_consensus.cc:606] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:50.029206  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:50.031281  3998 raft_consensus.cc:1275] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:50.041289  3242 raft_consensus.cc:1275] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:50.053864  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:50.055251  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:50.055954  3379 tablet_service.cc:1558] Processing DeleteTablet for tablet 3098f015fb154cf490e147a8e71f65b6 with delete_type TABLET_DATA_TOMBSTONED (TS c93989d3dd634ef585d77b03354121a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54706
I20260709 12:23:50.067210  3144 catalog_manager.cc:5697] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:50.078622  4660 consensus_queue.cc:1048] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:50.093504  4660 raft_consensus.cc:993] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:50.094264  4660 raft_consensus.cc:1081] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:50.097445  4635 consensus_queue.cc:1048] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:50.100672  4676 tablet_replica.cc:333] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:23:50.103152  4660 raft_consensus.cc:993] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:50.103878  4660 raft_consensus.cc:1081] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:50.105477  3134 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 14193bb0c64048cf8ad561f477695970 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:50.122701  3242 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36494
I20260709 12:23:50.123492  3242 raft_consensus.cc:493] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.123895  3242 raft_consensus.cc:3060] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.126977  4625 raft_consensus.cc:993] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4: : Instructing follower 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:50.127563  4625 raft_consensus.cc:1081] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:50.138305  4671 consensus_queue.cc:1048] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:50.147372  4676 raft_consensus.cc:2243] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:50.148283  4676 raft_consensus.cc:2272] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:50.153602  3998 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34c56ebe809a4c09b75ee18faf980d8c"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:55592
I20260709 12:23:50.154333  3998 raft_consensus.cc:493] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.154676  3998 raft_consensus.cc:3060] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:50.159539  4634 raft_consensus.cc:2955] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:50.163686  3998 raft_consensus.cc:515] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:50.163074  3396 raft_consensus.cc:2955] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:50.167948  4676 ts_tablet_manager.cc:1916] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:50.184695  3140 catalog_manager.cc:5697] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 4 to 5, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:50.194692  3996 raft_consensus.cc:2955] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:50.210335  3320 raft_consensus.cc:2955] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:50.215616  3321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c56ebe809a4c09b75ee18faf980d8c" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:50.216518  3321 raft_consensus.cc:3060] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:50.226024  3323 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "412da949752f49049ec6148896078f29"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:51194
I20260709 12:23:50.226778  3323 raft_consensus.cc:493] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.227190  3323 raft_consensus.cc:3060] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.227161  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c56ebe809a4c09b75ee18faf980d8c" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:50.228227  3397 raft_consensus.cc:3055] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:50.228557  3397 raft_consensus.cc:740] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:50.230381  3998 leader_election.cc:290] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:50.216503  3244 raft_consensus.cc:2955] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:50.234882  3397 consensus_queue.cc:260] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:50.246892  3397 raft_consensus.cc:3060] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:50.247224  3242 raft_consensus.cc:515] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:50.250276  3239 tablet_service.cc:1558] Processing DeleteTablet for tablet 14193bb0c64048cf8ad561f477695970 with delete_type TABLET_DATA_TOMBSTONED (TS 7cdaa0234c64450eb4ce4500b51b8a81 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:53720
I20260709 12:23:50.263520  4423 raft_consensus.cc:993] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4: : Instructing follower 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:50.264138  4423 raft_consensus.cc:1081] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:50.264072  3321 raft_consensus.cc:2468] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 3.
I20260709 12:23:50.266455  3963 leader_election.cc:304] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:23:50.267452  4582 raft_consensus.cc:2804] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:50.267966  4582 raft_consensus.cc:697] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:23:50.268872  4582 consensus_queue.cc:237] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:50.298074  3320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d167141acd1450192e10b3c44347860"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:51194
I20260709 12:23:50.298882  3320 raft_consensus.cc:493] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.299317  3320 raft_consensus.cc:3060] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.307231  3323 raft_consensus.cc:515] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:50.312568  3145 catalog_manager.cc:5697] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:50.323366  3323 leader_election.cc:290] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:50.327356  3242 leader_election.cc:290] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:23:50.328559  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:50.329312  3396 raft_consensus.cc:3055] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:50.329609  3396 raft_consensus.cc:740] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:50.329813  4677 tablet_replica.cc:333] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:23:50.330600  3396 consensus_queue.cc:260] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:50.331226  4677 raft_consensus.cc:2243] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:50.332480  4677 raft_consensus.cc:2272] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:50.334194  3396 raft_consensus.cc:3060] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.335119  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:50.335894  3247 raft_consensus.cc:3060] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.346315  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 00dbb3c83d794b5db75532d598235d9a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:50.350965  3998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:23:50.351672  3998 raft_consensus.cc:3060] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.346952  3394 consensus_queue.cc:237] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:50.365584  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:50.366353  3391 raft_consensus.cc:3055] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:50.366626  3391 raft_consensus.cc:740] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:50.367663  3397 raft_consensus.cc:2468] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 3.
I20260709 12:23:50.367470  3391 consensus_queue.cc:260] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:50.374323  3391 raft_consensus.cc:3060] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.380520  4677 ts_tablet_manager.cc:1916] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:50.384547  3311 tablet_service.cc:1558] Processing DeleteTablet for tablet 00dbb3c83d794b5db75532d598235d9a with delete_type TABLET_DATA_TOMBSTONED (TS 8525f77f43714a58bf158d872c1db3e4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40236
I20260709 12:23:50.384356  3320 raft_consensus.cc:515] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:50.395717  3144 catalog_manager.cc:5697] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 2 to 3, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 3 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:50.367455  3998 raft_consensus.cc:2468] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:50.406260  3320 leader_election.cc:290] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Requested vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:23:50.389495  3391 raft_consensus.cc:2468] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:50.408473  3396 raft_consensus.cc:2468] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:50.375273  3247 raft_consensus.cc:2468] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:50.411775  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d167141acd1450192e10b3c44347860" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:50.412550  3397 raft_consensus.cc:3055] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:50.412906  3397 raft_consensus.cc:740] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:50.410668  3992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d167141acd1450192e10b3c44347860" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:23:50.417512  3992 raft_consensus.cc:3060] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.419915  3213 leader_election.cc:304] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:23:50.422091  4547 raft_consensus.cc:2804] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:50.427235  4547 raft_consensus.cc:697] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:50.433317  3319 raft_consensus.cc:1275] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:50.434151  3996 raft_consensus.cc:1275] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:50.429032  4547 consensus_queue.cc:237] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:50.439183  3288 leader_election.cc:304] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:50.442215  4393 raft_consensus.cc:2804] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:50.442976  4393 raft_consensus.cc:697] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:50.444128  4393 consensus_queue.cc:237] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:50.447966  4532 consensus_queue.cc:1048] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:50.447907  3397 consensus_queue.cc:260] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:50.469611  4634 consensus_queue.cc:1048] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:50.476238  4635 raft_consensus.cc:993] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:50.476854  4635 raft_consensus.cc:1081] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:50.479869  4625 raft_consensus.cc:2955] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:50.492426  3244 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36494
I20260709 12:23:50.493325  3244 raft_consensus.cc:493] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.493892  3244 raft_consensus.cc:3060] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.486328  3319 raft_consensus.cc:2955] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:50.500661  3397 raft_consensus.cc:3060] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.516286  3397 raft_consensus.cc:2468] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:50.527992  3992 raft_consensus.cc:2468] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:50.528142  3289 leader_election.cc:304] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:50.530918  4622 raft_consensus.cc:2804] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:50.531513  4622 raft_consensus.cc:697] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:50.532567  4622 consensus_queue.cc:237] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:50.534257  4679 tablet_replica.cc:333] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:23:50.534582  4677 ts_tablet_manager.cc:1929] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:23:50.535431  4677 log.cc:1199] T 14193bb0c64048cf8ad561f477695970 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/wals/14193bb0c64048cf8ad561f477695970
I20260709 12:23:50.535741  4679 raft_consensus.cc:2243] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:50.536552  4679 raft_consensus.cc:2272] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:50.538661  3132 catalog_manager.cc:5028] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): tablet 14193bb0c64048cf8ad561f477695970 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:50.543871  4679 ts_tablet_manager.cc:1916] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:50.554131  3244 raft_consensus.cc:515] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:50.565205  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:50.565918  3323 raft_consensus.cc:3060] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.572187  3244 leader_election.cc:290] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:50.575227  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
W20260709 12:23:50.576709  4680 log.cc:927] Time spent T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:23:50.583877  3323 raft_consensus.cc:2468] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:50.575919  3391 raft_consensus.cc:3055] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:50.584946  3391 raft_consensus.cc:740] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:50.589530  3144 catalog_manager.cc:5697] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:50.590889  4676 ts_tablet_manager.cc:1929] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:50.591351  4676 log.cc:1199] T 3098f015fb154cf490e147a8e71f65b6 P c93989d3dd634ef585d77b03354121a4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/wals/3098f015fb154cf490e147a8e71f65b6
I20260709 12:23:50.591737  3320 raft_consensus.cc:1240] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Rejecting Update request from peer c93989d3dd634ef585d77b03354121a4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:50.593423  3133 catalog_manager.cc:5028] TS c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): tablet 3098f015fb154cf490e147a8e71f65b6 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:50.598296  3992 raft_consensus.cc:2955] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:50.606071  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet caed93336fbb415c82acd602adce7969 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:50.609361  3391 consensus_queue.cc:260] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:50.611444  4532 consensus_queue.cc:1059] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:50.609779  3215 leader_election.cc:304] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:50.613428  4653 raft_consensus.cc:2804] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:50.613991  4653 raft_consensus.cc:697] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:50.626240  4653 consensus_queue.cc:237] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:50.635612  3145 catalog_manager.cc:5697] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:50.647310  3391 raft_consensus.cc:3060] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.672825  3239 tablet_service.cc:1558] Processing DeleteTablet for tablet caed93336fbb415c82acd602adce7969 with delete_type TABLET_DATA_TOMBSTONED (TS 7cdaa0234c64450eb4ce4500b51b8a81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53720
I20260709 12:23:50.673949  4677 tablet_replica.cc:333] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:23:50.675261  4677 raft_consensus.cc:2243] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:50.676187  4677 raft_consensus.cc:2272] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:50.682840  3391 raft_consensus.cc:2468] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:50.684923  4677 ts_tablet_manager.cc:1916] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:50.678750  3140 catalog_manager.cc:5697] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:50.704211  3145 catalog_manager.cc:5697] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:50.726480  4679 ts_tablet_manager.cc:1929] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:50.741315  4679 log.cc:1199] T 00dbb3c83d794b5db75532d598235d9a P 8525f77f43714a58bf158d872c1db3e4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/wals/00dbb3c83d794b5db75532d598235d9a
I20260709 12:23:50.742453  4677 ts_tablet_manager.cc:1929] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:50.743201  4677 log.cc:1199] T caed93336fbb415c82acd602adce7969 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/wals/caed93336fbb415c82acd602adce7969
I20260709 12:23:50.697496  3140 catalog_manager.cc:5697] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:50.746832  3132 catalog_manager.cc:5028] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): tablet caed93336fbb415c82acd602adce7969 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:50.756999  3319 raft_consensus.cc:1275] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:50.778277  3391 raft_consensus.cc:1275] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:50.782092  4582 consensus_queue.cc:1048] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:50.795028  3134 catalog_manager.cc:5028] TS 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): tablet 00dbb3c83d794b5db75532d598235d9a (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:50.796963  4641 consensus_queue.cc:1048] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:50.843473  3391 raft_consensus.cc:1275] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:50.851402  4653 consensus_queue.cc:1048] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:50.883668  3244 raft_consensus.cc:1275] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:50.887055  4622 consensus_queue.cc:1048] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:50.943368  3992 raft_consensus.cc:1275] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:50.947568  4653 consensus_queue.cc:1048] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:50.951421  3397 raft_consensus.cc:1275] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:50.959415  4671 consensus_queue.cc:1048] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:50.994688  3396 raft_consensus.cc:1275] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:50.996587  4622 consensus_queue.cc:1048] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:51.027212  3996 raft_consensus.cc:1275] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:51.030707  4671 consensus_queue.cc:1048] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:51.101217  3323 raft_consensus.cc:1275] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:51.104633  4523 consensus_queue.cc:1048] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:51.160514  3396 raft_consensus.cc:1275] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:51.162932  4648 consensus_queue.cc:1048] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:51.448482  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:51.496470  3247 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b70b620cf8243f4b39cf03eaae44881"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:23:51.497087  3247 raft_consensus.cc:606] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Received request to transfer leadership to TS c93989d3dd634ef585d77b03354121a4
I20260709 12:23:51.500516  3394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c9b725784044c8da903f092febf7c05"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:51.501142  3394 raft_consensus.cc:606] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:51.504279  3247 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:23:51.504822  3247 raft_consensus.cc:606] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:51.508742  3320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:48592
I20260709 12:23:51.509562  3320 raft_consensus.cc:606] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:51.512791  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:23:51.515213  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:51.516202  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:51.805925  3247 consensus_queue.cc:237] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
W20260709 12:23:51.813655  3213 consensus_peers.cc:597] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 054906db16c94967bdef7c9dd5a47ea3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:51.814206  3323 raft_consensus.cc:1275] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:51.814441  3396 raft_consensus.cc:1275] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:51.816110  4547 consensus_queue.cc:1048] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:51.817415  4523 consensus_queue.cc:1048] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:51.831017  4523 raft_consensus.cc:2955] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:51.833148  3323 raft_consensus.cc:2955] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:51.844689  3396 raft_consensus.cc:2955] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:51.869836  3145 catalog_manager.cc:5697] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:51.864856  3247 consensus_queue.cc:237] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
W20260709 12:23:51.906977  3213 consensus_peers.cc:597] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a9cc3d1278674852b3b6c8c39f4f1b59. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:51.910799  3396 raft_consensus.cc:1275] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:51.914283  4648 consensus_queue.cc:1048] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:51.923727  3323 raft_consensus.cc:1275] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:51.927310  4523 consensus_queue.cc:1048] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 12:23:51.972672  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 67191.9us
I20260709 12:23:51.990864  4537 raft_consensus.cc:993] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:51.992136  4671 raft_consensus.cc:1081] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:51.992040  4523 raft_consensus.cc:2955] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:51.996090  3391 raft_consensus.cc:2955] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:51.999212  4547 raft_consensus.cc:993] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:23:51.999843  4648 raft_consensus.cc:1081] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Signalling peer c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:23:52.006336  3396 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b70b620cf8243f4b39cf03eaae44881"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:41792
I20260709 12:23:52.007241  3396 raft_consensus.cc:493] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:52.007885  3244 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36508
I20260709 12:23:52.008559  3244 raft_consensus.cc:493] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:52.008914  3244 raft_consensus.cc:3060] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.010614  3322 raft_consensus.cc:2955] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:52.007694  3396 raft_consensus.cc:3060] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.019364  3244 raft_consensus.cc:515] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:52.022578  3244 leader_election.cc:290] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:52.029223  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:52.029908  3397 raft_consensus.cc:3060] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.034055  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:52.034747  3323 raft_consensus.cc:3055] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:52.035053  3323 raft_consensus.cc:740] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:52.039836  3396 raft_consensus.cc:515] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:52.043155  3323 consensus_queue.cc:260] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:52.044056  3396 leader_election.cc:290] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:52.047706  3323 raft_consensus.cc:3060] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.048780  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b70b620cf8243f4b39cf03eaae44881" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:52.049367  3322 raft_consensus.cc:3060] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.053498  3397 raft_consensus.cc:2468] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:52.057049  3242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b70b620cf8243f4b39cf03eaae44881" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:52.057847  3242 raft_consensus.cc:3055] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:52.058161  3242 raft_consensus.cc:740] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:52.058951  3242 consensus_queue.cc:260] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:52.069309  3242 raft_consensus.cc:3060] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.069430  3214 leader_election.cc:304] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:52.073446  3322 raft_consensus.cc:2468] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 2.
I20260709 12:23:52.077096  3364 leader_election.cc:304] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:52.087019  4625 raft_consensus.cc:993] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4: : Instructing follower 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:52.087034  4713 raft_consensus.cc:2804] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:52.087630  4625 raft_consensus.cc:1081] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:52.088030  4713 raft_consensus.cc:697] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:52.088798  4625 raft_consensus.cc:2804] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:52.085229  4547 raft_consensus.cc:993] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:52.093873  4547 raft_consensus.cc:1081] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:52.089402  4625 raft_consensus.cc:697] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:52.089576  4713 consensus_queue.cc:237] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:52.096444  4625 consensus_queue.cc:237] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:52.105492  3322 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c9b725784044c8da903f092febf7c05"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:51194
I20260709 12:23:52.106562  3322 raft_consensus.cc:493] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:52.109304  3322 raft_consensus.cc:3060] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:52.110103  3213 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 65350.6us
I20260709 12:23:52.118934  3242 raft_consensus.cc:2468] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 2.
I20260709 12:23:52.121932  3323 raft_consensus.cc:2468] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:52.127664  3996 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:55596
I20260709 12:23:52.128436  3996 raft_consensus.cc:493] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:52.128872  3996 raft_consensus.cc:3060] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:52.125424  3322 raft_consensus.cc:515] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:52.144862  3322 leader_election.cc:290] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:52.146677  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9b725784044c8da903f092febf7c05" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:52.147290  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9b725784044c8da903f092febf7c05" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:52.148061  3397 raft_consensus.cc:3055] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:52.148409  3397 raft_consensus.cc:740] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:52.147523  3247 raft_consensus.cc:3060] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.149300  3397 consensus_queue.cc:260] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:52.152974  3996 raft_consensus.cc:515] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:52.156111  3397 raft_consensus.cc:3060] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.154626  3144 catalog_manager.cc:5697] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:52.158121  3144 catalog_manager.cc:5697] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 2 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:52.159523  3244 consensus_queue.cc:237] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:52.180459  3247 raft_consensus.cc:2468] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:52.180228  3397 raft_consensus.cc:2468] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:52.181283  3242 raft_consensus.cc:1240] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Rejecting Update request from peer c93989d3dd634ef585d77b03354121a4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:52.183552  3289 leader_election.cc:304] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:52.186033  4704 raft_consensus.cc:2804] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:52.186971  4704 raft_consensus.cc:697] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:52.188199  4704 consensus_queue.cc:237] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:52.195922  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:52.196564  3249 raft_consensus.cc:3055] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:52.196938  3249 raft_consensus.cc:740] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:52.197268  3996 leader_election.cc:290] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:52.197333  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:52.198037  3249 consensus_queue.cc:260] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:52.199355  3249 raft_consensus.cc:3060] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:52.198114  3323 raft_consensus.cc:3060] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:52.210592  3323 raft_consensus.cc:2468] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 3.
I20260709 12:23:52.212272  3963 leader_election.cc:304] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:23:52.213312  4708 raft_consensus.cc:2804] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:52.213927  4708 raft_consensus.cc:697] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:23:52.214872  4708 consensus_queue.cc:237] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:52.215883  3397 raft_consensus.cc:1275] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:52.232103  4713 consensus_queue.cc:1048] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:52.223980  3249 raft_consensus.cc:2468] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 3.
I20260709 12:23:52.226680  3319 raft_consensus.cc:1275] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:23:52.228978  3213 consensus_peers.cc:597] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f85f93dbda424e3aa0102bc19a2ff837. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:52.245748  4713 consensus_queue.cc:1048] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:52.247171  4720 ts_tablet_manager.cc:933] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:52.254298  4720 tablet_copy_client.cc:323] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:23:52.258572  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet 054906db16c94967bdef7c9dd5a47ea3 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:23:52.259203  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet 054906db16c94967bdef7c9dd5a47ea3 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-054906db16c94967bdef7c9dd5a47ea3
I20260709 12:23:52.264771  4713 raft_consensus.cc:2955] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:52.274791  3322 raft_consensus.cc:2955] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:52.267477  3140 catalog_manager.cc:5697] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:52.274641  3145 catalog_manager.cc:5697] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 3 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:52.278517  3397 raft_consensus.cc:2955] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:52.282533  3144 catalog_manager.cc:5697] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:52.297302  3259 tablet_copy_source_session.cc:215] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:52.313223  4720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054906db16c94967bdef7c9dd5a47ea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:52.338752  3322 consensus_queue.cc:237] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:52.340605  3144 catalog_manager.cc:5697] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:52.384243  3391 raft_consensus.cc:1275] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:52.386219  3288 consensus_peers.cc:597] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d639c2cd28f9418682dc27d0b0ba86b2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:52.386554  4671 consensus_queue.cc:1048] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:52.388326  3247 raft_consensus.cc:1275] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:52.392607  4671 consensus_queue.cc:1048] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.400152  4720 tablet_copy_client.cc:806] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:52.401024  4720 tablet_copy_client.cc:670] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:52.410493  4720 tablet_copy_client.cc:538] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:52.442499  4622 raft_consensus.cc:2955] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:52.449961  3247 raft_consensus.cc:2955] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:52.455921  4720 tablet_bootstrap.cc:492] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:52.473773  3391 raft_consensus.cc:2955] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:52.488834  3145 catalog_manager.cc:5697] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:52.525450  4728 ts_tablet_manager.cc:933] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:52.530192  4720 tablet_bootstrap.cc:492] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:52.531378  4720 tablet_bootstrap.cc:492] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:52.532166  4720 ts_tablet_manager.cc:1403] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.077s	user 0.017s	sys 0.015s
I20260709 12:23:52.540243  3247 raft_consensus.cc:1275] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:52.542088  4705 consensus_queue.cc:1048] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.535792  4720 raft_consensus.cc:359] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:52.546808  4720 raft_consensus.cc:740] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:52.547623  4720 consensus_queue.cc:260] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:52.556556  4720 ts_tablet_manager.cc:1434] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.024s	user 0.001s	sys 0.010s
I20260709 12:23:52.558679  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-054906db16c94967bdef7c9dd5a47ea3 received from {username='slave'} at 127.0.0.1:57122
I20260709 12:23:52.559229  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-054906db16c94967bdef7c9dd5a47ea3 on tablet 054906db16c94967bdef7c9dd5a47ea3 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:52.562009  4728 tablet_copy_client.cc:323] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:23:52.564018  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet a9cc3d1278674852b3b6c8c39f4f1b59 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:23:52.564584  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet a9cc3d1278674852b3b6c8c39f4f1b59 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-a9cc3d1278674852b3b6c8c39f4f1b59
I20260709 12:23:52.573524  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:52.582639  3259 tablet_copy_source_session.cc:215] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:52.586691  4728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9cc3d1278674852b3b6c8c39f4f1b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:52.607316  3322 raft_consensus.cc:1275] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:52.613758  4532 consensus_queue.cc:1048] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.615239  3992 raft_consensus.cc:1217] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:23:52.652496  3249 raft_consensus.cc:1275] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:52.654865  4671 consensus_queue.cc:1048] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.662089  4728 tablet_copy_client.cc:806] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:52.665876  4728 tablet_copy_client.cc:670] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:52.669611  3396 raft_consensus.cc:1275] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:52.673815  4704 consensus_queue.cc:1048] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.688164  4728 tablet_copy_client.cc:538] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:52.688295  3319 raft_consensus.cc:1275] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:52.693527  4648 consensus_queue.cc:1048] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.714385  4728 tablet_bootstrap.cc:492] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:52.728003  3391 raft_consensus.cc:1275] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:52.734131  4648 consensus_queue.cc:1048] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.751137  3322 raft_consensus.cc:1275] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:52.753801  4641 consensus_queue.cc:1048] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:52.809062  3249 raft_consensus.cc:1275] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:52.812745  4582 consensus_queue.cc:1048] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:52.819232  4720 ts_tablet_manager.cc:933] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:52.853848  4720 tablet_copy_client.cc:323] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:23:52.854928  4728 tablet_bootstrap.cc:492] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:52.856426  4728 tablet_bootstrap.cc:492] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:52.857854  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet f85f93dbda424e3aa0102bc19a2ff837 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:23:52.858561  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet f85f93dbda424e3aa0102bc19a2ff837 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-f85f93dbda424e3aa0102bc19a2ff837
I20260709 12:23:52.881318  4728 ts_tablet_manager.cc:1403] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.167s	user 0.028s	sys 0.006s
I20260709 12:23:52.883116  3259 tablet_copy_source_session.cc:215] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:52.886996  4720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f85f93dbda424e3aa0102bc19a2ff837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:52.889290  4728 raft_consensus.cc:359] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:52.896258  4728 raft_consensus.cc:740] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:52.897027  4728 consensus_queue.cc:260] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:52.899645  4728 ts_tablet_manager.cc:1434] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 12:23:52.942723  4720 tablet_copy_client.cc:806] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:52.943421  4720 tablet_copy_client.cc:670] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:52.952188  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-a9cc3d1278674852b3b6c8c39f4f1b59 received from {username='slave'} at 127.0.0.1:57122
I20260709 12:23:52.952852  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-a9cc3d1278674852b3b6c8c39f4f1b59 on tablet a9cc3d1278674852b3b6c8c39f4f1b59 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:52.965904  4720 tablet_copy_client.cc:538] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:52.967649  3319 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "412da949752f49049ec6148896078f29"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:48592
I20260709 12:23:52.968569  3319 raft_consensus.cc:606] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:52.987152  3396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ba9db97635f4d28bf0f3079edf45a77"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:52.990885  3396 raft_consensus.cc:606] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:53.002640  4720 tablet_bootstrap.cc:492] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:53.014253  3247 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:23:53.015033  3247 raft_consensus.cc:606] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Received request to transfer leadership to TS c93989d3dd634ef585d77b03354121a4
I20260709 12:23:53.016569  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:23:53.018628  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:53.018580  4728 ts_tablet_manager.cc:933] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:53.019730  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:53.025794  4671 raft_consensus.cc:993] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:53.026559  4537 raft_consensus.cc:1081] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:53.030180  4728 tablet_copy_client.cc:323] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.194:33649
I20260709 12:23:53.045987  3249 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "412da949752f49049ec6148896078f29"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36508
I20260709 12:23:53.046679  3249 raft_consensus.cc:493] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:53.047020  3249 raft_consensus.cc:3060] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:53.056502  3334 tablet_copy_service.cc:140] P 8525f77f43714a58bf158d872c1db3e4: Received BeginTabletCopySession request for tablet d639c2cd28f9418682dc27d0b0ba86b2 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:33282)
W20260709 12:23:53.109822  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 49124.3us
I20260709 12:23:53.110540  3334 tablet_copy_service.cc:161] P 8525f77f43714a58bf158d872c1db3e4: Beginning new tablet copy session on tablet d639c2cd28f9418682dc27d0b0ba86b2 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:33282: session id = e38cc90af94c47f8a86be67ad610cbd4-d639c2cd28f9418682dc27d0b0ba86b2
I20260709 12:23:53.150951  4653 raft_consensus.cc:1064] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:53.154213  4653 consensus_queue.cc:237] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:53.157917  4532 raft_consensus.cc:993] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4: : Instructing follower 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:53.164729  4532 raft_consensus.cc:1081] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:53.167165  4720 tablet_bootstrap.cc:492] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:53.169174  3334 tablet_copy_source_session.cc:215] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:53.173040  4728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d639c2cd28f9418682dc27d0b0ba86b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:53.176869  4720 tablet_bootstrap.cc:492] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:53.178068  4720 ts_tablet_manager.cc:1403] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.176s	user 0.026s	sys 0.006s
I20260709 12:23:53.180507  3396 raft_consensus.cc:1275] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:53.182076  4711 consensus_queue.cc:1048] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:53.187873  4720 raft_consensus.cc:359] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:53.188982  4720 raft_consensus.cc:740] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:53.194396  3992 raft_consensus.cc:1275] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:53.197595  4711 consensus_queue.cc:1048] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:53.190456  4720 consensus_queue.cc:260] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:53.203805  4720 ts_tablet_manager.cc:1434] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 12:23:53.205839  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-f85f93dbda424e3aa0102bc19a2ff837 received from {username='slave'} at 127.0.0.1:57122
I20260709 12:23:53.206336  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-f85f93dbda424e3aa0102bc19a2ff837 on tablet f85f93dbda424e3aa0102bc19a2ff837 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:53.211915  3322 raft_consensus.cc:1275] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:53.214732  3996 raft_consensus.cc:1217] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:23:53.188367  3249 raft_consensus.cc:515] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:53.223839  4728 tablet_copy_client.cc:806] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:53.227264  4728 tablet_copy_client.cc:670] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:53.235816  4728 tablet_copy_client.cc:538] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:53.242684  4653 raft_consensus.cc:2955] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:53.244724  3996 raft_consensus.cc:2955] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:53.246486  3249 leader_election.cc:290] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:53.249344  3396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:53.249991  3396 raft_consensus.cc:3060] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:53.263168  3396 raft_consensus.cc:2468] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 3.
I20260709 12:23:53.264997  3214 leader_election.cc:304] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:53.266172  4648 raft_consensus.cc:2804] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:53.266767  4648 raft_consensus.cc:697] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:53.267724  4648 consensus_queue.cc:237] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:53.277755  3397 raft_consensus.cc:2955] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:53.286764  4653 consensus_queue.cc:1048] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:23:53.290884  4728 tablet_bootstrap.cc:492] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:23:53.307870  3996 raft_consensus.cc:1217] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:53.307974  3322 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ba9db97635f4d28bf0f3079edf45a77"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:51194
I20260709 12:23:53.308734  3322 raft_consensus.cc:493] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:53.309072  3322 raft_consensus.cc:3060] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:53.312532  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:53.313616  3319 raft_consensus.cc:3055] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:53.314114  3319 raft_consensus.cc:740] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:53.306797  3141 catalog_manager.cc:5697] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:53.315388  3319 consensus_queue.cc:260] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:53.336118  3319 raft_consensus.cc:3060] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:53.378286  3319 raft_consensus.cc:2468] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 3.
I20260709 12:23:53.380054  3322 raft_consensus.cc:515] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:53.391438  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 44897.5us
I20260709 12:23:53.398768  3322 leader_election.cc:290] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:23:53.444268  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 95461us
I20260709 12:23:53.455087  3145 catalog_manager.cc:5697] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 2 to 3, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 3 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:53.466261  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba9db97635f4d28bf0f3079edf45a77" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:53.467028  3397 raft_consensus.cc:3055] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:53.467394  3397 raft_consensus.cc:740] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:53.468369  3397 consensus_queue.cc:260] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:53.473029  3397 raft_consensus.cc:3060] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:53.473964  3323 raft_consensus.cc:2955] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:53.486152  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba9db97635f4d28bf0f3079edf45a77" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:53.497136  3397 raft_consensus.cc:2468] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:53.498325  3249 raft_consensus.cc:3060] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:53.516076  3249 raft_consensus.cc:2468] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:53.555217  3289 leader_election.cc:304] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:53.571302  4761 raft_consensus.cc:2804] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:53.571822  4761 raft_consensus.cc:697] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:53.700682  4761 consensus_queue.cc:237] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:53.703779  3962 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 76383.5us
W20260709 12:23:53.942253  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 58785.5us
W20260709 12:23:53.985215  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 198508us
W20260709 12:23:53.996372  4759 log.cc:927] Time spent T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4: Append to log took a long time: real 0.518s	user 0.013s	sys 0.000s
I20260709 12:23:54.008337  3323 raft_consensus.cc:1275] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:54.009516  4758 raft_consensus.cc:993] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:23:54.010466  4758 raft_consensus.cc:1081] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Signalling peer c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:23:54.030447  3391 raft_consensus.cc:1275] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:54.032444  4768 consensus_queue.cc:1048] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:54.039589  4648 raft_consensus.cc:1064] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:54.039574  3391 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:41792
I20260709 12:23:54.040581  3391 raft_consensus.cc:493] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:54.041018  3391 raft_consensus.cc:3060] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:54.043015  4648 consensus_queue.cc:237] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:23:54.057361  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 331027us
I20260709 12:23:54.068580  3391 raft_consensus.cc:515] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:54.072113  3391 leader_election.cc:290] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:23:54.074167  4755 consensus_queue.cc:1048] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:54.078930  4728 tablet_bootstrap.cc:492] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:54.080125  4728 tablet_bootstrap.cc:492] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:23:54.080938  4728 ts_tablet_manager.cc:1403] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.790s	user 0.018s	sys 0.022s
I20260709 12:23:54.084647  4728 raft_consensus.cc:359] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:54.085526  4728 raft_consensus.cc:740] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:23:54.104511  4728 consensus_queue.cc:260] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:54.116801  4782 raft_consensus.cc:1064] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:54.167713  3242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:54.168438  3242 raft_consensus.cc:3055] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:54.168736  3242 raft_consensus.cc:740] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:54.170089  3242 consensus_queue.cc:260] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:54.171280  3242 raft_consensus.cc:3060] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:54.186630  3994 raft_consensus.cc:1275] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:54.165582  4782 consensus_queue.cc:237] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:54.194584  3322 raft_consensus.cc:1275] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:54.200181  3140 catalog_manager.cc:5697] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:54.223248  3397 raft_consensus.cc:1275] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:54.226819  3247 raft_consensus.cc:1275] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:54.230984  4755 consensus_queue.cc:1048] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:54.232038  4786 consensus_queue.cc:1048] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:54.257933  4728 ts_tablet_manager.cc:1434] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.177s	user 0.002s	sys 0.010s
I20260709 12:23:54.263779  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:54.264721  3319 raft_consensus.cc:3060] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:54.271621  3998 raft_consensus.cc:1275] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:54.290596  4713 consensus_queue.cc:1048] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:54.280340  4753 consensus_queue.cc:1048] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 12:23:54.333709  3334 tablet_copy_service.cc:342] P 8525f77f43714a58bf158d872c1db3e4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-d639c2cd28f9418682dc27d0b0ba86b2 received from {username='slave'} at 127.0.0.1:33282
I20260709 12:23:54.334579  3334 tablet_copy_service.cc:434] P 8525f77f43714a58bf158d872c1db3e4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-d639c2cd28f9418682dc27d0b0ba86b2 on tablet d639c2cd28f9418682dc27d0b0ba86b2 with peer e38cc90af94c47f8a86be67ad610cbd4
W20260709 12:23:54.341704  3962 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 59988.1us
W20260709 12:23:54.341704  3961 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 75849.1us
I20260709 12:23:54.343884  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:54.344954  3242 raft_consensus.cc:2468] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
W20260709 12:23:54.350485  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 69049.2us
I20260709 12:23:54.353164  3319 raft_consensus.cc:2468] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
I20260709 12:23:54.357461  4788 consensus_queue.cc:1048] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:54.359479  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 77384.1us
W20260709 12:23:54.361613  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 68130.3us
I20260709 12:23:54.424913  3364 leader_election.cc:304] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
W20260709 12:23:54.362488  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 59064.7us
I20260709 12:23:54.429273  4747 raft_consensus.cc:2804] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:54.464876  3396 raft_consensus.cc:1275] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:54.474424  3393 raft_consensus.cc:1275] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:54.485966  3995 raft_consensus.cc:1217] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:54.460633  4747 raft_consensus.cc:697] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:54.493872  4756 consensus_queue.cc:1048] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 12:23:54.554342  4773 consensus_queue.cc:1048] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:23:54.527283  3321 raft_consensus.cc:1275] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:23:54.634953  3213 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 83432us
I20260709 12:23:54.565112  4747 consensus_queue.cc:237] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:54.648433  4775 raft_consensus.cc:2955] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:54.746337  3243 consensus_queue.cc:237] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:54.761153  3321 raft_consensus.cc:2955] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:54.766698  4786 consensus_queue.cc:1048] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:54.782042  3144 catalog_manager.cc:5697] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:54.783412  3397 raft_consensus.cc:2955] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:54.815491  3992 raft_consensus.cc:2955] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:54.827603  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 49625.9us
W20260709 12:23:54.832998  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 55976.3us
I20260709 12:23:54.857851  3241 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "054906db16c94967bdef7c9dd5a47ea3"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:23:54.866801  3140 catalog_manager.cc:5697] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:54.881510  3241 raft_consensus.cc:606] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Received request to transfer leadership to TS c93989d3dd634ef585d77b03354121a4
I20260709 12:23:54.882225  3241 raft_consensus.cc:630] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Rejecting request to transfer leadership because tablet server c93989d3dd634ef585d77b03354121a4 is not a voter in the active config
I20260709 12:23:54.896965  4782 raft_consensus.cc:2955] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:54.902444  3396 raft_consensus.cc:2955] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:54.928681  4622 raft_consensus.cc:1064] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:23:54.931895  3992 raft_consensus.cc:1275] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:54.932643  4622 consensus_queue.cc:237] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:54.943120  3992 raft_consensus.cc:2955] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:54.951567  4773 consensus_queue.cc:1048] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:54.961910  3243 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:23:54.962622  3243 raft_consensus.cc:606] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Received request to transfer leadership to TS c93989d3dd634ef585d77b03354121a4
I20260709 12:23:54.966624  3316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:48592
I20260709 12:23:54.967371  3316 raft_consensus.cc:606] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:54.971719  3316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d167141acd1450192e10b3c44347860"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
new_leader_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:48592
I20260709 12:23:54.972419  3316 raft_consensus.cc:606] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Received request to transfer leadership to TS c93989d3dd634ef585d77b03354121a4
I20260709 12:23:54.974140  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:23:54.977109  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:54.978207  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:55.002135  4782 raft_consensus.cc:2955] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:55.006651  3992 raft_consensus.cc:2955] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:55.021289  3994 raft_consensus.cc:1275] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:55.021916  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 054906db16c94967bdef7c9dd5a47ea3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:55.025507  3243 raft_consensus.cc:1275] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:55.026494  4756 consensus_queue.cc:1048] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:55.036870  4756 consensus_queue.cc:1048] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:55.039574  4756 raft_consensus.cc:993] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:55.041023  4765 raft_consensus.cc:1081] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:55.043102  3241 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36508
I20260709 12:23:55.043807  3241 raft_consensus.cc:493] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:55.044194  3241 raft_consensus.cc:3060] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:55.046192  3141 catalog_manager.cc:5697] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:23:55.058585  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 43774.7us
I20260709 12:23:55.057505  3241 raft_consensus.cc:515] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:55.079973  3321 raft_consensus.cc:1275] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:55.080329  4756 raft_consensus.cc:993] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4: : Instructing follower c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:23:55.080942  4756 raft_consensus.cc:1081] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Signalling peer c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:23:55.083979  3241 leader_election.cc:290] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:55.093225  3392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:55.094025  3392 raft_consensus.cc:3060] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:55.096170  3396 raft_consensus.cc:1275] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:55.104820  3396 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d167141acd1450192e10b3c44347860"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:41806
I20260709 12:23:55.105479  3396 raft_consensus.cc:493] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:55.106011  3396 raft_consensus.cc:3060] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:55.113543  4787 consensus_queue.cc:1048] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:55.124032  3396 raft_consensus.cc:515] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:55.124406  3140 catalog_manager.cc:5697] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 4 to 5, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:55.143571  3247 consensus_queue.cc:237] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:55.154282  3392 raft_consensus.cc:2468] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:55.167531  3214 leader_election.cc:304] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:55.172214  4787 raft_consensus.cc:2804] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:55.172839  4787 raft_consensus.cc:697] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:55.173906  4787 consensus_queue.cc:237] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:55.188247  3393 raft_consensus.cc:1240] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Rejecting Update request from peer 8525f77f43714a58bf158d872c1db3e4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:55.200321  3396 leader_election.cc:290] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:23:55.204897  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d167141acd1450192e10b3c44347860" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:23:55.205853  3995 raft_consensus.cc:3060] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:55.207438  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:55.208160  3319 raft_consensus.cc:3055] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:55.208540  3319 raft_consensus.cc:740] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:55.211761  3241 consensus_queue.cc:237] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:55.220515  4788 consensus_queue.cc:1048] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:55.199119  3318 raft_consensus.cc:2955] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:55.311151  3995 raft_consensus.cc:2468] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
I20260709 12:23:55.209430  3319 consensus_queue.cc:260] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:23:55.314859  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 90514.4us
W20260709 12:23:55.341260  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 116881us
I20260709 12:23:55.334923  3316 raft_consensus.cc:2955] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:55.367372  3379 tablet_service.cc:1558] Processing DeleteTablet for tablet 054906db16c94967bdef7c9dd5a47ea3 with delete_type TABLET_DATA_TOMBSTONED (TS c93989d3dd634ef585d77b03354121a4 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:54706
W20260709 12:23:55.421108  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 196264us
W20260709 12:23:55.414680  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 68353.3us
I20260709 12:23:55.436910  3319 raft_consensus.cc:3060] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:55.413820  4796 log.cc:927] Time spent T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:23:55.421052  4622 raft_consensus.cc:2955] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:55.481370  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 54280.2us
W20260709 12:23:55.482250  4793 log.cc:927] Time spent T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:23:55.491261  3319 raft_consensus.cc:2468] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
W20260709 12:23:55.503988  4820 log.cc:927] Time spent T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:23:55.530711  3995 raft_consensus.cc:1275] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:55.535015  3995 raft_consensus.cc:1240] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Rejecting Update request from peer 8525f77f43714a58bf158d872c1db3e4 for earlier term 2. Current term is 3. Ops: []
W20260709 12:23:55.537050  3961 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 119071us
I20260709 12:23:55.483819  4833 consensus_queue.cc:1059] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:23:55.545472  3362 leader_election.cc:304] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:23:55.567159  3396 raft_consensus.cc:1275] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:55.578202  3392 raft_consensus.cc:1275] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:55.596493  3241 raft_consensus.cc:1275] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:55.576314  4772 raft_consensus.cc:2804] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:23:55.600018  3213 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.607687  3994 raft_consensus.cc:1275] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:55.609398  4786 consensus_queue.cc:1048] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:55.619585  4648 consensus_queue.cc:1048] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:55.629717  4622 raft_consensus.cc:3055] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:55.630141  4622 raft_consensus.cc:740] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:55.573220  4782 consensus_queue.cc:1048] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 12:23:55.630863  4622 consensus_queue.cc:260] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:55.631906  4622 raft_consensus.cc:3060] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:55.634222  4837 consensus_queue.cc:1048] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:55.631170  4772 raft_consensus.cc:697] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:55.638022  3392 raft_consensus.cc:1275] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:23:55.657145  3288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.640363  4783 consensus_queue.cc:1048] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:23:55.690371  4772 consensus_queue.cc:237] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:23:55.702576  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 217106us
I20260709 12:23:55.695683  3145 catalog_manager.cc:5697] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:55.721835  3995 raft_consensus.cc:2955] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:55.723645  4836 raft_consensus.cc:2955] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:55.713851  4786 raft_consensus.cc:2955] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:55.740346  3288 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.2.4.254:39037, user_credentials={real_user=slave}} blocked reactor thread for 76445.1us
I20260709 12:23:55.736086  3994 raft_consensus.cc:2955] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:55.750331  3992 raft_consensus.cc:2955] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:55.746027  4839 tablet_replica.cc:333] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:23:55.770493  3247 raft_consensus.cc:2955] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:55.784948  3317 raft_consensus.cc:1275] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:55.797392  4839 raft_consensus.cc:2243] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:55.798547  4839 raft_consensus.cc:2272] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:55.812676  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d167141acd1450192e10b3c44347860" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
W20260709 12:23:55.837334  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 227181us
I20260709 12:23:55.845350  3319 raft_consensus.cc:2468] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
I20260709 12:23:55.859937  4783 consensus_queue.cc:1048] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:55.850396  4747 consensus_queue.cc:1048] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:23:55.879323  4839 ts_tablet_manager.cc:1916] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:55.861601  4783 raft_consensus.cc:993] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:23:55.938155  4783 raft_consensus.cc:1081] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Signalling peer c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:23:55.946223  4756 raft_consensus.cc:493] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.946835  4756 raft_consensus.cc:515] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:55.949142  4756 leader_election.cc:290] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:55.953775  3393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:41792
I20260709 12:23:55.954434  3393 raft_consensus.cc:493] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:55.954775  3393 raft_consensus.cc:3060] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:23:55.962330  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 111936us
I20260709 12:23:55.978511  3396 raft_consensus.cc:2955] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:55.985971  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:55.986248  3395 raft_consensus.cc:2955] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:55.999794  3391 raft_consensus.cc:2955] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:55.989686  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 95697.9us
I20260709 12:23:56.063644  3393 raft_consensus.cc:515] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:56.069141  3393 leader_election.cc:290] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:23:56.226034  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 223824us
W20260709 12:23:56.227999  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 221102us
I20260709 12:23:56.230366  4786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:53720 (request call id 82) took 1089 ms. Trace:
I20260709 12:23:56.230845  4786 rpcz_store.cc:276] 0709 12:23:55.141220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.141366 (+   146us) service_pool.cc:224] Handling call
0709 12:23:55.196165 (+ 54799us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b2c000564a8. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158ba0a 00007f39d158b693 00007f39d158b631 00007f39d2cbe3f1 00007f39d2bf79fe 00007f39d2c7e749 00007f39d2c7f76a 00007f39d2c7e91b 00007f39d2c8a03c 00007f39d2c896b3
0709 12:23:55.368778 (+172613us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b4c000716c0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d0f4fa20 00007f39d0f60a29 00007f39d0f26644 00007f39d0f4df2d 00007f39d0f4db93 00007f39d2bdc1af 00007f39d2bdbf3e 00007f39d2c61888 00007f39d2c7e75c 00007f39d2c7f76a 00007f39d2c7e91b
0709 12:23:55.369788 (+  1010us) log.cc:844] Serialized 487 byte log entry
0709 12:23:55.598391 (+228603us) spinlock_profiling.cc:243] Waited 42.8 ms on lock 0x7b540019f620. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bfc142 00007f39d2bdceaa 00007f39d2be3d53 00007f39d2be3cd9 00007f39d2be3c69 00007f39d2be3c31 00007f39d2be2dfd 0000000000388634 0000000000388479 00007f39cfb96475
0709 12:23:56.230259 (+631868us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6296,"raft.queue_time_us":10092,"raft.run_cpu_time_us":6875,"raft.run_wall_time_us":327113,"spinlock_wait_cycles":133361152}
I20260709 12:23:56.244087  3318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:56.244895  3318 raft_consensus.cc:3060] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:56.249346  3322 raft_consensus.cc:1275] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:56.250372  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a9cc3d1278674852b3b6c8c39f4f1b59 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:56.262296  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f85f93dbda424e3aa0102bc19a2ff837 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:56.265967  4788 raft_consensus.cc:493] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:23:56.266747  4788 raft_consensus.cc:515] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:56.272075  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:56.272784  3249 raft_consensus.cc:3055] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:56.273078  3249 raft_consensus.cc:740] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:56.280275  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "686eafc8b6bb49fbb5fd3dd0b68a3099" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:56.285168  3249 consensus_queue.cc:260] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:56.292146  3318 raft_consensus.cc:2468] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
I20260709 12:23:56.309966  3992 raft_consensus.cc:1275] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:56.311738  3288 leader_election.cc:304] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:23:56.323709  3364 leader_election.cc:304] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:56.324792  4826 raft_consensus.cc:2749] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:56.310565  4838 consensus_queue.cc:1048] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:23:56.360704  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:56.351866  4830 raft_consensus.cc:2804] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:56.383934  3992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:23:56.383301  4788 leader_election.cc:290] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
W20260709 12:23:56.394054  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 80646.8us
I20260709 12:23:56.379426  4830 raft_consensus.cc:697] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
W20260709 12:23:56.413504  4865 log.cc:927] Time spent T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
W20260709 12:23:56.448086  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 124491us
W20260709 12:23:56.453713  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 138348us
I20260709 12:23:56.447075  4747 consensus_queue.cc:1048] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:23:56.495857  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:56.510114  3288 leader_election.cc:304] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:56.528064  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:56.543668  3249 raft_consensus.cc:3060] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:56.454052  4830 consensus_queue.cc:237] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:56.548316  3318 raft_consensus.cc:1275] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:23:56.553457  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 43510.6us
I20260709 12:23:56.556233  4788 raft_consensus.cc:2749] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:23:56.615597  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 97821.2us
I20260709 12:23:56.618006  3249 raft_consensus.cc:2468] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
I20260709 12:23:56.692932  3140 catalog_manager.cc:5697] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:56.707849  3144 catalog_manager.cc:5697] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 2 to 3, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:56.722692  4850 raft_consensus.cc:493] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:23:56.724176  4839 ts_tablet_manager.cc:1929] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:23:56.724984  4839 log.cc:1199] T 054906db16c94967bdef7c9dd5a47ea3 P c93989d3dd634ef585d77b03354121a4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/wals/054906db16c94967bdef7c9dd5a47ea3
I20260709 12:23:56.717219  3141 catalog_manager.cc:5697] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 4 to 5, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:56.750460  3145 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54334 (request call id 279) took 1202 ms. Trace:
I20260709 12:23:56.724467  4850 raft_consensus.cc:515] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:56.771781  4850 leader_election.cc:290] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:23:56.774513  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9cc3d1278674852b3b6c8c39f4f1b59" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:23:56.780194  3994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9cc3d1278674852b3b6c8c39f4f1b59" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:23:56.794740  3288 leader_election.cc:304] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:56.796665  4829 raft_consensus.cc:2749] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:56.786819  4839 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54706 (request call id 80) took 1532 ms. Trace:
I20260709 12:23:56.751202  3145 rpcz_store.cc:276] 0709 12:23:55.548410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.549022 (+   612us) service_pool.cc:224] Handling call
0709 12:23:55.665126 (+116104us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b5400382554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bed090 00007f39d2c8bf3a 00007f39d8a41e9c 00007f39d8b4004e 00007f39d70012a4 00007f39d7001232 00007f39d7001161 00007f39d70010dc 00007f39d7000392 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:23:56.750351 (+1085225us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:55.883138 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:55.883814 (+   676us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:55.883859 (+    45us) write_op.cc:435] Acquired schema lock
0709 12:23:55.883880 (+    21us) tablet.cc:662] Decoding operations
0709 12:23:55.884320 (+   440us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:55.884406 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:55.884446 (+    40us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:23:55.884683 (+   237us) tablet.cc:701] Row locks acquired
0709 12:23:55.884706 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:23:55.884865 (+   159us) write_op.cc:270] Start()
0709 12:23:55.884994 (+   129us) write_op.cc:276] Timestamp: P: 1783599835884814 usec, L: 0
0709 12:23:55.885014 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:23:55.926449 (+ 41435us) log.cc:844] Serialized 535 byte log entry
0709 12:23:56.265777 (+339328us) op_driver.cc:464] REPLICATION: finished
0709 12:23:56.463324 (+197547us) write_op.cc:301] APPLY: starting
0709 12:23:56.478673 (+ 15349us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:56.486911 (+  8238us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:56.505393 (+ 18482us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:56.540112 (+ 34719us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:56.558467 (+ 18355us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:23:56.562289 (+  3822us) write_op.cc:312] APPLY: finished
0709 12:23:56.573115 (+ 10826us) log.cc:844] Serialized 23 byte log entry
0709 12:23:56.587124 (+ 14009us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:56.592815 (+  5691us) write_op.cc:454] Released schema lock
0709 12:23:56.595825 (+  3010us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:56.616336 (+ 20511us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:56.619457 (+  3121us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"spinlock_wait_cycles":42356992,"child_traces":[["op",{"apply.queue_time_us":175161,"apply.run_cpu_time_us":3107,"apply.run_wall_time_us":153714,"num_ops":1,"prepare.queue_time_us":141104,"prepare.run_cpu_time_us":3628,"prepare.run_wall_time_us":46301,"replication_time_us":380574,"thread_start_us":226701,"threads_started":2}]]}
I20260709 12:23:56.800518  4839 rpcz_store.cc:276] 0709 12:23:55.254712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.319227 (+ 64515us) service_pool.cc:224] Handling call
0709 12:23:55.715081 (+395854us) ts_tablet_manager.cc:1151] Deleting tablet 054906db16c94967bdef7c9dd5a47ea3
0709 12:23:55.719075 (+  3994us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:23:56.200941 (+481866us) tablet_metadata.cc:665] Metadata flushed
0709 12:23:56.723613 (+522672us) tablet_metadata.cc:665] Metadata flushed
0709 12:23:56.786684 (+ 63071us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":276284,"thread_start_us":125613,"threads_started":1}
I20260709 12:23:56.822605  3133 catalog_manager.cc:5028] TS c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): tablet 054906db16c94967bdef7c9dd5a47ea3 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
W20260709 12:23:56.826077  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 269152us
I20260709 12:23:56.847606  3318 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:48592
I20260709 12:23:56.848364  3318 raft_consensus.cc:606] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:56.830776  4837 consensus_queue.cc:1048] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:23:56.860138  3249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9cc3d1278674852b3b6c8c39f4f1b59"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:23:56.860867  3249 raft_consensus.cc:606] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:23:56.876327  3141 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58104 (request call id 171) took 1096 ms. Trace:
I20260709 12:23:56.876623  3397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:56.877427  3397 raft_consensus.cc:606] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:56.882117  3311 tablet_service.cc:1558] Processing DeleteTablet for tablet f85f93dbda424e3aa0102bc19a2ff837 with delete_type TABLET_DATA_TOMBSTONED (TS 8525f77f43714a58bf158d872c1db3e4 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:40236
I20260709 12:23:56.885739  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:23:56.887818  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:56.898855  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:56.925143  3140 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54338 (request call id 276) took 1165 ms. Trace:
I20260709 12:23:56.876770  3141 rpcz_store.cc:276] 0709 12:23:55.779837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.779991 (+   154us) service_pool.cc:224] Handling call
0709 12:23:56.876263 (+1096272us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:56.843344 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:56.864967 (+ 21623us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:56.865008 (+    41us) write_op.cc:435] Acquired schema lock
0709 12:23:56.865026 (+    18us) tablet.cc:662] Decoding operations
0709 12:23:56.865382 (+   356us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:56.865504 (+   122us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:56.865543 (+    39us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:23:56.865800 (+   257us) tablet.cc:701] Row locks acquired
0709 12:23:56.865823 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:23:56.865966 (+   143us) write_op.cc:270] Start()
0709 12:23:56.866085 (+   119us) write_op.cc:276] Timestamp: P: 1783599836865916 usec, L: 0
0709 12:23:56.866105 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:23:56.866947 (+   842us) log.cc:844] Serialized 530 byte log entry
0709 12:23:56.869766 (+  2819us) op_driver.cc:464] REPLICATION: finished
0709 12:23:56.870274 (+   508us) write_op.cc:301] APPLY: starting
0709 12:23:56.870360 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:56.870477 (+   117us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:56.870608 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:56.870886 (+   278us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:56.871049 (+   163us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:23:56.871094 (+    45us) write_op.cc:312] APPLY: finished
0709 12:23:56.871299 (+   205us) log.cc:844] Serialized 23 byte log entry
0709 12:23:56.871447 (+   148us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:56.871520 (+    73us) write_op.cc:454] Released schema lock
0709 12:23:56.871559 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:56.871892 (+   333us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:56.871916 (+    24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":42994,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":240,"apply.run_cpu_time_us":1593,"apply.run_wall_time_us":1593,"num_ops":1,"prepare.queue_time_us":91192,"prepare.run_cpu_time_us":3121,"prepare.run_wall_time_us":36500,"replication_time_us":3529,"thread_start_us":90723,"threads_started":1}]]}
I20260709 12:23:56.942404  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9cc3d1278674852b3b6c8c39f4f1b59" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:23:56.976450  3322 consensus_queue.cc:237] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:23:56.995496  3290 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.925575  3140 rpcz_store.cc:276] 0709 12:23:55.759655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.759993 (+   338us) service_pool.cc:224] Handling call
0709 12:23:56.925066 (+1165073us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:56.890131 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:56.890725 (+   594us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:56.890766 (+    41us) write_op.cc:435] Acquired schema lock
0709 12:23:56.890784 (+    18us) tablet.cc:662] Decoding operations
0709 12:23:56.891151 (+   367us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:56.891203 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:56.891230 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:23:56.891413 (+   183us) tablet.cc:701] Row locks acquired
0709 12:23:56.891429 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:23:56.891569 (+   140us) write_op.cc:270] Start()
0709 12:23:56.892580 (+  1011us) write_op.cc:276] Timestamp: P: 1783599836891522 usec, L: 0
0709 12:23:56.892617 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 12:23:56.893317 (+   700us) log.cc:844] Serialized 591 byte log entry
0709 12:23:56.904791 (+ 11474us) op_driver.cc:464] REPLICATION: finished
0709 12:23:56.905292 (+   501us) write_op.cc:301] APPLY: starting
0709 12:23:56.905376 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:56.905492 (+   116us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:56.905609 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:56.905924 (+   315us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:56.906124 (+   200us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:23:56.906172 (+    48us) write_op.cc:312] APPLY: finished
0709 12:23:56.906374 (+   202us) log.cc:844] Serialized 23 byte log entry
0709 12:23:56.906575 (+   201us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:56.906659 (+    84us) write_op.cc:454] Released schema lock
0709 12:23:56.906700 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:56.907356 (+   656us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:56.907427 (+    71us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":273,"apply.run_cpu_time_us":1745,"apply.run_wall_time_us":2005,"num_ops":1,"prepare.queue_time_us":120825,"prepare.run_cpu_time_us":2595,"prepare.run_wall_time_us":3459,"replication_time_us":12014}]]}
I20260709 12:23:57.030963  4889 tablet_replica.cc:333] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:23:57.032083  4889 raft_consensus.cc:2243] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:57.032925  4889 raft_consensus.cc:2272] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:57.036270  4889 ts_tablet_manager.cc:1916] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:57.053503  4771 raft_consensus.cc:993] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4: : Instructing follower 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:57.060065  4870 raft_consensus.cc:1081] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:57.070462  3249 raft_consensus.cc:1275] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:57.077883  3318 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:51194
I20260709 12:23:57.075340  3144 catalog_manager.cc:5697] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:57.078572  3318 raft_consensus.cc:493] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:57.078979  3318 raft_consensus.cc:3060] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.111647  3318 raft_consensus.cc:515] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:57.080313  4850 consensus_queue.cc:1048] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:23:57.121973  3318 leader_election.cc:290] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:57.124128  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:57.127588  3995 raft_consensus.cc:1275] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:57.127669  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:57.129413  3249 raft_consensus.cc:3060] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.139549  4876 consensus_queue.cc:1048] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:57.150296  3249 raft_consensus.cc:2468] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
W20260709 12:23:57.151623  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 52621.7us
I20260709 12:23:57.155406  3288 leader_election.cc:304] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:57.163010  3317 raft_consensus.cc:1275] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:57.172567  4773 raft_consensus.cc:993] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:57.173152  4878 raft_consensus.cc:1081] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:23:57.183436  3992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a9cc3d1278674852b3b6c8c39f4f1b59"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:55596
I20260709 12:23:57.184087  3992 raft_consensus.cc:493] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:57.184424  3992 raft_consensus.cc:3060] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.169018  4822 raft_consensus.cc:2804] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:57.185549  4884 consensus_queue.cc:1048] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:23:57.198172  3397 raft_consensus.cc:3055] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:57.198971  3397 raft_consensus.cc:740] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:57.199831  3397 consensus_queue.cc:260] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:57.201349  3397 raft_consensus.cc:3060] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.202749  4822 raft_consensus.cc:697] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:57.206555  4822 consensus_queue.cc:237] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:57.208986  3311 tablet_service.cc:1558] Processing DeleteTablet for tablet a9cc3d1278674852b3b6c8c39f4f1b59 with delete_type TABLET_DATA_TOMBSTONED (TS 8525f77f43714a58bf158d872c1db3e4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40236
I20260709 12:23:57.239077  3141 catalog_manager.cc:5697] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:57.249936  3397 raft_consensus.cc:2468] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:57.261819  3992 raft_consensus.cc:515] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:57.286672  3244 raft_consensus.cc:1275] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:57.288727  4831 consensus_queue.cc:1048] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:57.326865  3392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9cc3d1278674852b3b6c8c39f4f1b59" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:57.327711  3392 raft_consensus.cc:3060] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.332682  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9cc3d1278674852b3b6c8c39f4f1b59" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:57.333499  3244 raft_consensus.cc:3055] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:57.333961  3244 raft_consensus.cc:740] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:57.334779  3244 consensus_queue.cc:260] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:57.336001  3244 raft_consensus.cc:3060] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.337791  3392 raft_consensus.cc:2468] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:57.346069  3962 leader_election.cc:304] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:23:57.347007  4824 raft_consensus.cc:2804] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:57.353086  4824 raft_consensus.cc:697] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:23:57.336948  4876 raft_consensus.cc:2955] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:57.366241  3992 leader_election.cc:290] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Requested vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:23:57.374553  3244 raft_consensus.cc:2468] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:23:57.377802  4889 ts_tablet_manager.cc:1929] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:23:57.379374  4889 log.cc:1199] T f85f93dbda424e3aa0102bc19a2ff837 P 8525f77f43714a58bf158d872c1db3e4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/wals/f85f93dbda424e3aa0102bc19a2ff837
W20260709 12:23:57.391702  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 85287.2us
I20260709 12:23:57.378242  4824 consensus_queue.cc:237] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:57.485261  4889 tablet_replica.cc:333] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:23:57.489691  3134 catalog_manager.cc:5028] TS 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): tablet f85f93dbda424e3aa0102bc19a2ff837 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:57.502490  4889 raft_consensus.cc:2243] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:57.503448  4889 raft_consensus.cc:2272] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:57.507560  4889 ts_tablet_manager.cc:1916] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:57.514372  3249 raft_consensus.cc:2955] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:23:57.550160  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 63758.2us
I20260709 12:23:57.594233  3140 catalog_manager.cc:5697] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:57.597785  3134 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d639c2cd28f9418682dc27d0b0ba86b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:57.629438  3141 catalog_manager.cc:5697] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:57.647636  4843 raft_consensus.cc:993] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:57.648237  4900 raft_consensus.cc:1081] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:57.650578  3244 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36508
I20260709 12:23:57.651229  3244 raft_consensus.cc:493] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:57.651621  3244 raft_consensus.cc:3060] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.654713  4889 ts_tablet_manager.cc:1929] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:57.655203  4889 log.cc:1199] T a9cc3d1278674852b3b6c8c39f4f1b59 P 8525f77f43714a58bf158d872c1db3e4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/wals/a9cc3d1278674852b3b6c8c39f4f1b59
I20260709 12:23:57.658627  3244 raft_consensus.cc:515] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:57.663333  3134 catalog_manager.cc:5028] TS 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): tablet a9cc3d1278674852b3b6c8c39f4f1b59 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:57.669878  3249 raft_consensus.cc:1275] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:57.684433  4900 consensus_queue.cc:1048] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:57.703956  3244 leader_election.cc:290] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:57.703609  3994 raft_consensus.cc:2955] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:23:57.708691  3141 catalog_manager.cc:5697] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:57.715150  3317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:23:57.716374  3317 raft_consensus.cc:3055] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:57.716704  3317 raft_consensus.cc:740] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:57.717706  3317 consensus_queue.cc:260] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:57.720343  3317 raft_consensus.cc:3060] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.728875  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:57.729612  3395 raft_consensus.cc:3060] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.739493  3398 raft_consensus.cc:1275] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:57.741138  4900 consensus_queue.cc:1048] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:57.751839  3395 raft_consensus.cc:2468] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:57.783082  3317 raft_consensus.cc:2468] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:57.784688  3215 leader_election.cc:304] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:23:57.794601  4878 raft_consensus.cc:2804] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:57.807305  4878 raft_consensus.cc:697] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:57.808350  4878 consensus_queue.cc:237] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:57.824587  3379 tablet_service.cc:1558] Processing DeleteTablet for tablet d639c2cd28f9418682dc27d0b0ba86b2 with delete_type TABLET_DATA_TOMBSTONED (TS c93989d3dd634ef585d77b03354121a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54706
I20260709 12:23:57.856793  4909 tablet_replica.cc:333] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:23:57.858352  4909 raft_consensus.cc:2243] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:57.869756  4909 raft_consensus.cc:2272] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:57.869036  3144 catalog_manager.cc:5697] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:57.877288  4909 ts_tablet_manager.cc:1916] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:57.901472  4909 ts_tablet_manager.cc:1929] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:57.902079  4909 log.cc:1199] T d639c2cd28f9418682dc27d0b0ba86b2 P c93989d3dd634ef585d77b03354121a4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/wals/d639c2cd28f9418682dc27d0b0ba86b2
I20260709 12:23:57.904279  3133 catalog_manager.cc:5028] TS c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): tablet d639c2cd28f9418682dc27d0b0ba86b2 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:23:58.041729  3244 raft_consensus.cc:1275] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:58.043895  4824 consensus_queue.cc:1048] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:58.067585  3398 raft_consensus.cc:1275] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:58.069155  4845 consensus_queue.cc:1048] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:58.250283  3321 raft_consensus.cc:1275] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:58.251943  4880 consensus_queue.cc:1048] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:58.267052  3398 raft_consensus.cc:1275] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:58.268863  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:58.268891  4908 consensus_queue.cc:1048] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:23:58.350749  3244 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "412da949752f49049ec6148896078f29"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:23:58.351442  3244 raft_consensus.cc:606] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 LEADER]: Received request to transfer leadership to TS 8525f77f43714a58bf158d872c1db3e4
W20260709 12:23:58.579861  4020 diagnostics_log.cc:230] Unable to collect stacks to diagnostics log: Incomplete: not collecting stack trace since debugger or strace is attached
I20260709 12:23:59.114069  3398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7494971ced1402da04932982df157fd"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:59.115646  3398 raft_consensus.cc:606] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:23:59.133978  3398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "caed93336fbb415c82acd602adce7969"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:23:59.134835  3398 raft_consensus.cc:606] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 8525f77f43714a58bf158d872c1db3e4
I20260709 12:23:59.144985  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
W20260709 12:23:59.187496  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 187989us
I20260709 12:23:59.224299  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:59.226270  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:23:59.249760  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 48617.2us
I20260709 12:23:59.279316  4778 raft_consensus.cc:993] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:59.280055  4771 raft_consensus.cc:1081] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:23:59.283285  3241 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a7494971ced1402da04932982df157fd"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36494
I20260709 12:23:59.284050  3241 raft_consensus.cc:493] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:59.284572  3241 raft_consensus.cc:3060] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.301402  4921 raft_consensus.cc:993] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:59.302199  4921 raft_consensus.cc:1081] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 LEADER]: Signalling peer 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:59.307837  4771 raft_consensus.cc:993] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4: : Instructing follower 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:59.308543  4771 raft_consensus.cc:1081] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:23:59.345079  3398 consensus_queue.cc:237] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:59.350239  3319 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "caed93336fbb415c82acd602adce7969"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:51194
I20260709 12:23:59.351097  3319 raft_consensus.cc:493] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:59.351528  3319 raft_consensus.cc:3060] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.357471  3241 raft_consensus.cc:515] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:59.364516  3322 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "412da949752f49049ec6148896078f29"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:51186
I20260709 12:23:59.365168  3322 raft_consensus.cc:493] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:59.365556  3322 raft_consensus.cc:3060] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:59.365747  3249 raft_consensus.cc:1275] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:59.367734  3319 raft_consensus.cc:515] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:59.371815  3319 leader_election.cc:290] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Requested vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:23:59.373036  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caed93336fbb415c82acd602adce7969" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:23:59.374114  3322 raft_consensus.cc:515] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:59.373466  3241 leader_election.cc:290] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:23:59.375595  3992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7494971ced1402da04932982df157fd" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:23:59.377815  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caed93336fbb415c82acd602adce7969" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:59.378193  3992 raft_consensus.cc:3060] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.378525  3398 raft_consensus.cc:3055] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:59.378908  3398 raft_consensus.cc:740] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:59.380085  3392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:59.377013  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:23:59.380755  3392 raft_consensus.cc:3060] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:59.380858  3249 raft_consensus.cc:3055] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:59.381233  3321 raft_consensus.cc:1275] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:59.381381  3249 raft_consensus.cc:740] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:59.382267  3249 consensus_queue.cc:260] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:59.383523  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7494971ced1402da04932982df157fd" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:23:59.384074  3397 raft_consensus.cc:3055] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:59.384339  3397 raft_consensus.cc:740] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:23:59.384608  4919 consensus_queue.cc:1048] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:59.385313  3397 consensus_queue.cc:260] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:59.386564  3397 raft_consensus.cc:3060] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.387508  3992 raft_consensus.cc:2468] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
W20260709 12:23:59.389271  3362 consensus_peers.cc:597] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d02f86799bc54984b41ede0f7db226e6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:59.389223  3213 leader_election.cc:304] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:23:59.392127  3249 raft_consensus.cc:3060] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:59.374118  3995 raft_consensus.cc:3060] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.392804  3392 raft_consensus.cc:2468] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 4.
I20260709 12:23:59.393585  4922 raft_consensus.cc:2804] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:59.394258  4922 raft_consensus.cc:697] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:23:59.395635  4922 consensus_queue.cc:237] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:59.379825  3398 consensus_queue.cc:260] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:59.407588  4919 consensus_queue.cc:1048] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:59.411553  3398 raft_consensus.cc:3060] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.412396  3289 leader_election.cc:304] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:23:59.377096  3322 leader_election.cc:290] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 4 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:23:59.418781  4932 raft_consensus.cc:2804] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:59.419451  4932 raft_consensus.cc:697] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 4 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:59.420089  3995 raft_consensus.cc:2468] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:59.421227  4932 consensus_queue.cc:237] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:23:59.424437  3288 leader_election.cc:304] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:23:59.425474  4900 raft_consensus.cc:2804] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:59.425979  4900 raft_consensus.cc:697] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:23:59.427011  4900 consensus_queue.cc:237] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:23:59.430552  4937 raft_consensus.cc:2955] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:59.439196  3397 raft_consensus.cc:2468] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:23:59.459940  3398 raft_consensus.cc:2468] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:23:59.474402  3247 raft_consensus.cc:2955] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:59.463289  3141 catalog_manager.cc:5697] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.484763  3249 raft_consensus.cc:2468] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 4.
I20260709 12:23:59.520164  3319 raft_consensus.cc:2955] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:59.542659  3321 consensus_queue.cc:237] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:59.562403  3144 catalog_manager.cc:5697] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 3 to 4, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 4 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.573868  3140 catalog_manager.cc:5697] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:59.578536  3288 consensus_peers.cc:597] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ba9db97635f4d28bf0f3079edf45a77. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:59.581215  3397 raft_consensus.cc:1275] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:59.584474  3247 raft_consensus.cc:1275] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:59.585234  4924 consensus_queue.cc:1048] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:59.591749  4924 consensus_queue.cc:1048] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:59.624341  3144 catalog_manager.cc:5697] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.635434  4924 raft_consensus.cc:2955] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:59.638887  3247 raft_consensus.cc:2955] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:59.658511  3398 raft_consensus.cc:2955] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:59.671773  3144 catalog_manager.cc:5697] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.675210  3397 consensus_queue.cc:237] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
W20260709 12:23:59.703689  3362 consensus_peers.cc:597] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6b70b620cf8243f4b39cf03eaae44881. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:59.703758  3321 raft_consensus.cc:1275] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:59.705651  4937 consensus_queue.cc:1048] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:59.710215  3247 raft_consensus.cc:1275] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:59.712321  4934 consensus_queue.cc:1048] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:59.717952  4919 raft_consensus.cc:2955] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:59.736701  3321 raft_consensus.cc:2955] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:59.742208  3319 consensus_queue.cc:237] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:23:59.750388  3141 catalog_manager.cc:5697] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 2 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.761950  3249 raft_consensus.cc:2955] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
W20260709 12:23:59.768724  3288 consensus_peers.cc:597] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb11c9e1f3b048769b242da5c7ae7cd7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:59.770651  3244 raft_consensus.cc:1275] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:59.773133  4900 consensus_queue.cc:1048] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:59.793174  4924 raft_consensus.cc:2955] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:59.795836  3241 raft_consensus.cc:2955] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:23:59.821240  3141 catalog_manager.cc:5697] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.837069  3397 raft_consensus.cc:1275] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:59.838665  3398 raft_consensus.cc:1275] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:59.846153  3992 raft_consensus.cc:1275] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:59.847947  4900 consensus_queue.cc:1048] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:59.856480  4932 consensus_queue.cc:1048] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:23:59.858723  3270 debug-util.cc:398] Leaking SignalData structure 0x7b08001892c0 after lost signal to thread 2070
W20260709 12:23:59.859963  3270 debug-util.cc:398] Leaking SignalData structure 0x7b0800159ec0 after lost signal to thread 3181
W20260709 12:23:59.861447  3270 debug-util.cc:398] Leaking SignalData structure 0x7b08002e3ca0 after lost signal to thread 3273
W20260709 12:23:59.862664  3270 debug-util.cc:398] Leaking SignalData structure 0x7b0800049de0 after lost signal to thread 3348
W20260709 12:23:59.863900  3270 debug-util.cc:398] Leaking SignalData structure 0x7b08002cdd00 after lost signal to thread 3422
W20260709 12:23:59.865060  3270 debug-util.cc:398] Leaking SignalData structure 0x7b08000ef060 after lost signal to thread 4023
I20260709 12:23:59.866245  4932 consensus_queue.cc:1048] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:59.878433  3249 raft_consensus.cc:1275] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:59.878433  3392 raft_consensus.cc:1275] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:59.883193  4932 consensus_queue.cc:1048] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:59.884689  4932 consensus_queue.cc:1048] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:59.917871  3992 raft_consensus.cc:1275] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:59.920181  4908 consensus_queue.cc:1048] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:59.931283  3397 raft_consensus.cc:2955] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
W20260709 12:24:00.015053  4952 log.cc:927] Time spent T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:24:00.040218  4965 ts_tablet_manager.cc:933] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:00.173931  4965 tablet_copy_client.cc:323] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:24:00.179219  3392 raft_consensus.cc:1275] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:00.182947  4975 consensus_queue.cc:1048] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:00.213040  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 152106us
I20260709 12:24:00.236899  4977 ts_tablet_manager.cc:933] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:00.286705  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 99193.5us
W20260709 12:24:00.361135  3961 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 101536us
I20260709 12:24:00.362969  4981 ts_tablet_manager.cc:933] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:00.360384  4977 tablet_copy_client.cc:323] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.194:33649
I20260709 12:24:00.390865  3334 tablet_copy_service.cc:140] P 8525f77f43714a58bf158d872c1db3e4: Received BeginTabletCopySession request for tablet 5ba9db97635f4d28bf0f3079edf45a77 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:33282)
I20260709 12:24:00.391821  3334 tablet_copy_service.cc:161] P 8525f77f43714a58bf158d872c1db3e4: Beginning new tablet copy session on tablet 5ba9db97635f4d28bf0f3079edf45a77 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:33282: session id = e38cc90af94c47f8a86be67ad610cbd4-5ba9db97635f4d28bf0f3079edf45a77
I20260709 12:24:00.430325  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet d02f86799bc54984b41ede0f7db226e6 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:24:00.431336  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet d02f86799bc54984b41ede0f7db226e6 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-d02f86799bc54984b41ede0f7db226e6
I20260709 12:24:00.438231  4981 tablet_copy_client.cc:323] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:24:00.413553  3334 tablet_copy_source_session.cc:215] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:00.443662  4977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ba9db97635f4d28bf0f3079edf45a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:00.479996  3408 tablet_copy_source_session.cc:215] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:00.481127  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:00.501027  4965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d02f86799bc54984b41ede0f7db226e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:00.504185  4985 ts_tablet_manager.cc:933] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:00.513109  4985 tablet_copy_client.cc:323] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.194:33649
I20260709 12:24:00.515122  3334 tablet_copy_service.cc:140] P 8525f77f43714a58bf158d872c1db3e4: Received BeginTabletCopySession request for tablet bb11c9e1f3b048769b242da5c7ae7cd7 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:33282)
I20260709 12:24:00.515759  3334 tablet_copy_service.cc:161] P 8525f77f43714a58bf158d872c1db3e4: Beginning new tablet copy session on tablet bb11c9e1f3b048769b242da5c7ae7cd7 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:33282: session id = e38cc90af94c47f8a86be67ad610cbd4-bb11c9e1f3b048769b242da5c7ae7cd7
I20260709 12:24:00.519393  4977 tablet_copy_client.cc:806] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:00.520196  4977 tablet_copy_client.cc:670] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:00.525987  4977 tablet_copy_client.cc:538] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:00.544263  3334 tablet_copy_source_session.cc:215] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:00.547163  4977 tablet_bootstrap.cc:492] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:24:00.527551  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet 6b70b620cf8243f4b39cf03eaae44881 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:24:00.555313  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet 6b70b620cf8243f4b39cf03eaae44881 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-6b70b620cf8243f4b39cf03eaae44881
I20260709 12:24:00.564241  3408 tablet_copy_source_session.cc:215] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:00.568151  4981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b70b620cf8243f4b39cf03eaae44881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:00.569024  4985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb11c9e1f3b048769b242da5c7ae7cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:00.601836  4965 tablet_copy_client.cc:806] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:00.604406  4965 tablet_copy_client.cc:670] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:00.610922  4965 tablet_copy_client.cc:538] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:00.614315  4981 tablet_copy_client.cc:806] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:00.629055  4981 tablet_copy_client.cc:670] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:00.648202  4985 tablet_copy_client.cc:806] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:00.663501  4985 tablet_copy_client.cc:670] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:00.656284  4981 tablet_copy_client.cc:538] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:00.678449  4965 tablet_bootstrap.cc:492] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:24:00.684656  4985 tablet_copy_client.cc:538] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:00.765398  3249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:24:00.766137  3249 raft_consensus.cc:606] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Received request to transfer leadership to TS c93989d3dd634ef585d77b03354121a4
I20260709 12:24:00.767463  4981 tablet_bootstrap.cc:492] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:24:00.790896  4921 raft_consensus.cc:993] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:24:00.791644  4922 raft_consensus.cc:1081] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Signalling peer c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:24:00.794054  3398 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:41792
I20260709 12:24:00.794653  3398 raft_consensus.cc:493] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:00.794987  3398 raft_consensus.cc:3060] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.833739  3317 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c9b725784044c8da903f092febf7c05"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:48592
I20260709 12:24:00.834794  3317 raft_consensus.cc:606] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:00.889478  3322 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:48592
I20260709 12:24:00.910494  3322 raft_consensus.cc:606] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:00.913800  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:24:00.915573  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:00.917088  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:00.919202  4977 tablet_bootstrap.cc:492] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:00.925601  3398 raft_consensus.cc:515] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:00.926059  4977 tablet_bootstrap.cc:492] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:24:00.966768  4977 ts_tablet_manager.cc:1403] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.420s	user 0.031s	sys 0.014s
I20260709 12:24:00.970727  4977 raft_consensus.cc:359] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:00.971805  4977 raft_consensus.cc:740] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:24:00.972644  4977 consensus_queue.cc:260] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:00.991864  4977 ts_tablet_manager.cc:1434] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.025s	user 0.002s	sys 0.005s
I20260709 12:24:00.974984  3398 leader_election.cc:290] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:00.980274  3249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:24:01.002228  3249 raft_consensus.cc:3055] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:01.002555  3249 raft_consensus.cc:740] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:01.003271  3249 consensus_queue.cc:260] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:00.981328  4981 tablet_bootstrap.cc:492] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:00.971264  4965 tablet_bootstrap.cc:492] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:00.988341  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:00.954071  4985 tablet_bootstrap.cc:492] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:24:01.012784  3322 raft_consensus.cc:3060] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:01.013960  3249 raft_consensus.cc:3060] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:01.033332  4981 tablet_bootstrap.cc:492] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:24:01.034415  4965 tablet_bootstrap.cc:492] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:24:01.037911  4981 ts_tablet_manager.cc:1403] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.276s	user 0.013s	sys 0.022s
I20260709 12:24:01.038232  3334 tablet_copy_service.cc:342] P 8525f77f43714a58bf158d872c1db3e4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-5ba9db97635f4d28bf0f3079edf45a77 received from {username='slave'} at 127.0.0.1:33282
I20260709 12:24:01.038856  3334 tablet_copy_service.cc:434] P 8525f77f43714a58bf158d872c1db3e4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-5ba9db97635f4d28bf0f3079edf45a77 on tablet 5ba9db97635f4d28bf0f3079edf45a77 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:01.035202  4965 ts_tablet_manager.cc:1403] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.357s	user 0.030s	sys 0.009s
I20260709 12:24:01.051127  3322 raft_consensus.cc:2468] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 2.
I20260709 12:24:01.058400  3249 raft_consensus.cc:2468] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 2.
I20260709 12:24:01.059926  3362 leader_election.cc:304] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:24:01.064273  4937 raft_consensus.cc:2804] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:01.064816  4937 raft_consensus.cc:697] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:01.079147  4981 raft_consensus.cc:359] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:01.080255  4981 raft_consensus.cc:740] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:24:01.075206  4965 raft_consensus.cc:359] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:01.090888  4965 raft_consensus.cc:740] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:24:01.091863  4965 consensus_queue.cc:260] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:01.065694  4937 consensus_queue.cc:237] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:01.087013  4981 consensus_queue.cc:260] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:01.098096  4981 ts_tablet_manager.cc:1434] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.060s	user 0.004s	sys 0.004s
I20260709 12:24:01.107749  4965 ts_tablet_manager.cc:1434] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.065s	user 0.003s	sys 0.004s
I20260709 12:24:01.111267  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-6b70b620cf8243f4b39cf03eaae44881 received from {username='slave'} at 127.0.0.1:34216
I20260709 12:24:01.129522  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-6b70b620cf8243f4b39cf03eaae44881 on tablet 6b70b620cf8243f4b39cf03eaae44881 with peer e38cc90af94c47f8a86be67ad610cbd4
W20260709 12:24:01.115134  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 54204.2us
I20260709 12:24:01.158339  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-d02f86799bc54984b41ede0f7db226e6 received from {username='slave'} at 127.0.0.1:34216
I20260709 12:24:01.158811  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-d02f86799bc54984b41ede0f7db226e6 on tablet d02f86799bc54984b41ede0f7db226e6 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:01.186546  3144 catalog_manager.cc:5697] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 2 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:01.208498  4985 tablet_bootstrap.cc:492] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:01.216813  4985 tablet_bootstrap.cc:492] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:24:01.217797  4985 ts_tablet_manager.cc:1403] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.264s	user 0.029s	sys 0.016s
I20260709 12:24:01.218222  3994 raft_consensus.cc:1217] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:01.231276  4985 raft_consensus.cc:359] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:01.232611  4985 raft_consensus.cc:740] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:24:01.233745  4985 consensus_queue.cc:260] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:01.240648  4985 ts_tablet_manager.cc:1434] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:24:01.247306  3334 tablet_copy_service.cc:342] P 8525f77f43714a58bf158d872c1db3e4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-bb11c9e1f3b048769b242da5c7ae7cd7 received from {username='slave'} at 127.0.0.1:33282
I20260709 12:24:01.247895  3334 tablet_copy_service.cc:434] P 8525f77f43714a58bf158d872c1db3e4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-bb11c9e1f3b048769b242da5c7ae7cd7 on tablet bb11c9e1f3b048769b242da5c7ae7cd7 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:01.256162  4971 raft_consensus.cc:993] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:01.256755  4971 raft_consensus.cc:1081] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:01.259554  3241 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c9b725784044c8da903f092febf7c05"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36508
I20260709 12:24:01.260241  3241 raft_consensus.cc:493] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:01.260610  3241 raft_consensus.cc:3060] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.265867  3994 raft_consensus.cc:1217] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:24:01.308286  3994 raft_consensus.cc:1217] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:24:01.344930  3241 raft_consensus.cc:515] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:01.393157  3392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9b725784044c8da903f092febf7c05" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:01.393826  3392 raft_consensus.cc:3060] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.394933  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9b725784044c8da903f092febf7c05" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:01.395502  3241 leader_election.cc:290] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:01.396241  3322 raft_consensus.cc:3055] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:01.397140  3322 raft_consensus.cc:740] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:24:01.398227  3322 consensus_queue.cc:260] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:01.399473  3322 raft_consensus.cc:3060] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.409544  3392 raft_consensus.cc:2468] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 3.
I20260709 12:24:01.410717  3397 raft_consensus.cc:1240] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Rejecting Update request from peer 8525f77f43714a58bf158d872c1db3e4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:01.411207  3214 leader_election.cc:304] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:24:01.412230  4922 raft_consensus.cc:2804] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:01.412791  4922 raft_consensus.cc:697] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:01.413805  4922 consensus_queue.cc:237] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:01.417269  4974 raft_consensus.cc:993] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:01.417951  4932 raft_consensus.cc:1081] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:01.419968  3241 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36508
I20260709 12:24:01.420565  3241 raft_consensus.cc:493] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:01.420976  3241 raft_consensus.cc:3060] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.452392  3995 raft_consensus.cc:1217] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:24:01.470181  3322 raft_consensus.cc:2468] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 3.
I20260709 12:24:01.481105  3144 catalog_manager.cc:5697] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 2 to 3, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 3 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:01.505767  3241 raft_consensus.cc:515] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:01.508117  3241 leader_election.cc:290] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:01.518735  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:01.519490  3322 raft_consensus.cc:3055] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:01.519986  3322 raft_consensus.cc:740] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:24:01.520377  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:01.521039  3322 consensus_queue.cc:260] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:01.522558  3322 raft_consensus.cc:3060] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.521085  3397 raft_consensus.cc:3060] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.532526  3322 raft_consensus.cc:2468] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 3.
I20260709 12:24:01.534056  3215 leader_election.cc:304] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:24:01.535212  3241 raft_consensus.cc:1275] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:01.536181  4922 raft_consensus.cc:2804] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:01.536695  4922 raft_consensus.cc:697] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:01.539005  4771 consensus_queue.cc:1048] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:01.544811  4922 consensus_queue.cc:237] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:01.568518  3322 raft_consensus.cc:1275] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:01.570638  4771 consensus_queue.cc:1048] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:01.584477  3397 raft_consensus.cc:2468] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 3.
I20260709 12:24:01.665786  3145 catalog_manager.cc:5697] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 2 to 3, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 3 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:01.866986  4971 raft_consensus.cc:1064] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:24:01.878082  4971 consensus_queue.cc:237] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:01.896073  3317 raft_consensus.cc:1275] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:01.907282  4937 raft_consensus.cc:1064] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:24:01.910645  4937 consensus_queue.cc:237] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:01.924666  4990 consensus_queue.cc:1048] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:01.937566  3995 raft_consensus.cc:1275] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:01.939812  4924 consensus_queue.cc:1048] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:01.943558  4771 raft_consensus.cc:1064] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:24:01.927182  3241 raft_consensus.cc:1275] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:01.929768  3397 raft_consensus.cc:1275] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:01.959515  4974 consensus_queue.cc:1048] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:24:01.979647  4932 consensus_queue.cc:1048] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:01.949572  4771 consensus_queue.cc:237] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:02.034505  3322 raft_consensus.cc:1275] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:02.068931  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:02.107717  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 47199.9us
I20260709 12:24:02.141877  4932 raft_consensus.cc:1064] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
W20260709 12:24:02.154117  5011 log.cc:927] Time spent T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4: Append to log took a long time: real 0.095s	user 0.011s	sys 0.012s
W20260709 12:24:02.154781  5007 log.cc:927] Time spent T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 12:24:02.162081  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 52369.6us
I20260709 12:24:02.167878  3322 raft_consensus.cc:1275] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 12:24:02.175467  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 145074us
I20260709 12:24:02.194599  3247 raft_consensus.cc:1275] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:02.195719  3249 raft_consensus.cc:1275] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:02.197083  3322 raft_consensus.cc:1275] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:02.145290  4932 consensus_queue.cc:237] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:02.204840  3994 raft_consensus.cc:1275] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:02.206331  3998 raft_consensus.cc:1275] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:02.244908  3998 raft_consensus.cc:2955] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:02.233179  5017 raft_consensus.cc:2955] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:02.268544  5022 consensus_queue.cc:1048] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:24:02.269873  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 44619.8us
I20260709 12:24:02.294325  3247 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:24:02.295032  3247 raft_consensus.cc:606] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Received request to transfer leadership to TS 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:02.295521  3994 raft_consensus.cc:1275] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:02.299376  4971 consensus_queue.cc:1048] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:24:02.307050  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 122663us
I20260709 12:24:02.312160  3249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:24:02.313805  3249 raft_consensus.cc:606] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Received request to transfer leadership to TS c93989d3dd634ef585d77b03354121a4
W20260709 12:24:02.320798  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 96354.8us
I20260709 12:24:02.322924  5018 consensus_queue.cc:1048] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:02.323731  5003 consensus_queue.cc:1048] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:02.331916  4771 consensus_queue.cc:1048] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:02.334908  3392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:24:02.335654  3392 raft_consensus.cc:606] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:02.336086  5018 consensus_queue.cc:1048] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:02.337994  3397 raft_consensus.cc:1275] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:02.337368  3398 raft_consensus.cc:1275] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:02.340193  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:24:02.342398  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:02.343326  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:02.366667  3247 raft_consensus.cc:1275] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:02.375545  3144 catalog_manager.cc:5697] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:02.379258  5020 consensus_queue.cc:1048] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:02.381573  5023 consensus_queue.cc:1048] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:24:02.386204  5019 consensus_queue.cc:1048] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:02.389045  3392 raft_consensus.cc:2955] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:02.410328  3398 raft_consensus.cc:1275] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 12:24:02.438324  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 56454.6us
I20260709 12:24:02.455505  5027 consensus_queue.cc:1048] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:02.477787  4932 consensus_queue.cc:1048] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:02.501591  3242 raft_consensus.cc:2955] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:24:02.538394  5011 log.cc:927] Time spent T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4: Append to log took a long time: real 0.072s	user 0.006s	sys 0.006s
W20260709 12:24:02.541924  5009 log.cc:927] Time spent T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4: Append to log took a long time: real 0.053s	user 0.002s	sys 0.010s
W20260709 12:24:02.561959  4994 log.cc:927] Time spent T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392: Append to log took a long time: real 0.110s	user 0.002s	sys 0.013s
I20260709 12:24:02.558063  4932 consensus_queue.cc:1048] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
W20260709 12:24:02.593255  5033 log.cc:927] Time spent T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 12:24:02.594938  5034 log.cc:927] Time spent T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:24:02.627519  4922 raft_consensus.cc:993] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower 8525f77f43714a58bf158d872c1db3e4 to start an election
W20260709 12:24:02.654902  5030 log.cc:927] Time spent T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4: Append to log took a long time: real 0.055s	user 0.004s	sys 0.000s
W20260709 12:24:02.683456  5032 log.cc:927] Time spent T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4: Append to log took a long time: real 0.077s	user 0.000s	sys 0.004s
I20260709 12:24:02.741060  5048 raft_consensus.cc:1081] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Signalling peer 8525f77f43714a58bf158d872c1db3e4 to start an election
W20260709 12:24:02.744659  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 167867us
W20260709 12:24:02.744910  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 168499us
W20260709 12:24:02.816339  5038 log.cc:927] Time spent T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 12:24:02.827642  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 79876.7us
I20260709 12:24:02.840560  5039 raft_consensus.cc:2955] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:02.845906  5047 raft_consensus.cc:993] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:24:02.892607  5054 raft_consensus.cc:1081] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Signalling peer c93989d3dd634ef585d77b03354121a4 to start an election
W20260709 12:24:02.894605  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 60329.1us
W20260709 12:24:02.888675  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 255103us
W20260709 12:24:02.949457  5044 log.cc:927] Time spent T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
W20260709 12:24:02.995424  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 64233.1us
I20260709 12:24:03.018293  3319 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:51186
W20260709 12:24:03.022238  3961 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 115245us
I20260709 12:24:03.056854  3321 raft_consensus.cc:2955] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:03.062111  3241 raft_consensus.cc:2955] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:03.038062  5050 raft_consensus.cc:2955] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:03.157852  3392 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:41792
I20260709 12:24:03.158767  3392 raft_consensus.cc:493] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:03.159169  3392 raft_consensus.cc:3060] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:03.174693  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 71790.5us
I20260709 12:24:03.185274  3248 raft_consensus.cc:2955] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:03.229246  5055 raft_consensus.cc:993] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
W20260709 12:24:03.242007  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 141949us
I20260709 12:24:03.246604  3392 raft_consensus.cc:515] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:03.254810  3392 leader_election.cc:290] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:03.245026  5055 raft_consensus.cc:1081] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:03.255749  5072 raft_consensus.cc:2955] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:03.278894  3319 raft_consensus.cc:493] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:03.279409  3319 raft_consensus.cc:3060] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:03.303355  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 45778.1us
I20260709 12:24:03.314853  3319 raft_consensus.cc:515] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:03.319831  3319 leader_election.cc:290] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:24:03.354096  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 99436.7us
I20260709 12:24:03.365761  3320 raft_consensus.cc:1240] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Rejecting Update request from peer 7cdaa0234c64450eb4ce4500b51b8a81 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:03.371778  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:24:03.372644  3248 raft_consensus.cc:3055] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:03.373049  3248 raft_consensus.cc:740] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:03.399948  3994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:24:03.400738  3994 raft_consensus.cc:3060] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.395936  3992 raft_consensus.cc:2955] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:03.406598  3995 raft_consensus.cc:2955] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:03.408527  3144 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58104 (request call id 191) took 1037 ms. Trace:
I20260709 12:24:03.434540  3319 raft_consensus.cc:2955] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:03.430909  5023 consensus_queue.cc:1059] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
W20260709 12:24:03.505131  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 96233.3us
I20260709 12:24:03.521267  3393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:03.525676  3393 raft_consensus.cc:3060] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.545400  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:24:03.546396  3247 raft_consensus.cc:3055] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:03.546809  3247 raft_consensus.cc:740] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:03.555810  3241 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36494
I20260709 12:24:03.556700  3241 raft_consensus.cc:493] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:03.557190  3241 raft_consensus.cc:3060] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:03.559687  5083 log.cc:927] Time spent T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:24:03.553459  3247 consensus_queue.cc:260] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:03.563239  3247 raft_consensus.cc:3060] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.566934  3242 raft_consensus.cc:2955] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:03.569425  3994 raft_consensus.cc:2468] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
I20260709 12:24:03.421294  3144 rpcz_store.cc:276] 0709 12:24:02.360735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.361744 (+  1009us) service_pool.cc:224] Handling call
0709 12:24:03.398181 (+1036437us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:02.439893 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:02.440567 (+   674us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:02.440606 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:24:02.440626 (+    20us) tablet.cc:662] Decoding operations
0709 12:24:02.441012 (+   386us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:02.441094 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:02.441125 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:02.441335 (+   210us) tablet.cc:701] Row locks acquired
0709 12:24:02.441355 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:24:02.441520 (+   165us) write_op.cc:270] Start()
0709 12:24:02.444123 (+  2603us) write_op.cc:276] Timestamp: P: 1783599842441469 usec, L: 0
0709 12:24:02.444151 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:24:02.445917 (+  1766us) log.cc:844] Serialized 591 byte log entry
0709 12:24:02.891751 (+445834us) op_driver.cc:464] REPLICATION: finished
0709 12:24:03.181246 (+289495us) spinlock_profiling.cc:243] Waited 8.39 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39cfb790ac 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d66b9ef0
0709 12:24:03.341795 (+160549us) write_op.cc:301] APPLY: starting
0709 12:24:03.341907 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:03.342034 (+   127us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:03.342060 (+    26us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:03.342346 (+   286us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:03.342550 (+   204us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:03.342610 (+    60us) write_op.cc:312] APPLY: finished
0709 12:24:03.342864 (+   254us) log.cc:844] Serialized 23 byte log entry
0709 12:24:03.343068 (+   204us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:03.343165 (+    97us) write_op.cc:454] Released schema lock
0709 12:24:03.343213 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:03.343631 (+   418us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:03.343663 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":5376,"child_traces":[["op",{"apply.queue_time_us":435010,"apply.run_cpu_time_us":1893,"apply.run_wall_time_us":7384,"num_ops":1,"prepare.queue_time_us":17272,"prepare.run_cpu_time_us":3170,"prepare.run_wall_time_us":83903,"replication_time_us":414083,"spinlock_wait_cycles":19295104,"thread_start_us":307139,"threads_started":2}]]}
I20260709 12:24:03.588573  3396 raft_consensus.cc:1240] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Rejecting Update request from peer 7cdaa0234c64450eb4ce4500b51b8a81 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:03.590260  3241 raft_consensus.cc:515] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:24:03.615041  3962 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 92204.8us
I20260709 12:24:03.617244  3393 raft_consensus.cc:2468] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 3.
I20260709 12:24:03.618419  3392 raft_consensus.cc:1240] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Rejecting Update request from peer 7cdaa0234c64450eb4ce4500b51b8a81 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:03.630343  3212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.639195  3241 leader_election.cc:290] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:03.648236  3247 raft_consensus.cc:2468] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
I20260709 12:24:03.609686  3993 raft_consensus.cc:2955] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:24:03.700114  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 51682.3us
I20260709 12:24:03.706578  3362 leader_election.cc:304] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
W20260709 12:24:03.702679  5044 log.cc:927] Time spent T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:24:03.721930  3961 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 291242us
I20260709 12:24:03.730770  5019 raft_consensus.cc:2804] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:03.732199  5019 raft_consensus.cc:697] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:03.733235  5019 consensus_queue.cc:237] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:03.730319  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:03.615638  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:03.759544  3140 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54334 (request call id 304) took 1178 ms. Trace:
I20260709 12:24:03.762830  3140 rpcz_store.cc:276] 0709 12:24:02.568729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.568910 (+   181us) service_pool.cc:224] Handling call
0709 12:24:03.746720 (+1177810us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96,"reported_tablets":1}
I20260709 12:24:03.775597  5061 raft_consensus.cc:493] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e38cc90af94c47f8a86be67ad610cbd4)
I20260709 12:24:03.776186  5061 raft_consensus.cc:515] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:03.799400  3391 raft_consensus.cc:3055] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:03.799861  3391 raft_consensus.cc:740] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:03.800640  3391 consensus_queue.cc:260] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:03.802326  3248 consensus_queue.cc:260] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:03.803498  3248 raft_consensus.cc:3060] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.806386  3391 raft_consensus.cc:3060] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:03.817148  3141 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54346 (request call id 282) took 1300 ms. Trace:
W20260709 12:24:03.821329  3364 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.847950  3140 catalog_manager.cc:5697] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:03.833380  3391 raft_consensus.cc:2468] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:24:03.854379  3140 catalog_manager.cc:5697] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:03.859845  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:03.860555  3319 raft_consensus.cc:3060] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:03.860376  3323 consensus_queue.cc:237] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:03.869906  3145 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54338 (request call id 304) took 1437 ms. Trace:
I20260709 12:24:03.870430  3145 rpcz_store.cc:276] 0709 12:24:02.432877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.433034 (+   157us) service_pool.cc:224] Handling call
0709 12:24:03.869838 (+1436804us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 12:24:03.875916  3319 raft_consensus.cc:2468] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:24:03.884447  3241 raft_consensus.cc:1240] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Rejecting Update request from peer c93989d3dd634ef585d77b03354121a4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:03.886735  3289 leader_election.cc:304] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:24:03.888641  5017 raft_consensus.cc:2804] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:03.889482  5017 raft_consensus.cc:697] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
W20260709 12:24:03.890542  3287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.891228  5017 consensus_queue.cc:237] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:03.892242  3214 leader_election.cc:304] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:24:03.894197  5071 raft_consensus.cc:2804] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:03.895023  5071 raft_consensus.cc:697] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:03.896096  5071 consensus_queue.cc:237] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:03.902230  3996 raft_consensus.cc:1240] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Rejecting Update request from peer 7cdaa0234c64450eb4ce4500b51b8a81 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:03.904140  5061 leader_election.cc:290] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:03.936887  3141 rpcz_store.cc:276] 0709 12:24:02.516490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.516922 (+   432us) service_pool.cc:224] Handling call
0709 12:24:02.536953 (+ 20031us) spinlock_profiling.cc:243] Waited 8.39 ms on lock 0x7b5400382554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bed090 00007f39d2c8bf3a 00007f39d8a41e9c 00007f39d8b4004e 00007f39d70012a4 00007f39d7001232 00007f39d7001161 00007f39d70010dc 00007f39d7000392 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:24:03.817079 (+1280126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142275,"reported_tablets":1,"spinlock_wait_cycles":19286400}
I20260709 12:24:03.949200  3248 raft_consensus.cc:2468] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 3.
W20260709 12:24:03.954067  5020 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
W20260709 12:24:03.980726  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 47132.1us
I20260709 12:24:03.996385  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73563dc7466d4636a64ebd4d363935b1" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:04.004642  3996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73563dc7466d4636a64ebd4d363935b1" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:04.043473  3213 leader_election.cc:304] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:04.053346  5020 raft_consensus.cc:2749] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:04.085920  3395 raft_consensus.cc:2955] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:04.107504  3141 catalog_manager.cc:5697] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:04.126184  3145 catalog_manager.cc:5697] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:04.152491  3395 consensus_queue.cc:237] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:04.166329  3996 raft_consensus.cc:1275] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:04.168846  5077 consensus_queue.cc:1048] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:04.181836  4937 raft_consensus.cc:2955] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:04.194100  3994 raft_consensus.cc:2955] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:04.198623  3996 raft_consensus.cc:1275] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:04.203092  3395 consensus_queue.cc:237] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:04.209599  3247 raft_consensus.cc:1275] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:04.211944  3241 raft_consensus.cc:1275] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:04.214532  3996 raft_consensus.cc:1275] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:04.237159  3319 raft_consensus.cc:1275] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:04.257893  3145 catalog_manager.cc:5697] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:04.227404  5077 consensus_queue.cc:1048] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:24:04.267315  3141 catalog_manager.cc:5697] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:04.271040  3392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:24:04.272069  3392 raft_consensus.cc:606] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:04.242203  4971 consensus_queue.cc:1048] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:24:04.285095  5088 consensus_queue.cc:1048] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:04.287792  3391 raft_consensus.cc:1275] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:04.288714  5041 consensus_queue.cc:1048] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:04.295557  5039 raft_consensus.cc:993] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:04.295590  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d02f86799bc54984b41ede0f7db226e6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:04.296505  5027 raft_consensus.cc:1081] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:04.301045  3248 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36494
I20260709 12:24:04.301746  3248 raft_consensus.cc:493] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:04.302119  3248 raft_consensus.cc:3060] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.310915  3241 raft_consensus.cc:1275] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:04.318408  3248 raft_consensus.cc:515] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:04.327316  3392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:04.328028  3392 raft_consensus.cc:3055] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:04.328397  3392 raft_consensus.cc:740] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:04.330916  3248 leader_election.cc:290] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:04.348244  3996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:24:04.349229  3996 raft_consensus.cc:3060] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.348590  3248 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:24:04.349823  3248 raft_consensus.cc:606] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Received request to transfer leadership to TS 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:04.350247  3248 raft_consensus.cc:614] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:04.312016  5014 consensus_queue.cc:1048] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:24:04.322211  5080 consensus_queue.cc:1048] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:24:04.357776  3392 consensus_queue.cc:260] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:04.364602  3392 raft_consensus.cc:3060] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.325611  5075 consensus_queue.cc:1048] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:24:04.355048  3323 raft_consensus.cc:2955] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:04.405345  3996 raft_consensus.cc:2468] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:24:04.412741  3213 leader_election.cc:304] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:24:04.413827  5061 raft_consensus.cc:2804] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:04.414538  5061 raft_consensus.cc:697] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:04.415403  3242 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:24:04.416165  3242 raft_consensus.cc:606] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:04.415681  5061 consensus_queue.cc:237] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:04.416561  3242 raft_consensus.cc:614] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:04.413800  3392 raft_consensus.cc:2468] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:24:04.424026  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:24:04.454365  3242 raft_consensus.cc:1275] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:04.456357  5102 consensus_queue.cc:1048] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:04.464998  5014 raft_consensus.cc:2955] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:04.483918  3991 raft_consensus.cc:2955] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:04.489506  3392 raft_consensus.cc:1275] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:04.485013  5019 raft_consensus.cc:2955] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:04.503059  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:04.507020  3239 tablet_service.cc:1558] Processing DeleteTablet for tablet d02f86799bc54984b41ede0f7db226e6 with delete_type TABLET_DATA_TOMBSTONED (TS 7cdaa0234c64450eb4ce4500b51b8a81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53720
I20260709 12:24:04.513051  3994 raft_consensus.cc:1275] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:04.516928  3144 catalog_manager.cc:5697] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 3 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:04.522318  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:04.538503  3241 raft_consensus.cc:2955] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:04.562382  5041 consensus_queue.cc:1048] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:24:04.547142  4930 consensus_queue.cc:1048] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:24:04.631098  5111 tablet_replica.cc:333] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
W20260709 12:24:04.668924  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 45611.1us
I20260709 12:24:04.680554  5111 raft_consensus.cc:2243] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:04.685655  5111 raft_consensus.cc:2272] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:04.689880  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 113901us
W20260709 12:24:04.743813  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 51677.4us
W20260709 12:24:04.745999  3961 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 50382.4us
I20260709 12:24:04.749114  5088 raft_consensus.cc:493] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:04.778424  3134 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5ba9db97635f4d28bf0f3079edf45a77 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:24:04.779170  3134 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 90586.7us
I20260709 12:24:04.790215  3323 raft_consensus.cc:1275] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:04.792316  5088 raft_consensus.cc:515] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:04.827945  5111 ts_tablet_manager.cc:1916] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:24:04.830514  5123 log.cc:927] Time spent T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:24:04.838673  3243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba9db97635f4d28bf0f3079edf45a77" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:24:04.841569  3321 consensus_queue.cc:237] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:04.854346  5022 consensus_queue.cc:1048] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:04.868835  3140 catalog_manager.cc:5697] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:04.966588  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba9db97635f4d28bf0f3079edf45a77" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:04.975292  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6b70b620cf8243f4b39cf03eaae44881 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:24:04.981354  3961 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 77821.8us
I20260709 12:24:05.039441  5088 leader_election.cc:290] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
W20260709 12:24:05.040195  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 56243.3us
I20260709 12:24:05.071671  3996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba9db97635f4d28bf0f3079edf45a77" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:05.075732  3248 raft_consensus.cc:2955] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:05.095327  3397 raft_consensus.cc:1275] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:05.096323  3364 leader_election.cc:304] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:05.101692  3997 raft_consensus.cc:2955] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:05.104038  5088 raft_consensus.cc:2749] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:05.137526  3995 raft_consensus.cc:1275] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:24:05.144315  5138 log.cc:927] Time spent T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:24:05.165735  3397 raft_consensus.cc:1275] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:24:05.188522  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 63749.2us
I20260709 12:24:05.205327  3146 catalog_manager.cc:5697] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 4 to 5, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:05.212987  3145 catalog_manager.cc:5697] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index 4 to 5, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New cstate: current_term: 2 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:05.245591  3991 raft_consensus.cc:1275] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:05.246950  4971 consensus_queue.cc:1048] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:05.206725  5120 consensus_queue.cc:1048] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:05.265794  5020 consensus_queue.cc:1048] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:05.298264  3311 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b70b620cf8243f4b39cf03eaae44881 with delete_type TABLET_DATA_TOMBSTONED (TS 8525f77f43714a58bf158d872c1db3e4 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:40236
I20260709 12:24:05.322419  5128 consensus_queue.cc:1048] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
W20260709 12:24:05.413837  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 55704.2us
I20260709 12:24:05.441069  5142 tablet_replica.cc:333] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:05.492362  5074 raft_consensus.cc:493] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c93989d3dd634ef585d77b03354121a4)
I20260709 12:24:05.507567  5115 raft_consensus.cc:2955] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:05.508939  3379 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ba9db97635f4d28bf0f3079edf45a77 with delete_type TABLET_DATA_TOMBSTONED (TS c93989d3dd634ef585d77b03354121a4 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:54706
W20260709 12:24:05.504632  4994 log.cc:927] Time spent T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392: Append to log took a long time: real 0.064s	user 0.004s	sys 0.000s
I20260709 12:24:05.522998  5111 ts_tablet_manager.cc:1929] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:05.530813  5111 log.cc:1199] T d02f86799bc54984b41ede0f7db226e6 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/wals/d02f86799bc54984b41ede0f7db226e6
I20260709 12:24:05.535110  5111 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53720 (request call id 84) took 1028 ms. Trace:
I20260709 12:24:05.492996  5074 raft_consensus.cc:515] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:05.614313  5142 raft_consensus.cc:2243] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:05.617046  3243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b70b620cf8243f4b39cf03eaae44881" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:24:05.553057  5111 rpcz_store.cc:276] 0709 12:24:04.506291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.506717 (+   426us) service_pool.cc:224] Handling call
0709 12:24:04.598737 (+ 92020us) ts_tablet_manager.cc:1151] Deleting tablet d02f86799bc54984b41ede0f7db226e6
0709 12:24:04.610119 (+ 11382us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:24:05.218690 (+608571us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:05.520285 (+301595us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:05.534985 (+ 14700us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37049,"tablet-delete.queue_time_us":74793,"thread_start_us":46236,"threads_started":1}
W20260709 12:24:05.657364  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 155018us
W20260709 12:24:05.664176  5148 log.cc:927] Time spent T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:24:05.666193  3132 catalog_manager.cc:5028] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): tablet d02f86799bc54984b41ede0f7db226e6 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:24:05.652868  5142 raft_consensus.cc:2272] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:05.655277  5150 tablet_replica.cc:333] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:05.667912  3397 raft_consensus.cc:2955] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:05.671849  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b70b620cf8243f4b39cf03eaae44881" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:05.667137  5074 leader_election.cc:290] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:05.710677  5150 raft_consensus.cc:2243] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:05.731053  3990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b70b620cf8243f4b39cf03eaae44881" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:05.738624  5142 ts_tablet_manager.cc:1916] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:05.750406  5150 raft_consensus.cc:2272] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:05.764655  5149 log.cc:927] Time spent T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 12:24:05.773803  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 58547.6us
I20260709 12:24:05.784955  3288 leader_election.cc:304] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4
W20260709 12:24:05.785778  3288 raft_consensus.cc:2686] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:24:05.788908  3134 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bb11c9e1f3b048769b242da5c7ae7cd7 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:05.823676  5150 ts_tablet_manager.cc:1916] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:05.831388  3990 raft_consensus.cc:2955] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:05.843415  3146 catalog_manager.cc:5697] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 4 to 5, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:05.856602  5142 ts_tablet_manager.cc:1929] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:24:05.861272  3239 tablet_service.cc:1558] Processing DeleteTablet for tablet bb11c9e1f3b048769b242da5c7ae7cd7 with delete_type TABLET_DATA_TOMBSTONED (TS 7cdaa0234c64450eb4ce4500b51b8a81 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:53720
I20260709 12:24:05.868817  5111 tablet_replica.cc:333] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:05.870177  5142 log.cc:1199] T 6b70b620cf8243f4b39cf03eaae44881 P 8525f77f43714a58bf158d872c1db3e4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/wals/6b70b620cf8243f4b39cf03eaae44881
I20260709 12:24:05.870435  5111 raft_consensus.cc:2243] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:05.871457  5111 raft_consensus.cc:2272] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:05.872501  3134 catalog_manager.cc:5028] TS 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): tablet 6b70b620cf8243f4b39cf03eaae44881 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:24:05.875636  5111 ts_tablet_manager.cc:1916] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:05.936870  5150 ts_tablet_manager.cc:1929] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:24:05.937553  5150 log.cc:1199] T 5ba9db97635f4d28bf0f3079edf45a77 P c93989d3dd634ef585d77b03354121a4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/wals/5ba9db97635f4d28bf0f3079edf45a77
I20260709 12:24:05.939782  3133 catalog_manager.cc:5028] TS c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): tablet 5ba9db97635f4d28bf0f3079edf45a77 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:24:05.971735  5111 ts_tablet_manager.cc:1929] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:24:05.972266  5111 log.cc:1199] T bb11c9e1f3b048769b242da5c7ae7cd7 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/wals/bb11c9e1f3b048769b242da5c7ae7cd7
I20260709 12:24:05.974138  3132 catalog_manager.cc:5028] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): tablet bb11c9e1f3b048769b242da5c7ae7cd7 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
W20260709 12:24:06.420559  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 45454.2us
W20260709 12:24:06.441921  3213 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 50315.1us
I20260709 12:24:06.488519  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:06.611778  3248 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c9b725784044c8da903f092febf7c05"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:24:06.612478  3248 raft_consensus.cc:606] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 LEADER]: Received request to transfer leadership to TS c93989d3dd634ef585d77b03354121a4
I20260709 12:24:06.615715  3248 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:24:06.616379  3248 raft_consensus.cc:606] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 LEADER]: Received request to transfer leadership to TS 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:06.622045  3319 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:48592
I20260709 12:24:06.622778  3319 raft_consensus.cc:606] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:06.625670  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:24:06.628479  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:06.629310  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:06.810159  5153 raft_consensus.cc:993] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:24:06.810844  5153 raft_consensus.cc:1081] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 LEADER]: Signalling peer c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:24:06.812726  3397 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c9b725784044c8da903f092febf7c05"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:41792
I20260709 12:24:06.813339  3397 raft_consensus.cc:493] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:06.813716  3397 raft_consensus.cc:3060] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.830842  3397 raft_consensus.cc:515] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:06.834591  3397 leader_election.cc:290] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:06.839581  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9b725784044c8da903f092febf7c05" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:24:06.840453  3247 raft_consensus.cc:3055] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:06.840847  3247 raft_consensus.cc:740] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:06.842319  3247 consensus_queue.cc:260] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:06.844915  3247 raft_consensus.cc:3060] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.853713  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9b725784044c8da903f092febf7c05" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:06.856716  3319 raft_consensus.cc:3060] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.865731  3247 raft_consensus.cc:2468] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 4.
I20260709 12:24:06.866739  3319 raft_consensus.cc:2468] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 4.
I20260709 12:24:06.867326  3362 leader_election.cc:304] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:24:06.868434  5129 raft_consensus.cc:2804] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:06.869349  5129 raft_consensus.cc:697] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 4 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:06.870386  5129 consensus_queue.cc:237] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:06.886147  3146 catalog_manager.cc:5697] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 3 to 4, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 4 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:06.977157  5017 raft_consensus.cc:993] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:06.977753  5050 raft_consensus.cc:1081] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:06.979388  3247 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36508
I20260709 12:24:06.979923  3247 raft_consensus.cc:493] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:06.980254  3247 raft_consensus.cc:3060] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:06.992446  3247 raft_consensus.cc:515] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:06.995467  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:06.995819  3247 leader_election.cc:290] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:06.996259  3319 raft_consensus.cc:3055] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:06.996618  3319 raft_consensus.cc:740] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:24:06.997788  3392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:06.998409  3392 raft_consensus.cc:3060] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:06.997501  3319 consensus_queue.cc:260] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:07.003454  3319 raft_consensus.cc:3060] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:07.007186  3392 raft_consensus.cc:2468] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:24:07.016474  3321 consensus_queue.cc:237] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:07.018738  3319 raft_consensus.cc:2468] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:24:07.019557  3214 leader_election.cc:304] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:24:07.020385  5117 raft_consensus.cc:2804] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:07.020903  5117 raft_consensus.cc:697] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:07.024039  5117 consensus_queue.cc:237] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:07.076833  3392 raft_consensus.cc:1275] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:24:07.078677  3288 consensus_peers.cc:597] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fe4a63fc24af42ca862894414f658194. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:07.079262  5074 consensus_queue.cc:1048] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:07.080132  3247 raft_consensus.cc:1275] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:07.081652  5074 consensus_queue.cc:1048] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:07.146198  3140 catalog_manager.cc:5697] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:07.172410  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 50627.3us
I20260709 12:24:07.175410  5074 raft_consensus.cc:2955] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:07.177428  5117 raft_consensus.cc:993] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:24:07.177961  5117 raft_consensus.cc:1081] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 LEADER]: Signalling peer 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:24:07.179685  3247 raft_consensus.cc:2955] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:07.182308  3319 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:51186
I20260709 12:24:07.182955  3319 raft_consensus.cc:493] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:07.183342  3319 raft_consensus.cc:3060] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.188068  3397 raft_consensus.cc:2955] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:07.191291  3319 raft_consensus.cc:515] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:07.193534  3319 leader_election.cc:290] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 4 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:07.194746  3392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:07.195453  3392 raft_consensus.cc:3060] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.204115  3392 raft_consensus.cc:2468] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 4.
I20260709 12:24:07.205282  3289 leader_election.cc:304] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:24:07.207172  5167 raft_consensus.cc:2804] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:07.207729  5167 raft_consensus.cc:697] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 4 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
W20260709 12:24:07.209220  3195 auto_rebalancer.cc:629] Failed to schedule move for tablet bfb1e1f3b50843ffbc685b3a019ea8cb: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:24:07.211956  3140 catalog_manager.cc:5697] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:07.217655  5167 consensus_queue.cc:237] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:07.226081  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:24:07.226816  3248 raft_consensus.cc:3055] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:07.227119  3248 raft_consensus.cc:740] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:07.227806  3248 consensus_queue.cc:260] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:07.228992  3248 raft_consensus.cc:3060] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.239673  3248 raft_consensus.cc:2468] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 4.
I20260709 12:24:07.243669  3140 catalog_manager.cc:5697] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 3 to 4, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 4 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:07.271987  3248 consensus_queue.cc:237] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:07.282480  3397 raft_consensus.cc:1275] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:24:07.285562  3213 consensus_peers.cc:597] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2bde98a54e9f47a0b9afe40221bcb336. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:07.285665  5165 consensus_queue.cc:1048] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:07.289069  3319 raft_consensus.cc:1275] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:24:07.294421  5164 consensus_queue.cc:1048] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:07.314986  5152 raft_consensus.cc:2955] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:07.318586  3397 raft_consensus.cc:2955] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:07.321493  3319 raft_consensus.cc:2955] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:07.337867  3321 raft_consensus.cc:1275] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 4 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:07.339998  5072 consensus_queue.cc:1048] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:24:07.346541  3195 auto_rebalancer.cc:629] Failed to schedule move for tablet 6c9b725784044c8da903f092febf7c05: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:24:07.353976  3146 catalog_manager.cc:5697] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:07.395154  3247 raft_consensus.cc:1275] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:07.397531  5072 consensus_queue.cc:1048] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:07.432590  3321 raft_consensus.cc:1275] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:07.434352  5152 consensus_queue.cc:1048] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:07.470158  3397 raft_consensus.cc:1275] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:07.472033  5152 consensus_queue.cc:1048] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:07.601807  5181 ts_tablet_manager.cc:933] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:07.604934  5181 tablet_copy_client.cc:323] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.194:33649
I20260709 12:24:07.607545  3334 tablet_copy_service.cc:140] P 8525f77f43714a58bf158d872c1db3e4: Received BeginTabletCopySession request for tablet fe4a63fc24af42ca862894414f658194 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:33282)
I20260709 12:24:07.608253  3334 tablet_copy_service.cc:161] P 8525f77f43714a58bf158d872c1db3e4: Beginning new tablet copy session on tablet fe4a63fc24af42ca862894414f658194 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:33282: session id = e38cc90af94c47f8a86be67ad610cbd4-fe4a63fc24af42ca862894414f658194
I20260709 12:24:07.619004  3334 tablet_copy_source_session.cc:215] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:07.631256  5181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe4a63fc24af42ca862894414f658194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:07.651361  5181 tablet_copy_client.cc:806] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:07.653276  5181 tablet_copy_client.cc:670] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:07.659121  5181 tablet_copy_client.cc:538] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:07.669319  3397 raft_consensus.cc:1275] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:07.669759  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:07.680475  5167 consensus_queue.cc:1048] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:07.681896  5181 tablet_bootstrap.cc:492] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:24:07.725122  3243 raft_consensus.cc:1275] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:07.727277  5167 consensus_queue.cc:1048] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:07.816197  5184 ts_tablet_manager.cc:933] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:24:07.891758  5184 tablet_copy_client.cc:323] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:24:07.892853  5181 tablet_bootstrap.cc:492] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:07.893918  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet 2bde98a54e9f47a0b9afe40221bcb336 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:24:07.894691  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet 2bde98a54e9f47a0b9afe40221bcb336 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-2bde98a54e9f47a0b9afe40221bcb336
I20260709 12:24:07.904951  3259 tablet_copy_source_session.cc:215] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:07.906215  5181 tablet_bootstrap.cc:492] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:24:07.908159  5181 ts_tablet_manager.cc:1403] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.227s	user 0.031s	sys 0.009s
I20260709 12:24:07.912552  5181 raft_consensus.cc:359] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:07.913640  5181 raft_consensus.cc:740] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:24:07.912580  5184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bde98a54e9f47a0b9afe40221bcb336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:07.914354  5181 consensus_queue.cc:260] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:07.921281  5181 ts_tablet_manager.cc:1434] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 12:24:07.927771  3334 tablet_copy_service.cc:342] P 8525f77f43714a58bf158d872c1db3e4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-fe4a63fc24af42ca862894414f658194 received from {username='slave'} at 127.0.0.1:33282
I20260709 12:24:07.928257  3334 tablet_copy_service.cc:434] P 8525f77f43714a58bf158d872c1db3e4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-fe4a63fc24af42ca862894414f658194 on tablet fe4a63fc24af42ca862894414f658194 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:07.950083  3391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:24:07.950762  3391 raft_consensus.cc:606] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Received request to transfer leadership to TS 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:07.953066  5184 tablet_copy_client.cc:806] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:07.953846  5184 tablet_copy_client.cc:670] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:07.954052  3321 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87fddcaae9194bef90d1047a518e45a6"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:48592
I20260709 12:24:07.954818  3321 raft_consensus.cc:606] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:07.959231  5184 tablet_copy_client.cc:538] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:07.965936  3243 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7494971ced1402da04932982df157fd"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:24:07.966624  3243 raft_consensus.cc:606] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:07.971261  5184 tablet_bootstrap.cc:492] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:24:07.972031  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:24:07.973841  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:07.974777  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:07.993845  3990 raft_consensus.cc:1217] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:08.048849  5184 tablet_bootstrap.cc:492] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:08.051051  5184 tablet_bootstrap.cc:492] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:24:08.052979  5184 ts_tablet_manager.cc:1403] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.082s	user 0.030s	sys 0.014s
I20260709 12:24:08.057250  5184 raft_consensus.cc:359] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:08.070170  5184 raft_consensus.cc:740] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:24:08.071630  5184 consensus_queue.cc:260] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:08.075690  5184 ts_tablet_manager.cc:1434] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 12:24:08.082079  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-2bde98a54e9f47a0b9afe40221bcb336 received from {username='slave'} at 127.0.0.1:57122
I20260709 12:24:08.082684  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-2bde98a54e9f47a0b9afe40221bcb336 on tablet 2bde98a54e9f47a0b9afe40221bcb336 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:08.129688  5186 raft_consensus.cc:993] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4: : Instructing follower 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:24:08.130228  5129 raft_consensus.cc:1081] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Signalling peer 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:24:08.132479  3321 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:51194
I20260709 12:24:08.133136  3321 raft_consensus.cc:493] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:08.133538  3321 raft_consensus.cc:3060] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.142508  3321 raft_consensus.cc:515] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:08.145937  3243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:24:08.146780  3243 raft_consensus.cc:3060] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.156230  3243 raft_consensus.cc:2468] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 4.
I20260709 12:24:08.157929  3321 leader_election.cc:290] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 4 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:08.159412  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:08.160161  3397 raft_consensus.cc:3055] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:08.160529  3397 raft_consensus.cc:740] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:08.161474  3288 leader_election.cc:304] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:24:08.161407  3397 consensus_queue.cc:260] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:08.162743  5074 raft_consensus.cc:2804] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:08.163532  5074 raft_consensus.cc:697] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 4 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:24:08.162779  3397 raft_consensus.cc:3060] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.164662  5074 consensus_queue.cc:237] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:08.182766  3397 raft_consensus.cc:2468] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 4.
I20260709 12:24:08.205603  3140 catalog_manager.cc:5697] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 3 to 4, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 4 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:08.212136  3995 raft_consensus.cc:1217] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:24:08.271610  5152 raft_consensus.cc:993] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:24:08.272161  5152 raft_consensus.cc:1081] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:24:08.274170  3995 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a7494971ced1402da04932982df157fd"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:55596
I20260709 12:24:08.274739  3995 raft_consensus.cc:493] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:08.275065  3995 raft_consensus.cc:3060] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.287369  3995 raft_consensus.cc:515] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:08.297102  3243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7494971ced1402da04932982df157fd" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:24:08.297809  3243 raft_consensus.cc:3055] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:08.298180  3243 raft_consensus.cc:740] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:08.298967  3243 consensus_queue.cc:260] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:08.300135  3243 raft_consensus.cc:3060] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.375629  5167 raft_consensus.cc:993] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:08.376310  5191 raft_consensus.cc:1081] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:08.384560  3241 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87fddcaae9194bef90d1047a518e45a6"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36508
I20260709 12:24:08.385279  3241 raft_consensus.cc:493] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:08.385723  3241 raft_consensus.cc:3060] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:08.450318  5167 raft_consensus.cc:1064] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:24:08.453590  5167 consensus_queue.cc:237] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:08.527717  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 85097.3us
I20260709 12:24:08.571478  3995 leader_election.cc:290] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:08.572758  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7494971ced1402da04932982df157fd" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:08.573401  3391 raft_consensus.cc:3060] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.584357  3391 raft_consensus.cc:2468] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 3.
I20260709 12:24:08.585543  3962 leader_election.cc:304] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:24:08.586444  5154 raft_consensus.cc:2804] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:08.587074  5154 raft_consensus.cc:697] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:24:08.588068  5154 consensus_queue.cc:237] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:08.607353  3248 raft_consensus.cc:1275] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:08.609395  5074 consensus_queue.cc:1048] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:08.616283  3241 raft_consensus.cc:515] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:08.556783  3243 raft_consensus.cc:2468] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 3.
W20260709 12:24:08.658120  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 68817.9us
I20260709 12:24:08.674067  3241 leader_election.cc:290] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:08.683327  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fddcaae9194bef90d1047a518e45a6" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:24:08.684005  3995 raft_consensus.cc:3060] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:08.692266  3995 raft_consensus.cc:2468] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:24:08.693802  3213 leader_election.cc:304] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:24:08.695418  5165 raft_consensus.cc:2804] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:08.695998  5165 raft_consensus.cc:697] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:08.696954  5165 consensus_queue.cc:237] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:08.779927  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 67197us
W20260709 12:24:08.805393  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 96463us
I20260709 12:24:08.835937  3995 raft_consensus.cc:1275] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:08.840416  3247 raft_consensus.cc:1275] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:24:08.853685  5203 log.cc:927] Time spent T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:24:08.874410  3140 catalog_manager.cc:5697] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 3 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:24:08.865123  5204 log.cc:927] Time spent T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81: Append to log took a long time: real 0.076s	user 0.004s	sys 0.000s
I20260709 12:24:08.880045  5209 consensus_queue.cc:1048] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
W20260709 12:24:08.892082  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 184053us
W20260709 12:24:08.893985  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 123669us
I20260709 12:24:08.895448  3146 catalog_manager.cc:5697] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:08.912986  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fddcaae9194bef90d1047a518e45a6" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:08.913823  3319 raft_consensus.cc:3055] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:08.914160  3319 raft_consensus.cc:740] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:24:08.921949  3319 consensus_queue.cc:260] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:08.923688  3319 raft_consensus.cc:3060] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:08.934545  3391 raft_consensus.cc:1275] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:08.938531  3392 raft_consensus.cc:1275] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:08.958729  3243 raft_consensus.cc:1240] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Rejecting Update request from peer 8525f77f43714a58bf158d872c1db3e4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:08.962976  3319 raft_consensus.cc:2468] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:24:08.993178  5205 consensus_queue.cc:1048] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:24:09.022177  5074 consensus_queue.cc:1048] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:09.033540  5191 consensus_queue.cc:1048] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:09.075387  5188 raft_consensus.cc:1064] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
W20260709 12:24:09.096117  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 46563.2us
I20260709 12:24:09.115800  5188 consensus_queue.cc:237] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:09.168164  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 79848.8us
W20260709 12:24:09.168965  3213 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 101602us
I20260709 12:24:09.236595  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:09.238313  3212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.286327  3241 raft_consensus.cc:1275] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:09.289129  5103 consensus_queue.cc:1048] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:09.309451  3995 raft_consensus.cc:1275] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 12:24:09.326574  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 151488us
I20260709 12:24:09.355448  5218 consensus_queue.cc:1048] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:09.358970  3398 raft_consensus.cc:1275] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:09.392899  5103 consensus_queue.cc:1048] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:24:09.417829  5074 raft_consensus.cc:2955] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:09.433897  3321 raft_consensus.cc:1275] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:09.461063  3397 raft_consensus.cc:1275] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:09.482944  3990 raft_consensus.cc:2955] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:09.537930  5188 consensus_queue.cc:1048] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:24:09.547207  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 50297.1us
I20260709 12:24:09.559952  3392 raft_consensus.cc:2955] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:09.563449  3146 catalog_manager.cc:5697] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:09.603035  3991 raft_consensus.cc:1275] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:09.615886  5188 consensus_queue.cc:1048] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:09.671159  3321 raft_consensus.cc:1275] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:09.711138  3321 consensus_queue.cc:237] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:09.735565  5237 raft_consensus.cc:2955] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:09.746516  3996 raft_consensus.cc:1275] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:09.749855  5191 consensus_queue.cc:1048] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:09.761159  5210 raft_consensus.cc:2955] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:09.762627  3995 raft_consensus.cc:2955] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:09.783047  3391 raft_consensus.cc:2955] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:09.789758  5217 consensus_queue.cc:1048] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:09.796563  3392 raft_consensus.cc:1275] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:09.797150  5188 consensus_queue.cc:1048] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:09.811247  3996 raft_consensus.cc:2955] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:09.814800  5220 consensus_queue.cc:1048] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:09.826176  3134 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fe4a63fc24af42ca862894414f658194 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:09.845898  3243 raft_consensus.cc:2955] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:09.863010  3146 catalog_manager.cc:5697] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New cstate: current_term: 1 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:09.889285  3145 catalog_manager.cc:5697] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:09.923183  3323 raft_consensus.cc:2955] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:09.923764  3391 raft_consensus.cc:2955] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:09.927404  3239 tablet_service.cc:1558] Processing DeleteTablet for tablet fe4a63fc24af42ca862894414f658194 with delete_type TABLET_DATA_TOMBSTONED (TS 7cdaa0234c64450eb4ce4500b51b8a81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53720
I20260709 12:24:09.975114  3241 consensus_queue.cc:237] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:10.034947  3995 raft_consensus.cc:1275] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:10.045729  5248 tablet_replica.cc:333] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:10.047173  5248 raft_consensus.cc:2243] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:10.050861  5215 consensus_queue.cc:1048] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:10.056986  5248 raft_consensus.cc:2272] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:10.073087  3391 raft_consensus.cc:1275] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:10.074666  3319 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:48592
I20260709 12:24:10.075455  3319 raft_consensus.cc:606] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:10.089243  5188 consensus_queue.cc:1048] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:10.091468  5217 raft_consensus.cc:2955] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:10.093883  3241 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:24:10.094605  3241 raft_consensus.cc:606] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Received request to transfer leadership to TS c93989d3dd634ef585d77b03354121a4
I20260709 12:24:10.096254  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:24:10.098556  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:10.099939  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:10.110422  3996 raft_consensus.cc:2955] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:10.119848  5248 ts_tablet_manager.cc:1916] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:10.150275  3397 raft_consensus.cc:2955] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:10.196448  5074 raft_consensus.cc:993] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:10.197103  5210 raft_consensus.cc:1081] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:10.217131  3146 catalog_manager.cc:5697] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index 4 to 5, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:10.254352  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2bde98a54e9f47a0b9afe40221bcb336 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:10.257055  3311 tablet_service.cc:1558] Processing DeleteTablet for tablet 2bde98a54e9f47a0b9afe40221bcb336 with delete_type TABLET_DATA_TOMBSTONED (TS 8525f77f43714a58bf158d872c1db3e4 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:40236
W20260709 12:24:10.269048  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 57948.9us
I20260709 12:24:10.272452  3241 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36508
I20260709 12:24:10.273140  3241 raft_consensus.cc:493] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:10.273545  3241 raft_consensus.cc:3060] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:10.285060  5251 raft_consensus.cc:993] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:24:10.285319  5252 tablet_replica.cc:333] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:10.286609  5251 raft_consensus.cc:1081] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Signalling peer c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:24:10.287593  5252 raft_consensus.cc:2243] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:10.288484  5252 raft_consensus.cc:2272] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:10.289047  3398 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:41792
I20260709 12:24:10.289737  3398 raft_consensus.cc:493] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:10.290386  3398 raft_consensus.cc:3060] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.295908  5252 ts_tablet_manager.cc:1916] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:10.298964  3241 raft_consensus.cc:515] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:10.302063  5248 ts_tablet_manager.cc:1929] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:10.318697  5248 log.cc:1199] T fe4a63fc24af42ca862894414f658194 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/wals/fe4a63fc24af42ca862894414f658194
I20260709 12:24:10.319901  3241 leader_election.cc:290] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:10.308706  3398 raft_consensus.cc:515] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:10.321591  3132 catalog_manager.cc:5028] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): tablet fe4a63fc24af42ca862894414f658194 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:24:10.322394  3398 leader_election.cc:290] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:10.323992  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:10.324810  3319 raft_consensus.cc:3055] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:10.325078  3319 raft_consensus.cc:740] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:24:10.326768  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:10.328907  3398 raft_consensus.cc:3060] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:10.330286  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:10.330932  3323 raft_consensus.cc:3060] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.325994  3319 consensus_queue.cc:260] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:10.328042  3241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:24:10.342227  3398 raft_consensus.cc:2468] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:24:10.342370  3241 raft_consensus.cc:3055] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:10.342919  3241 raft_consensus.cc:740] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:10.343894  3214 leader_election.cc:304] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:24:10.343875  3241 consensus_queue.cc:260] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:10.345293  5188 raft_consensus.cc:2804] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:10.345963  5188 raft_consensus.cc:697] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:10.346987  3241 raft_consensus.cc:3060] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.346949  5188 consensus_queue.cc:237] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:10.357822  3241 raft_consensus.cc:2468] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
I20260709 12:24:10.362123  3362 leader_election.cc:304] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:24:10.363197  5199 raft_consensus.cc:2804] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:10.363816  5199 raft_consensus.cc:697] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:10.364980  5199 consensus_queue.cc:237] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:10.377955  5252 ts_tablet_manager.cc:1929] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:24:10.378527  5252 log.cc:1199] T 2bde98a54e9f47a0b9afe40221bcb336 P 8525f77f43714a58bf158d872c1db3e4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/wals/2bde98a54e9f47a0b9afe40221bcb336
I20260709 12:24:10.335762  3319 raft_consensus.cc:3060] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:10.385003  3134 catalog_manager.cc:5028] TS 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): tablet 2bde98a54e9f47a0b9afe40221bcb336 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:24:10.418203  3323 raft_consensus.cc:2468] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
I20260709 12:24:10.425815  3319 raft_consensus.cc:2468] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:24:10.445559  3145 catalog_manager.cc:5697] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:10.457675  3146 catalog_manager.cc:5697] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:10.753129  3398 raft_consensus.cc:1275] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:10.760589  5188 consensus_queue.cc:1048] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:10.774958  3323 raft_consensus.cc:1275] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:10.777110  5188 consensus_queue.cc:1048] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:10.876080  3319 raft_consensus.cc:1275] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:10.879530  5186 consensus_queue.cc:1048] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:10.941461  3243 raft_consensus.cc:1275] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:10.943217  5186 consensus_queue.cc:1048] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:24:10.946971  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 55617.5us
W20260709 12:24:11.153206  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 52482.8us
W20260709 12:24:11.183764  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 53277.2us
W20260709 12:24:11.198611  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 98210.7us
I20260709 12:24:11.269003  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:11.501291  3248 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:24:11.502029  3248 raft_consensus.cc:606] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Received request to transfer leadership to TS 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:11.511106  3397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4a1156be8e3418790f41cd380136b93"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:24:11.511775  3397 raft_consensus.cc:606] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:11.513904  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:24:11.515938  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:11.516837  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:11.631387  3323 consensus_queue.cc:237] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:11.705823  3391 raft_consensus.cc:1275] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:11.708536  3248 raft_consensus.cc:1275] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:11.710212  5074 consensus_queue.cc:1048] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:11.716740  5272 consensus_queue.cc:1048] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:11.725715  3288 consensus_peers.cc:597] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b8131bbfd8374661bd9731b4a57468ee. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:11.796473  5272 raft_consensus.cc:2955] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:11.803335  3397 raft_consensus.cc:2955] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:11.841382  3248 raft_consensus.cc:2955] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:11.867280  5266 raft_consensus.cc:993] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:24:11.867954  5274 raft_consensus.cc:1081] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Signalling peer 8525f77f43714a58bf158d872c1db3e4 to start an election
I20260709 12:24:11.871065  3319 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
 from {username='slave'} at 127.0.0.1:51186
I20260709 12:24:11.871713  3319 raft_consensus.cc:493] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:11.872084  3319 raft_consensus.cc:3060] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:11.874569  3141 catalog_manager.cc:5697] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 5, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 4 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:11.888809  3241 consensus_queue.cc:237] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:11.914541  3319 raft_consensus.cc:515] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:11.918118  3319 leader_election.cc:290] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:11.918049  3243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:24:11.918905  3243 raft_consensus.cc:3055] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:11.919250  3243 raft_consensus.cc:740] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:11.920060  3243 consensus_queue.cc:260] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:11.921244  3243 raft_consensus.cc:3060] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:11.934463  3243 raft_consensus.cc:2468] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 3.
I20260709 12:24:11.951897  3288 leader_election.cc:304] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:24:11.936659  3397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:11.944381  3395 raft_consensus.cc:1275] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:11.953153  3397 raft_consensus.cc:3060] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:11.953619  5074 raft_consensus.cc:2804] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:11.950439  3319 raft_consensus.cc:1275] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:11.954605  5074 raft_consensus.cc:697] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:24:11.955240  5251 consensus_queue.cc:1048] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:11.955982  5074 consensus_queue.cc:237] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:11.959484  5251 consensus_queue.cc:1048] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:24:11.977573  3213 consensus_peers.cc:597] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8bcfe15edc4947b988fd204a12c79094. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:11.978363  3397 raft_consensus.cc:2468] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 3.
I20260709 12:24:11.990798  3395 raft_consensus.cc:2955] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:11.995810  3319 raft_consensus.cc:2955] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:11.987610  5266 raft_consensus.cc:2955] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:12.053565  3395 consensus_queue.cc:237] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:12.056919  5275 raft_consensus.cc:993] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:24:12.057559  5275 raft_consensus.cc:1081] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:24:12.060089  3995 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4a1156be8e3418790f41cd380136b93"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:55592
I20260709 12:24:12.060871  3995 raft_consensus.cc:493] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:12.061311  3995 raft_consensus.cc:3060] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:12.080299  3319 raft_consensus.cc:1275] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 12:24:12.081061  3362 consensus_peers.cc:597] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bfb1e1f3b50843ffbc685b3a019ea8cb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:12.081848  3248 raft_consensus.cc:1275] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:12.091137  3995 raft_consensus.cc:515] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:12.095350  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4a1156be8e3418790f41cd380136b93" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:12.096068  3395 raft_consensus.cc:3055] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:12.096396  3395 raft_consensus.cc:740] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:12.087378  3145 catalog_manager.cc:5697] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:12.099653  3323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4a1156be8e3418790f41cd380136b93" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:12.100348  3323 raft_consensus.cc:3060] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:12.102650  3995 leader_election.cc:290] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:12.107242  3395 consensus_queue.cc:260] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:12.114688  3395 raft_consensus.cc:3060] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:12.081935  5199 consensus_queue.cc:1048] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:24:12.130887  5270 consensus_queue.cc:1048] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:12.134843  3395 raft_consensus.cc:2468] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:24:12.152555  3323 raft_consensus.cc:2468] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:24:12.154985  3963 leader_election.cc:304] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:24:12.156388  5154 raft_consensus.cc:2804] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:12.157011  5154 raft_consensus.cc:697] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:24:12.157997  5154 consensus_queue.cc:237] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:12.203063  3140 catalog_manager.cc:5697] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:12.206187  3141 catalog_manager.cc:5697] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 3 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:12.256510  5294 ts_tablet_manager.cc:933] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:12.312207  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 49724.9us
I20260709 12:24:12.313860  5270 raft_consensus.cc:2955] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:12.338714  3146 catalog_manager.cc:5697] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index -1 to 4, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 3 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:12.430768  3319 raft_consensus.cc:2955] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:12.451807  3398 raft_consensus.cc:1275] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:12.452605  3243 raft_consensus.cc:2955] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:12.454898  5294 tablet_copy_client.cc:323] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.194:33649
I20260709 12:24:12.457165  3334 tablet_copy_service.cc:140] P 8525f77f43714a58bf158d872c1db3e4: Received BeginTabletCopySession request for tablet b8131bbfd8374661bd9731b4a57468ee from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:33282)
I20260709 12:24:12.459239  3334 tablet_copy_service.cc:161] P 8525f77f43714a58bf158d872c1db3e4: Beginning new tablet copy session on tablet b8131bbfd8374661bd9731b4a57468ee from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:33282: session id = e38cc90af94c47f8a86be67ad610cbd4-b8131bbfd8374661bd9731b4a57468ee
I20260709 12:24:12.472839  3334 tablet_copy_source_session.cc:215] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:12.491412  5294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8131bbfd8374661bd9731b4a57468ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:12.505085  3395 consensus_queue.cc:237] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:12.512856  5272 consensus_queue.cc:1048] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:12.554396  3319 raft_consensus.cc:1275] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:12.560709  5275 consensus_queue.cc:1048] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:12.565662  5294 tablet_copy_client.cc:806] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:12.566432  5294 tablet_copy_client.cc:670] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:12.576767  3241 raft_consensus.cc:1275] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:12.586362  5275 consensus_queue.cc:1048] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:24:12.589303  3362 consensus_peers.cc:597] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f3795979cf042008b3447b47c75de09. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:12.593470  3243 raft_consensus.cc:1275] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:12.594820  5298 ts_tablet_manager.cc:933] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:24:12.595047  5191 consensus_queue.cc:1048] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:12.602599  5302 ts_tablet_manager.cc:933] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:12.618822  3319 raft_consensus.cc:1275] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:12.622184  5294 tablet_copy_client.cc:538] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:12.623557  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:12.627117  5154 consensus_queue.cc:1048] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:12.717458  3316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8d7a917f21741e9827c30e5433a38e3"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
new_leader_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:48592
I20260709 12:24:12.718380  3316 raft_consensus.cc:606] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Received request to transfer leadership to TS c93989d3dd634ef585d77b03354121a4
I20260709 12:24:12.735392  5298 tablet_copy_client.cc:323] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:24:12.740464  3398 raft_consensus.cc:1275] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:12.743016  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet 8bcfe15edc4947b988fd204a12c79094 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:24:12.743770  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet 8bcfe15edc4947b988fd204a12c79094 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-8bcfe15edc4947b988fd204a12c79094
I20260709 12:24:12.749758  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:24:12.758160  5290 consensus_queue.cc:1048] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:12.770108  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:12.773826  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:12.793123  3259 tablet_copy_source_session.cc:215] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:12.880475  5302 tablet_copy_client.cc:323] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
W20260709 12:24:12.981333  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 69071.7us
W20260709 12:24:13.021430  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 126766us
W20260709 12:24:13.024446  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 172278us
W20260709 12:24:13.073781  5311 log.cc:927] Time spent T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4: Append to log took a long time: real 0.092s	user 0.002s	sys 0.000s
I20260709 12:24:13.080148  5294 tablet_bootstrap.cc:492] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
W20260709 12:24:13.102677  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 86264us
W20260709 12:24:13.222286  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 238665us
W20260709 12:24:13.293212  5307 log.cc:927] Time spent T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81: Append to log took a long time: real 0.250s	user 0.001s	sys 0.003s
I20260709 12:24:13.260462  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet bfb1e1f3b50843ffbc685b3a019ea8cb from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:24:13.294019  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet bfb1e1f3b50843ffbc685b3a019ea8cb from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-bfb1e1f3b50843ffbc685b3a019ea8cb
I20260709 12:24:13.303339  5298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bcfe15edc4947b988fd204a12c79094. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:13.336222  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 81960.6us
W20260709 12:24:13.340845  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 54248.6us
I20260709 12:24:13.314160  5330 raft_consensus.cc:2955] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
W20260709 12:24:13.380956  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 94167.7us
W20260709 12:24:13.427776  3361 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.001s	sys 0.021s
W20260709 12:24:13.428370  3361 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.001s	sys 0.021s
I20260709 12:24:13.430229  3408 tablet_copy_source_session.cc:215] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:24:13.457933  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 85416.6us
W20260709 12:24:13.461694  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 123668us
W20260709 12:24:13.463513  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 45189.1us
W20260709 12:24:13.486207  3961 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 100441us
I20260709 12:24:13.500883  5302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb1e1f3b50843ffbc685b3a019ea8cb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:13.515543  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 55989.3us
W20260709 12:24:13.515739  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 47697.2us
I20260709 12:24:13.529884  5330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:34210 (request call id 16) took 1027 ms. Trace:
I20260709 12:24:13.530422  5330 rpcz_store.cc:276] 0709 12:24:12.502141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.502322 (+   181us) service_pool.cc:224] Handling call
0709 12:24:12.508620 (+  6298us) log.cc:844] Serialized 492 byte log entry
0709 12:24:12.550301 (+ 41681us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b54002f4a20. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bed090 00007f39d2bfa6b8 00007f39d2bfa2fa 00007f39d2c7e965 00007f39d2c8a03c 00007f39d2c896b3 00007f39d7c49d29 00007f39d11c81d4 00007f39d11c8162 00007f39d11c8091 00007f39d11c800c
0709 12:24:13.529808 (+979507us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2012,"raft.run_cpu_time_us":2078,"raft.run_wall_time_us":11778,"spinlock_wait_cycles":3325696,"thread_start_us":15025,"threads_started":1,"wal-append.queue_time_us":13785}
I20260709 12:24:13.569007  3323 raft_consensus.cc:2955] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
W20260709 12:24:13.572497  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 112059us
W20260709 12:24:13.664842  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 83403.1us
W20260709 12:24:13.701665  3213 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 121740us
W20260709 12:24:13.767437  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 58713.2us
I20260709 12:24:13.784114  5352 ts_tablet_manager.cc:933] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:13.797430  3241 raft_consensus.cc:2955] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
W20260709 12:24:13.802366  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 56477.2us
I20260709 12:24:13.842103  5327 raft_consensus.cc:993] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4: : Instructing follower c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:24:13.843196  5340 raft_consensus.cc:1081] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Signalling peer c93989d3dd634ef585d77b03354121a4 to start an election
W20260709 12:24:13.850570  5353 log.cc:927] Time spent T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:24:13.870715  3141 catalog_manager.cc:5697] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index -1 to 2, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
W20260709 12:24:13.890127  3962 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 147792us
I20260709 12:24:13.916940  5275 raft_consensus.cc:493] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:13.922204  5275 raft_consensus.cc:515] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:13.924523  5275 leader_election.cc:290] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:13.919721  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 157034us
I20260709 12:24:13.940831  5298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:55596 (request call id 828) took 1406 ms. Trace:
I20260709 12:24:13.941376  5298 rpcz_store.cc:276] 0709 12:24:12.534795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.534980 (+   185us) service_pool.cc:224] Handling call
0709 12:24:12.595191 (+ 60211us) ts_tablet_manager.cc:934] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
0709 12:24:13.218125 (+622934us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:13.716063 (+497938us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:13.732065 (+ 16002us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:13.921497 (+189432us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:13.940319 (+ 18822us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:13.940667 (+   348us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:13.940723 (+    56us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":33859,"tablet-copy.queue_time_us":59267,"thread_start_us":124810,"threads_started":2}
I20260709 12:24:13.944263  5298 tablet_copy_client.cc:806] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:13.944115  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:24:13.947429  5298 tablet_copy_client.cc:670] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:13.966697  5352 tablet_copy_client.cc:323] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:24:14.005949  3396 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8d7a917f21741e9827c30e5433a38e3"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:41806
I20260709 12:24:14.006655  3396 raft_consensus.cc:493] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:14.007030  3396 raft_consensus.cc:3060] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.011216  5294 tablet_bootstrap.cc:492] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:14.012542  5294 tablet_bootstrap.cc:492] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:24:14.013423  5294 ts_tablet_manager.cc:1403] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.934s	user 0.059s	sys 0.027s
I20260709 12:24:14.016150  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet 1f3795979cf042008b3447b47c75de09 from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:24:14.016420  5298 tablet_copy_client.cc:538] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:14.016769  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet 1f3795979cf042008b3447b47c75de09 from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-1f3795979cf042008b3447b47c75de09
I20260709 12:24:14.029709  3396 raft_consensus.cc:515] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:14.039569  3408 tablet_copy_source_session.cc:215] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:14.040827  3396 leader_election.cc:290] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:14.045331  5352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f3795979cf042008b3447b47c75de09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:14.056376  5294 raft_consensus.cc:359] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:14.060923  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d7a917f21741e9827c30e5433a38e3" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:14.058856  5302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:55592 (request call id 726) took 1480 ms. Trace:
I20260709 12:24:14.061666  3322 raft_consensus.cc:3055] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:14.062012  3322 raft_consensus.cc:740] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:24:14.063532  3322 consensus_queue.cc:260] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:14.063844  3318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:14.064816  3322 raft_consensus.cc:3060] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.061678  5302 rpcz_store.cc:276] 0709 12:24:12.578521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.578706 (+   185us) service_pool.cc:224] Handling call
0709 12:24:12.631077 (+ 52371us) ts_tablet_manager.cc:934] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
0709 12:24:13.499374 (+868297us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:13.843784 (+344410us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:13.843924 (+   140us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:14.058018 (+214094us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:14.058360 (+   342us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:14.058710 (+   350us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:14.058782 (+    72us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":25085,"tablet-copy.queue_time_us":23320,"thread_start_us":220175,"threads_started":2}
I20260709 12:24:14.066761  5302 tablet_copy_client.cc:806] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:14.067361  5302 tablet_copy_client.cc:670] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:14.073221  5298 tablet_bootstrap.cc:492] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:24:14.057454  5294 raft_consensus.cc:740] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:24:14.079631  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d7a917f21741e9827c30e5433a38e3" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:24:14.084344  3364 leader_election.cc:304] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:14.160888  5324 raft_consensus.cc:2749] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:14.088909  3247 raft_consensus.cc:3060] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:14.162374  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 78195.2us
I20260709 12:24:14.132565  3322 raft_consensus.cc:2468] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 2.
W20260709 12:24:14.140914  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 50141.9us
I20260709 12:24:14.180086  5294 consensus_queue.cc:260] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:14.182423  5294 ts_tablet_manager.cc:1434] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.168s	user 0.012s	sys 0.006s
I20260709 12:24:14.101658  5302 tablet_copy_client.cc:538] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:14.200546  3364 leader_election.cc:304] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:24:14.203938  5275 raft_consensus.cc:2804] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:14.204469  5275 raft_consensus.cc:697] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:14.205355  5275 consensus_queue.cc:237] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:14.217694  3247 raft_consensus.cc:2468] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 2.
I20260709 12:24:14.230587  3334 tablet_copy_service.cc:342] P 8525f77f43714a58bf158d872c1db3e4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-b8131bbfd8374661bd9731b4a57468ee received from {username='slave'} at 127.0.0.1:33282
I20260709 12:24:14.231227  3334 tablet_copy_service.cc:434] P 8525f77f43714a58bf158d872c1db3e4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-b8131bbfd8374661bd9731b4a57468ee on tablet b8131bbfd8374661bd9731b4a57468ee with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:14.251407  5352 tablet_copy_client.cc:806] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:14.265270  5352 tablet_copy_client.cc:670] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:14.284473  5302 tablet_bootstrap.cc:492] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:24:14.290170  5352 tablet_copy_client.cc:538] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:14.295310  3995 raft_consensus.cc:1217] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.4->[4.5-4.5]   Dedup: 4.5->[]
I20260709 12:24:14.325750  3140 catalog_manager.cc:5697] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 1 to 2, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 2 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:14.347818  5352 tablet_bootstrap.cc:492] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:24:14.417904  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:14.493727  5298 tablet_bootstrap.cc:492] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:14.494884  5298 tablet_bootstrap.cc:492] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:24:14.495618  5298 ts_tablet_manager.cc:1403] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.423s	user 0.043s	sys 0.026s
I20260709 12:24:14.499289  5298 raft_consensus.cc:359] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:14.500221  5298 raft_consensus.cc:740] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:24:14.500955  5298 consensus_queue.cc:260] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:14.523960  5298 ts_tablet_manager.cc:1434] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:24:14.527972  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-8bcfe15edc4947b988fd204a12c79094 received from {username='slave'} at 127.0.0.1:57122
I20260709 12:24:14.528484  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-8bcfe15edc4947b988fd204a12c79094 on tablet 8bcfe15edc4947b988fd204a12c79094 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:14.604905  3395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7048879a65244efa5a140c52d782803"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:24:14.605599  3395 raft_consensus.cc:606] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:14.616259  5302 tablet_bootstrap.cc:492] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:14.626992  5302 tablet_bootstrap.cc:492] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:24:14.627890  5302 ts_tablet_manager.cc:1403] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.344s	user 0.022s	sys 0.014s
I20260709 12:24:14.629813  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:24:14.631934  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:14.633404  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:14.654251  5302 raft_consensus.cc:359] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:14.655373  5302 raft_consensus.cc:740] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:24:14.656111  5302 consensus_queue.cc:260] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:14.679328  5302 ts_tablet_manager.cc:1434] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.001s
I20260709 12:24:14.748689  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-bfb1e1f3b50843ffbc685b3a019ea8cb received from {username='slave'} at 127.0.0.1:34216
I20260709 12:24:14.749246  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-bfb1e1f3b50843ffbc685b3a019ea8cb on tablet bfb1e1f3b50843ffbc685b3a019ea8cb with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:14.781755  3248 raft_consensus.cc:1275] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:14.791107  3990 raft_consensus.cc:1217] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 12:24:14.798902  5324 consensus_queue.cc:1048] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:24:14.799398  5352 tablet_bootstrap.cc:492] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:14.811167  3321 raft_consensus.cc:1275] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:14.819844  3995 raft_consensus.cc:1217] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:14.822469  5352 tablet_bootstrap.cc:492] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:24:14.832273  5352 ts_tablet_manager.cc:1403] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.485s	user 0.019s	sys 0.029s
I20260709 12:24:14.840368  5324 consensus_queue.cc:1048] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:24:14.877154  3961 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 76035.7us
I20260709 12:24:14.906932  5346 raft_consensus.cc:993] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:14.907166  5352 raft_consensus.cc:359] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:14.908159  5352 raft_consensus.cc:740] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:24:14.908605  5346 raft_consensus.cc:1081] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:14.920977  5337 raft_consensus.cc:1064] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:24:14.925196  3248 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7048879a65244efa5a140c52d782803"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36494
I20260709 12:24:14.925915  3248 raft_consensus.cc:493] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:14.926262  3248 raft_consensus.cc:3060] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.931039  5337 consensus_queue.cc:237] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:14.908964  5352 consensus_queue.cc:260] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:14.936450  5352 ts_tablet_manager.cc:1434] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.095s	user 0.008s	sys 0.000s
I20260709 12:24:14.938447  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-1f3795979cf042008b3447b47c75de09 received from {username='slave'} at 127.0.0.1:34216
I20260709 12:24:14.938876  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-1f3795979cf042008b3447b47c75de09 on tablet 1f3795979cf042008b3447b47c75de09 with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:14.939564  3248 raft_consensus.cc:515] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:24:14.940110  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 86897.9us
W20260709 12:24:14.951201  3213 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 103767us
W20260709 12:24:14.956828  3961 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 76508.5us
I20260709 12:24:14.957731  3248 leader_election.cc:290] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:14.959903  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7048879a65244efa5a140c52d782803" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:14.960615  3395 raft_consensus.cc:3055] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:14.960929  3395 raft_consensus.cc:740] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:14.961810  3395 consensus_queue.cc:260] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:14.963205  3395 raft_consensus.cc:3060] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.968158  3321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7048879a65244efa5a140c52d782803" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:14.968855  3321 raft_consensus.cc:3060] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:14.998073  3395 raft_consensus.cc:2468] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:24:15.003428  3214 leader_election.cc:304] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:24:15.014262  5335 raft_consensus.cc:2804] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:15.014941  5335 raft_consensus.cc:697] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:15.017382  5335 consensus_queue.cc:237] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:15.031306  3995 raft_consensus.cc:1275] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [term 4 LEARNER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:15.032899  3247 raft_consensus.cc:1275] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:15.038436  3321 raft_consensus.cc:2468] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:24:15.042949  3397 raft_consensus.cc:1275] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:24:15.048321  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 44978.6us
W20260709 12:24:15.068275  3213 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 64009.8us
I20260709 12:24:15.044034  5320 consensus_queue.cc:1048] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:24:15.125497  5348 consensus_queue.cc:1048] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:15.126909  5191 consensus_queue.cc:1048] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:15.149470  3145 catalog_manager.cc:5697] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:15.269991  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 131385us
I20260709 12:24:15.281945  3994 raft_consensus.cc:1217] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:24:15.352052  3961 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 77471.3us
W20260709 12:24:15.394364  5393 log.cc:927] Time spent T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
W20260709 12:24:15.419229  5394 log.cc:927] Time spent T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:24:15.501315  5405 raft_consensus.cc:2955] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:15.521239  5377 raft_consensus.cc:1064] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:24:15.524190  5377 consensus_queue.cc:237] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:15.549476  3321 raft_consensus.cc:1275] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:15.553704  3995 raft_consensus.cc:2955] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:15.576246  5401 raft_consensus.cc:1064] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:24:15.580529  3391 raft_consensus.cc:2955] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:15.590991  5401 consensus_queue.cc:237] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:15.601131  5383 consensus_queue.cc:1048] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:15.611119  3994 raft_consensus.cc:1275] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:15.614782  5349 consensus_queue.cc:1048] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:15.595719  3140 catalog_manager.cc:5697] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 5 to 6, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:15.602181  3242 raft_consensus.cc:2955] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:15.635224  3318 raft_consensus.cc:1275] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:15.642412  3392 raft_consensus.cc:1275] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:15.642892  3395 raft_consensus.cc:1275] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:15.650883  5377 consensus_queue.cc:1048] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:15.658558  5324 raft_consensus.cc:1064] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:24:15.660647  5388 consensus_queue.cc:1048] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:15.658326  5335 consensus_queue.cc:1048] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:15.668566  3321 consensus_queue.cc:237] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:15.677596  5324 consensus_queue.cc:237] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:15.687824  3994 raft_consensus.cc:1275] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:24:15.688756  3995 raft_consensus.cc:1275] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:15.689046  3248 raft_consensus.cc:1275] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:15.692996  5402 consensus_queue.cc:1048] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:15.696719  5401 consensus_queue.cc:1048] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:15.699369  5191 consensus_queue.cc:1048] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:15.712998  5348 raft_consensus.cc:2955] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 4 LEADER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:15.715936  3990 raft_consensus.cc:2955] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:15.731184  3321 raft_consensus.cc:1275] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:15.734252  3322 raft_consensus.cc:1275] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:15.738656  3248 raft_consensus.cc:1275] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:24:15.740706  5191 consensus_queue.cc:1048] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:15.740483  3141 catalog_manager.cc:5697] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 6 to 7, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New cstate: current_term: 4 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:15.746368  3242 raft_consensus.cc:2955] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:15.750053  5401 consensus_queue.cc:1048] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:15.754657  3379 tablet_service.cc:1558] Processing DeleteTablet for tablet b8131bbfd8374661bd9731b4a57468ee with delete_type TABLET_DATA_TOMBSTONED (TS c93989d3dd634ef585d77b03354121a4 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:54706
I20260709 12:24:15.774111  3995 raft_consensus.cc:1275] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:15.775599  5401 consensus_queue.cc:1048] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:15.776651  3242 raft_consensus.cc:1275] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:15.815985  5332 consensus_queue.cc:1048] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:15.819691  3134 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b8131bbfd8374661bd9731b4a57468ee with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:15.826647  5415 tablet_replica.cc:333] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:15.832481  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:15.831136  5346 consensus_queue.cc:1048] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:24:15.894939  5377 raft_consensus.cc:2955] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:24:15.898672  3962 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 60084.2us
I20260709 12:24:15.930635  3318 raft_consensus.cc:2955] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:24:15.975740  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 135290us
I20260709 12:24:15.985180  3395 raft_consensus.cc:2955] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:15.988065  5199 raft_consensus.cc:2955] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.001503  3322 raft_consensus.cc:2955] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.006145  5415 raft_consensus.cc:2243] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Raft consensus shutting down.
W20260709 12:24:16.125159  3962 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 81119.8us
W20260709 12:24:16.130620  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 80254.1us
W20260709 12:24:16.138688  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 89864.2us
I20260709 12:24:16.216856  3248 raft_consensus.cc:2955] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.276444  3146 catalog_manager.cc:5697] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 4 to 5, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.291231  3140 catalog_manager.cc:5697] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.286819  5415 raft_consensus.cc:2272] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:16.330536  3994 raft_consensus.cc:2955] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.345377  3991 raft_consensus.cc:2955] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } attrs { replace: true } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.348896  5415 ts_tablet_manager.cc:1916] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:24:16.366267  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 153518us
I20260709 12:24:16.380743  5402 raft_consensus.cc:2955] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.471239  3247 raft_consensus.cc:2955] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.502558  3991 raft_consensus.cc:2955] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.538388  3318 raft_consensus.cc:2955] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.540676  5415 ts_tablet_manager.cc:1929] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.6
I20260709 12:24:16.541173  5415 log.cc:1199] T b8131bbfd8374661bd9731b4a57468ee P c93989d3dd634ef585d77b03354121a4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/wals/b8131bbfd8374661bd9731b4a57468ee
I20260709 12:24:16.565268  3133 catalog_manager.cc:5028] TS c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): tablet b8131bbfd8374661bd9731b4a57468ee (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:24:16.577646  3247 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:24:16.580559  3247 raft_consensus.cc:606] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Received request to transfer leadership to TS c93989d3dd634ef585d77b03354121a4
I20260709 12:24:16.590459  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:24:16.592559  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:16.619248  5376 raft_consensus.cc:993] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:24:16.619822  5376 raft_consensus.cc:1081] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Signalling peer c93989d3dd634ef585d77b03354121a4 to start an election
I20260709 12:24:16.625734  3391 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
 from {username='slave'} at 127.0.0.1:41792
I20260709 12:24:16.626873  3391 raft_consensus.cc:493] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:16.627446  3391 raft_consensus.cc:3060] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:16.640879  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:16.655803  3141 catalog_manager.cc:5697] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 2 to 3, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.689324  3247 consensus_queue.cc:237] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:16.695593  3392 consensus_queue.cc:237] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:16.696696  3991 raft_consensus.cc:1275] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:16.701997  5376 consensus_queue.cc:1048] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:16.702567  3991 raft_consensus.cc:1275] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:16.704295  5420 consensus_queue.cc:1048] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:16.705230  3391 raft_consensus.cc:515] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:16.699776  3393 consensus_queue.cc:237] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:16.712086  3391 leader_election.cc:290] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:16.712765  3319 raft_consensus.cc:1275] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:16.714623  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:24:16.715236  3248 raft_consensus.cc:3055] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:16.715517  3248 raft_consensus.cc:740] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:16.716246  3248 consensus_queue.cc:260] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:16.716697  5431 consensus_queue.cc:1048] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:16.719589  3994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:24:16.720183  3997 raft_consensus.cc:1275] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:16.720211  3994 raft_consensus.cc:3060] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:16.707612  3247 raft_consensus.cc:1275] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:16.717553  3248 raft_consensus.cc:3060] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:16.728451  3994 raft_consensus.cc:2468] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
I20260709 12:24:16.732029  5199 consensus_queue.cc:1048] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:16.747066  3362 leader_election.cc:304] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:24:16.749305  3248 raft_consensus.cc:2468] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
I20260709 12:24:16.729849  5401 consensus_queue.cc:1048] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:24:16.753762  3319 raft_consensus.cc:1275] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:16.752899  5199 raft_consensus.cc:2955] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.757872  3994 raft_consensus.cc:2955] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.751572  5332 raft_consensus.cc:2804] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:16.769610  5427 consensus_queue.cc:1048] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:24:16.762392  5376 raft_consensus.cc:2955] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.790109  5324 raft_consensus.cc:2955] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.790392  3991 raft_consensus.cc:2955] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.792234  3146 catalog_manager.cc:5697] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New cstate: current_term: 1 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.801391  3248 raft_consensus.cc:2955] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.800840  3994 raft_consensus.cc:2955] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.780241  5332 raft_consensus.cc:697] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:16.811743  5332 consensus_queue.cc:237] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:16.823449  3141 catalog_manager.cc:5697] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: config changed from index 5 to 6, VOTER 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) evicted. New cstate: current_term: 3 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.830565  3319 raft_consensus.cc:2955] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.834514  3318 raft_consensus.cc:2955] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.839752  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1f3795979cf042008b3447b47c75de09 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:16.854909  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bfb1e1f3b50843ffbc685b3a019ea8cb with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:16.875823  3132 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8bcfe15edc4947b988fd204a12c79094 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:16.902818  3311 tablet_service.cc:1558] Processing DeleteTablet for tablet bfb1e1f3b50843ffbc685b3a019ea8cb with delete_type TABLET_DATA_TOMBSTONED (TS 8525f77f43714a58bf158d872c1db3e4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:40236
I20260709 12:24:16.906231  3239 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f3795979cf042008b3447b47c75de09 with delete_type TABLET_DATA_TOMBSTONED (TS 7cdaa0234c64450eb4ce4500b51b8a81 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53720
I20260709 12:24:16.926879  3140 catalog_manager.cc:5697] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 3 to 4, VOTER c93989d3dd634ef585d77b03354121a4 (127.2.4.195) evicted. New cstate: current_term: 1 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:16.950387  3379 tablet_service.cc:1558] Processing DeleteTablet for tablet 8bcfe15edc4947b988fd204a12c79094 with delete_type TABLET_DATA_TOMBSTONED (TS c93989d3dd634ef585d77b03354121a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54706
I20260709 12:24:16.951608  5415 tablet_replica.cc:333] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:16.981842  5439 tablet_replica.cc:333] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:16.988274  5438 tablet_replica.cc:333] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:16.988065  3145 catalog_manager.cc:5697] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:16.999717  5438 raft_consensus.cc:2243] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:17.005328  5438 raft_consensus.cc:2272] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:17.010527  5438 ts_tablet_manager.cc:1916] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:17.012360  5415 raft_consensus.cc:2243] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:17.013381  5415 raft_consensus.cc:2272] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:17.020876  5415 ts_tablet_manager.cc:1916] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:17.046540  5439 raft_consensus.cc:2243] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:17.047381  5439 raft_consensus.cc:2272] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:17.051672  5439 ts_tablet_manager.cc:1916] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:17.067191  5415 ts_tablet_manager.cc:1929] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:17.072800  5438 ts_tablet_manager.cc:1929] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 12:24:17.073411  5438 log.cc:1199] T bfb1e1f3b50843ffbc685b3a019ea8cb P 8525f77f43714a58bf158d872c1db3e4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/wals/bfb1e1f3b50843ffbc685b3a019ea8cb
I20260709 12:24:17.076278  3134 catalog_manager.cc:5028] TS 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): tablet bfb1e1f3b50843ffbc685b3a019ea8cb (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:24:17.078830  5415 log.cc:1199] T 8bcfe15edc4947b988fd204a12c79094 P c93989d3dd634ef585d77b03354121a4: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/wals/8bcfe15edc4947b988fd204a12c79094
I20260709 12:24:17.079094  5439 ts_tablet_manager.cc:1929] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:17.079813  5439 log.cc:1199] T 1f3795979cf042008b3447b47c75de09 P 7cdaa0234c64450eb4ce4500b51b8a81: Deleting WAL directory at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/wals/1f3795979cf042008b3447b47c75de09
I20260709 12:24:17.081204  3133 catalog_manager.cc:5028] TS c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): tablet 8bcfe15edc4947b988fd204a12c79094 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:24:17.082913  3132 catalog_manager.cc:5028] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): tablet 1f3795979cf042008b3447b47c75de09 (table test-workload [id=7474d7cac0bc41b4bb8668c33c08e34a]) successfully deleted
I20260709 12:24:17.203346  3248 raft_consensus.cc:1275] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:17.205138  5432 consensus_queue.cc:1048] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:17.219300  3994 raft_consensus.cc:1275] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:17.221385  5324 consensus_queue.cc:1048] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:17.783033  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:17.916965  3391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d373e8b022274a08b08f00c0057ae3d3"
dest_uuid: "c93989d3dd634ef585d77b03354121a4"
mode: GRACEFUL
new_leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:24:17.917809  3391 raft_consensus.cc:606] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Received request to transfer leadership to TS 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:17.919741  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:24:17.921520  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:17.922453  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:18.051903  3391 consensus_queue.cc:237] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:18.060544  3319 raft_consensus.cc:1275] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:24:18.061744  3362 consensus_peers.cc:597] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 11921fa3d80340c18f076a4d4499340b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:18.061949  3249 raft_consensus.cc:1275] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:24:18.063328  5324 consensus_queue.cc:1048] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:18.064666  5401 consensus_queue.cc:1048] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:18.090922  5324 raft_consensus.cc:2955] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:18.102656  3319 raft_consensus.cc:2955] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:18.120667  3249 raft_consensus.cc:2955] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:18.127310  3145 catalog_manager.cc:5697] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 2 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:18.145529  5445 raft_consensus.cc:993] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4: : Instructing follower 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:18.146679  5445 raft_consensus.cc:1081] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Signalling peer 7cdaa0234c64450eb4ce4500b51b8a81 to start an election
I20260709 12:24:18.192806  3248 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d373e8b022274a08b08f00c0057ae3d3"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
 from {username='slave'} at 127.0.0.1:36494
I20260709 12:24:18.193702  3248 raft_consensus.cc:493] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:18.194200  3248 raft_consensus.cc:3060] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:18.207538  3248 raft_consensus.cc:515] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:18.209895  3248 leader_election.cc:290] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:18.210541  3318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d373e8b022274a08b08f00c0057ae3d3" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:18.211362  3318 raft_consensus.cc:3060] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:18.212392  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d373e8b022274a08b08f00c0057ae3d3" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:18.213161  3391 raft_consensus.cc:3055] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:18.213498  3391 raft_consensus.cc:740] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:18.214321  3391 consensus_queue.cc:260] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:18.216168  3391 raft_consensus.cc:3060] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:18.226799  3248 consensus_queue.cc:237] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:18.227499  3318 raft_consensus.cc:2468] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
I20260709 12:24:18.231643  3215 leader_election.cc:304] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4; no voters: 
I20260709 12:24:18.232873  5335 raft_consensus.cc:2804] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:18.236898  5335 raft_consensus.cc:697] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Leader. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:18.250070  5335 consensus_queue.cc:237] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:18.265873  3391 raft_consensus.cc:2468] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cdaa0234c64450eb4ce4500b51b8a81 in term 2.
W20260709 12:24:18.271798  3213 consensus_peers.cc:597] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 -> Peer e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287): Couldn't send request to peer e38cc90af94c47f8a86be67ad610cbd4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 84e58c891d3b452eb575499063d5d34f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:18.273479  3391 raft_consensus.cc:1275] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:24:18.276010  3318 raft_consensus.cc:1275] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:24:18.276033  5388 consensus_queue.cc:1048] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:24:18.280346  5388 consensus_queue.cc:1048] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:18.320521  5428 raft_consensus.cc:2955] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:18.328145  3318 raft_consensus.cc:2955] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:18.346822  3141 catalog_manager.cc:5697] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 reported cstate change: term changed from 1 to 2, leader changed from c93989d3dd634ef585d77b03354121a4 (127.2.4.195) to 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:18.350842  3392 raft_consensus.cc:2955] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:18.366240  3140 catalog_manager.cc:5697] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index -1 to 3, NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) added. New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } } }
I20260709 12:24:18.577411  5463 ts_tablet_manager.cc:933] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:24:18.585062  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 62875.9us
I20260709 12:24:18.591472  5463 tablet_copy_client.cc:323] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.195:41453
I20260709 12:24:18.595657  3408 tablet_copy_service.cc:140] P c93989d3dd634ef585d77b03354121a4: Received BeginTabletCopySession request for tablet 11921fa3d80340c18f076a4d4499340b from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:34216)
I20260709 12:24:18.596290  3408 tablet_copy_service.cc:161] P c93989d3dd634ef585d77b03354121a4: Beginning new tablet copy session on tablet 11921fa3d80340c18f076a4d4499340b from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:34216: session id = e38cc90af94c47f8a86be67ad610cbd4-11921fa3d80340c18f076a4d4499340b
I20260709 12:24:18.612651  3408 tablet_copy_source_session.cc:215] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:18.616586  5463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11921fa3d80340c18f076a4d4499340b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:18.643476  5463 tablet_copy_client.cc:806] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:18.644258  5463 tablet_copy_client.cc:670] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:18.648754  5463 tablet_copy_client.cc:538] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:18.658475  5463 tablet_bootstrap.cc:492] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:24:18.682943  5463 tablet_bootstrap.cc:492] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:18.683718  5463 tablet_bootstrap.cc:492] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:24:18.684366  5463 ts_tablet_manager.cc:1403] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.004s
I20260709 12:24:18.687780  5463 raft_consensus.cc:359] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:18.689163  5463 raft_consensus.cc:740] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:24:18.689910  5463 consensus_queue.cc:260] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:18.698190  5463 ts_tablet_manager.cc:1434] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.004s
I20260709 12:24:18.700034  3408 tablet_copy_service.cc:342] P c93989d3dd634ef585d77b03354121a4: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-11921fa3d80340c18f076a4d4499340b received from {username='slave'} at 127.0.0.1:34216
I20260709 12:24:18.700522  3408 tablet_copy_service.cc:434] P c93989d3dd634ef585d77b03354121a4: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-11921fa3d80340c18f076a4d4499340b on tablet 11921fa3d80340c18f076a4d4499340b with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:18.728200  3318 raft_consensus.cc:1275] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:18.739400  5335 consensus_queue.cc:1048] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:18.752269  3391 raft_consensus.cc:1275] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:18.755558  5456 consensus_queue.cc:1048] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:18.789669  5463 ts_tablet_manager.cc:933] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4: Initiating tablet copy from peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779)
I20260709 12:24:18.819211  5463 tablet_copy_client.cc:323] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.4.193:37779
I20260709 12:24:18.821007  3259 tablet_copy_service.cc:140] P 7cdaa0234c64450eb4ce4500b51b8a81: Received BeginTabletCopySession request for tablet 84e58c891d3b452eb575499063d5d34f from peer e38cc90af94c47f8a86be67ad610cbd4 ({username='slave'} at 127.0.0.1:57122)
I20260709 12:24:18.821929  3259 tablet_copy_service.cc:161] P 7cdaa0234c64450eb4ce4500b51b8a81: Beginning new tablet copy session on tablet 84e58c891d3b452eb575499063d5d34f from peer e38cc90af94c47f8a86be67ad610cbd4 at {username='slave'} at 127.0.0.1:57122: session id = e38cc90af94c47f8a86be67ad610cbd4-84e58c891d3b452eb575499063d5d34f
I20260709 12:24:18.840165  3259 tablet_copy_source_session.cc:215] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:18.844790  5463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84e58c891d3b452eb575499063d5d34f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:18.893740  5463 tablet_copy_client.cc:806] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:18.894464  5463 tablet_copy_client.cc:670] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:18.901736  5463 tablet_copy_client.cc:538] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:18.912258  5463 tablet_bootstrap.cc:492] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap starting.
I20260709 12:24:18.954965  5463 tablet_bootstrap.cc:492] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:18.956197  5463 tablet_bootstrap.cc:492] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4: Bootstrap complete.
I20260709 12:24:18.956933  5463 ts_tablet_manager.cc:1403] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4: Time spent bootstrapping tablet: real 0.045s	user 0.020s	sys 0.012s
I20260709 12:24:18.960188  5463 raft_consensus.cc:359] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:18.961090  5463 raft_consensus.cc:740] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Initialized, Role: LEARNER
I20260709 12:24:18.961871  5463 consensus_queue.cc:260] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:18.972505  5463 ts_tablet_manager.cc:1434] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.004s
I20260709 12:24:18.984102  3259 tablet_copy_service.cc:342] P 7cdaa0234c64450eb4ce4500b51b8a81: Request end of tablet copy session e38cc90af94c47f8a86be67ad610cbd4-84e58c891d3b452eb575499063d5d34f received from {username='slave'} at 127.0.0.1:57122
I20260709 12:24:18.984592  3259 tablet_copy_service.cc:434] P 7cdaa0234c64450eb4ce4500b51b8a81: ending tablet copy session e38cc90af94c47f8a86be67ad610cbd4-84e58c891d3b452eb575499063d5d34f on tablet 84e58c891d3b452eb575499063d5d34f with peer e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:18.994597  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:19.015466  3991 raft_consensus.cc:1217] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:24:19.100523  3248 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87fddcaae9194bef90d1047a518e45a6"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:24:19.101291  3248 raft_consensus.cc:606] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:19.104708  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:24:19.106891  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:19.109544  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:19.247447  3991 raft_consensus.cc:1217] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:24:19.520308  5473 raft_consensus.cc:1064] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
W20260709 12:24:19.535365  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 80345.5us
I20260709 12:24:19.527369  5473 consensus_queue.cc:237] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:19.613432  5335 raft_consensus.cc:993] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:24:19.615540  5335 raft_consensus.cc:1081] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:24:19.622156  3996 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87fddcaae9194bef90d1047a518e45a6"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:55596
I20260709 12:24:19.623234  3996 raft_consensus.cc:493] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:19.623734  3996 raft_consensus.cc:3060] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:19.649209  3996 raft_consensus.cc:515] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:19.652279  3996 leader_election.cc:290] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:19.654161  3319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fddcaae9194bef90d1047a518e45a6" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:19.655355  3319 raft_consensus.cc:3060] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:19.655967  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fddcaae9194bef90d1047a518e45a6" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:24:19.656926  3248 raft_consensus.cc:3055] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:19.657353  3248 raft_consensus.cc:740] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:19.658753  3248 consensus_queue.cc:260] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:19.661083  3248 raft_consensus.cc:3060] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:19.679270  3319 raft_consensus.cc:2468] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 3.
I20260709 12:24:19.698539  3963 leader_election.cc:304] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:24:19.699862  5398 raft_consensus.cc:2804] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:19.700487  5398 raft_consensus.cc:697] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:24:19.701766  5398 consensus_queue.cc:237] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:19.747684  5335 raft_consensus.cc:1064] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81: attempting to promote NON_VOTER e38cc90af94c47f8a86be67ad610cbd4 to VOTER
I20260709 12:24:19.754199  5335 consensus_queue.cc:237] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:20.005986  3248 raft_consensus.cc:2468] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 3.
W20260709 12:24:20.119880  5480 log.cc:927] Time spent T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4: Append to log took a long time: real 0.139s	user 0.002s	sys 0.004s
I20260709 12:24:20.122879  3994 raft_consensus.cc:1275] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:20.159308  5324 consensus_queue.cc:1048] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:20.261596  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:20.677289  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 659088us
I20260709 12:24:20.748755  3315 raft_consensus.cc:1275] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:20.764864  5493 raft_consensus.cc:493] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:20.765746  5493 raft_consensus.cc:515] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:20.768610  5493 leader_election.cc:290] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:20.819360  3241 raft_consensus.cc:1275] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:20.848383  3145 catalog_manager.cc:5697] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 3 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:20.878525  5401 raft_consensus.cc:493] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:20.879487  5401 raft_consensus.cc:515] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:20.873241  3245 raft_consensus.cc:1275] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:20.881583  5401 leader_election.cc:290] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:20.913960  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 233911us
I20260709 12:24:20.935166  5479 raft_consensus.cc:493] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e38cc90af94c47f8a86be67ad610cbd4)
I20260709 12:24:20.935912  5479 raft_consensus.cc:515] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:20.935922  3323 raft_consensus.cc:1275] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:20.942379  5479 leader_election.cc:290] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:20.945250  3323 raft_consensus.cc:1275] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:20.961158  3994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f76214f21f642889fe174df15bb7047" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:20.962760  5405 raft_consensus.cc:493] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c93989d3dd634ef585d77b03354121a4)
I20260709 12:24:20.963791  5405 raft_consensus.cc:515] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:20.967056  5405 leader_election.cc:290] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:20.979035  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a494da53f44c1d9780c046a155769a" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:20.988366  5476 consensus_queue.cc:1048] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 12:24:20.995805  3321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d639c2cd28f9418682dc27d0b0ba86b2" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:20.997582  5473 consensus_queue.cc:1048] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:21.006541  5324 raft_consensus.cc:493] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:21.007186  5324 raft_consensus.cc:515] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:21.012915  5462 consensus_queue.cc:1048] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:24:21.017046  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f76214f21f642889fe174df15bb7047" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:21.022661  5473 raft_consensus.cc:493] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:21.025552  3994 raft_consensus.cc:1275] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEARNER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:21.026342  5473 raft_consensus.cc:515] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:21.027173  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:21.030906  5473 leader_election.cc:290] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:21.037987  5401 raft_consensus.cc:493] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:21.038035  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d639c2cd28f9418682dc27d0b0ba86b2" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:21.042687  5401 raft_consensus.cc:515] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:21.044420  5335 consensus_queue.cc:1048] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:21.045295  5401 leader_election.cc:290] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:21.050146  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:24:21.052819  3289 leader_election.cc:304] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:21.059266  5324 leader_election.cc:290] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:21.064710  5478 raft_consensus.cc:2749] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:21.068351  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:21.073127  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:21.073508  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:21.074707  3215 leader_election.cc:304] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:21.078763  3288 leader_election.cc:304] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:21.092829  3245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a494da53f44c1d9780c046a155769a" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:24:21.104753  5456 raft_consensus.cc:2749] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:21.105404  3961 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 81873.5us
I20260709 12:24:21.107283  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:24:21.107348  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:24:21.108582  5498 raft_consensus.cc:493] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:21.099989  5478 raft_consensus.cc:2749] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:21.122876  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:24:21.109186  5498 raft_consensus.cc:515] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:21.137560  5398 raft_consensus.cc:493] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c93989d3dd634ef585d77b03354121a4)
I20260709 12:24:21.170241  5498 leader_election.cc:290] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:21.179742  5477 raft_consensus.cc:493] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c93989d3dd634ef585d77b03354121a4)
I20260709 12:24:21.180464  5477 raft_consensus.cc:515] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: NON_VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: true } }
I20260709 12:24:21.186165  3244 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3af545b01cb24f1cacb0056f804a7692"
dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
mode: GRACEFUL
new_leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:38288
I20260709 12:24:21.187036  3244 raft_consensus.cc:606] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Received request to transfer leadership to TS e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:21.189599  5462 consensus_queue.cc:1048] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:21.191188  5477 leader_election.cc:290] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:21.192236  5507 raft_consensus.cc:493] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:21.193202  5507 raft_consensus.cc:515] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:21.196520  5401 raft_consensus.cc:493] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:21.197240  5401 raft_consensus.cc:515] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:21.198717  3321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:21.199600  5401 leader_election.cc:290] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:21.200201  5401 raft_consensus.cc:493] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:21.200381  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:24:21.177829  5398 raft_consensus.cc:515] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:21.201074  5401 raft_consensus.cc:515] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:21.203492  5398 leader_election.cc:290] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:21.203778  5401 leader_election.cc:290] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:21.204603  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:21.195986  5507 leader_election.cc:290] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:21.207412  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
W20260709 12:24:21.223626  3345 debug-util.cc:398] Leaking SignalData structure 0x7b08001f35c0 after lost signal to thread 2070
W20260709 12:24:21.224869  3345 debug-util.cc:398] Leaking SignalData structure 0x7b08001e6c20 after lost signal to thread 3181
W20260709 12:24:21.226208  3345 debug-util.cc:398] Leaking SignalData structure 0x7b0800113fc0 after lost signal to thread 3273
W20260709 12:24:21.227381  3345 debug-util.cc:398] Leaking SignalData structure 0x7b08001168a0 after lost signal to thread 3348
W20260709 12:24:21.228821  3345 debug-util.cc:398] Leaking SignalData structure 0x7b08002ab480 after lost signal to thread 3422
I20260709 12:24:21.231675  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7048879a65244efa5a140c52d782803" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
W20260709 12:24:21.232124  3345 debug-util.cc:398] Leaking SignalData structure 0x7b0800082da0 after lost signal to thread 4023
I20260709 12:24:21.232101  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:21.235975  5507 raft_consensus.cc:2955] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:21.238974  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:21.239197  3320 raft_consensus.cc:2955] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:24:21.223048  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 76266.2us
I20260709 12:24:21.266604  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:21.272209  3362 leader_election.cc:304] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:21.284235  5324 raft_consensus.cc:2749] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:21.278067  5401 consensus_queue.cc:1048] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:24:21.288727  3962 leader_election.cc:304] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:21.289768  5504 raft_consensus.cc:2749] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.290992  3362 leader_election.cc:304] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:21.291931  5324 raft_consensus.cc:2749] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.293527  3362 leader_election.cc:304] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:21.296041  3362 leader_election.cc:304] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:21.300145  5512 raft_consensus.cc:2749] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:21.305521  3245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7048879a65244efa5a140c52d782803" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:24:21.307154  3244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:24:21.308228  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11921fa3d80340c18f076a4d4499340b" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:21.310035  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:21.325551  3990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff1130a2f274bccbc5b15389deacdc1" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:21.331027  3247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff1130a2f274bccbc5b15389deacdc1" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:24:21.333195  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" is_pre_election: true
I20260709 12:24:21.334086  3391 raft_consensus.cc:1275] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer 7cdaa0234c64450eb4ce4500b51b8a81: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:21.326000  5498 raft_consensus.cc:2749] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:21.349848  5476 raft_consensus.cc:2955] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:21.361794  3321 raft_consensus.cc:2955] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:24:21.406486  3961 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 114168us
I20260709 12:24:21.433254  5428 raft_consensus.cc:993] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
W20260709 12:24:21.442521  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 81202.9us
I20260709 12:24:21.438128  3141 catalog_manager.cc:5697] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:21.444234  5477 raft_consensus.cc:1081] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:24:21.452200  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:21.460246  3997 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3af545b01cb24f1cacb0056f804a7692"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:55596
I20260709 12:24:21.461035  3997 raft_consensus.cc:493] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:21.461426  3997 raft_consensus.cc:3060] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:21.463827  5335 consensus_queue.cc:1048] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:21.465130  3362 leader_election.cc:304] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:21.473021  3362 leader_election.cc:304] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:21.477818  5507 raft_consensus.cc:2749] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:21.483034  3362 leader_election.cc:304] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:21.483942  5324 raft_consensus.cc:2749] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:21.484226  3362 leader_election.cc:304] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:21.489838  3997 raft_consensus.cc:515] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:21.493804  5521 raft_consensus.cc:2749] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:21.494609  5521 raft_consensus.cc:2749] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:21.494827  3997 leader_election.cc:290] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:21.497171  3321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af545b01cb24f1cacb0056f804a7692" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:21.498008  3321 raft_consensus.cc:3060] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:21.520373  3248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af545b01cb24f1cacb0056f804a7692" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7cdaa0234c64450eb4ce4500b51b8a81"
I20260709 12:24:21.523572  3248 raft_consensus.cc:3055] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:21.523875  3248 raft_consensus.cc:740] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cdaa0234c64450eb4ce4500b51b8a81, State: Running, Role: LEADER
I20260709 12:24:21.530102  3248 consensus_queue.cc:260] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:21.536334  3321 raft_consensus.cc:2468] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:24:21.539146  3248 raft_consensus.cc:3060] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:21.541749  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 61398.2us
I20260709 12:24:21.542532  3214 leader_election.cc:304] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:21.542739  3992 raft_consensus.cc:2955] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:21.538681  3963 leader_election.cc:304] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:24:21.570961  5518 raft_consensus.cc:2804] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:21.567493  3995 raft_consensus.cc:2955] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:21.592343  5518 raft_consensus.cc:697] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:24:21.593326  5518 consensus_queue.cc:237] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:21.605701  3248 raft_consensus.cc:2468] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:24:21.663015  3243 raft_consensus.cc:2955] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } attrs { replace: true } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:21.683144  5428 raft_consensus.cc:2749] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:21.682791  2067 tablet_server.cc:179] TabletServer@127.2.4.193:0 shutting down...
W20260709 12:24:21.692734  3961 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
I20260709 12:24:21.699990  3389 raft_consensus.cc:2955] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:24:21.789395  3961 consensus_peers.cc:597] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:21.808336  3362 consensus_peers.cc:597] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:21.820264  3362 consensus_peers.cc:597] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:21.822455  3962 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 124627us
W20260709 12:24:21.828598  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 129388us
I20260709 12:24:21.832748  3141 catalog_manager.cc:5697] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: config changed from index 3 to 4, e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:24:21.879752  3362 consensus_peers.cc:597] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:21.849298  3389 consensus_queue.cc:237] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:21.876021  3146 catalog_manager.cc:5697] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:24:21.890796  3961 consensus_peers.cc:597] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:21.891247  3362 consensus_peers.cc:597] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:21.891495  3961 consensus_peers.cc:597] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:21.909849  3961 consensus_peers.cc:597] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:21.910935  3362 consensus_peers.cc:597] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:21.953720  3322 raft_consensus.cc:1275] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:21.958173  5401 consensus_queue.cc:1048] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:21.959649  3992 raft_consensus.cc:1275] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:24:21.962330  3961 consensus_peers.cc:597] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:21.969614  3288 consensus_peers.cc:597] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:21.971879  3362 consensus_peers.cc:597] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:21.971482  5525 consensus_queue.cc:1048] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
W20260709 12:24:21.976671  3362 consensus_peers.cc:597] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:21.999552  3961 consensus_peers.cc:597] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:21.984234  3288 consensus_peers.cc:597] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:22.006902  3362 consensus_peers.cc:597] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:22.004447  5525 raft_consensus.cc:2955] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:24:22.018465  3362 consensus_peers.cc:597] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:22.015463  3990 raft_consensus.cc:2955] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
I20260709 12:24:22.026751  3321 raft_consensus.cc:1275] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:24:22.010614  3288 consensus_peers.cc:597] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:22.030071  5532 consensus_queue.cc:1048] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:22.033456  3288 consensus_peers.cc:597] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:22.035771  3288 consensus_peers.cc:597] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:22.038014  3288 consensus_peers.cc:597] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:22.039778  3322 raft_consensus.cc:2955] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:24:22.067305  3288 consensus_peers.cc:597] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:22.090966  3141 catalog_manager.cc:5697] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: config changed from index 4 to 5, VOTER 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) evicted. New cstate: current_term: 2 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } } }
W20260709 12:24:22.096200  3132 catalog_manager.cc:4755] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): ChangeConfig:REMOVE_PEER RPC failed for tablet 84e58c891d3b452eb575499063d5d34f: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:24:22.092224  3288 consensus_peers.cc:597] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:22.127578  3961 consensus_peers.cc:597] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:22.129935  3288 consensus_peers.cc:597] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:22.142074  3362 consensus_peers.cc:597] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:22.226363  3133 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 11921fa3d80340c18f076a4d4499340b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:24:22.234331  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 63491us
W20260709 12:24:22.237239  3961 consensus_peers.cc:597] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:22.270260  5524 raft_consensus.cc:493] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c93989d3dd634ef585d77b03354121a4)
I20260709 12:24:22.270961  5524 raft_consensus.cc:515] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:22.276139  5524 leader_election.cc:290] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
W20260709 12:24:22.290828  3288 consensus_peers.cc:597] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:22.293787  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:22.310357  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85f93dbda424e3aa0102bc19a2ff837" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:22.341200  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:22.343833  2067 tablet_replica.cc:333] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:22.346596  2067 raft_consensus.cc:2243] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:22.354337  2067 raft_consensus.cc:2272] T c04a8b7530d5486a880a89a4ba0faac8 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:22.354632  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 115047us
I20260709 12:24:22.358034  2067 tablet_replica.cc:333] T f8a494da53f44c1d9780c046a155769a P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:22.362617  2067 raft_consensus.cc:2243] T f8a494da53f44c1d9780c046a155769a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:22.387202  2067 raft_consensus.cc:2272] T f8a494da53f44c1d9780c046a155769a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:22.389997  3961 consensus_peers.cc:597] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:22.393385  3961 consensus_peers.cc:597] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:22.401445  3213 leader_election.cc:304] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:22.403429  2067 tablet_replica.cc:333] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:22.411304  2067 raft_consensus.cc:2243] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:22.438732  2067 raft_consensus.cc:2272] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:22.421234  5524 raft_consensus.cc:2749] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:22.599524  3288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 84430.6us
W20260709 12:24:22.624500  3214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 123945us
W20260709 12:24:22.626940  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 54284.7us
W20260709 12:24:22.649422  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:45287, user_credentials={real_user=slave}} blocked reactor thread for 136235us
I20260709 12:24:22.686329  2067 tablet_replica.cc:333] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:22.697970  5548 raft_consensus.cc:493] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e38cc90af94c47f8a86be67ad610cbd4)
I20260709 12:24:22.698772  5548 raft_consensus.cc:515] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:22.702517  3992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9cc3d1278674852b3b6c8c39f4f1b59" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:22.712546  2067 raft_consensus.cc:2243] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:22.748853  2067 raft_consensus.cc:2272] T 11921fa3d80340c18f076a4d4499340b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:22.713650  5548 leader_election.cc:290] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
W20260709 12:24:22.757361  3962 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 70251.5us
I20260709 12:24:22.742146  5524 raft_consensus.cc:493] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e38cc90af94c47f8a86be67ad610cbd4)
I20260709 12:24:22.776230  2067 tablet_replica.cc:333] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:22.769773  5524 raft_consensus.cc:515] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:22.777439  2067 raft_consensus.cc:2243] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:22.801143  2067 raft_consensus.cc:2272] T 73563dc7466d4636a64ebd4d363935b1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:22.813373  5524 leader_election.cc:290] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:22.822726  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fddcaae9194bef90d1047a518e45a6" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
W20260709 12:24:22.845376  3215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 72325us
I20260709 12:24:22.891167  2067 tablet_replica.cc:333] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:22.897701  2067 raft_consensus.cc:2243] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:22.897746  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9cc3d1278674852b3b6c8c39f4f1b59" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:22.899012  2067 raft_consensus.cc:2272] T 7ff1130a2f274bccbc5b15389deacdc1 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:22.902235  5565 raft_consensus.cc:493] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:22.902922  5565 raft_consensus.cc:515] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:22.905974  2067 tablet_replica.cc:333] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:22.905375  5565 leader_election.cc:290] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:22.921995  2067 raft_consensus.cc:2243] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:22.922614  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fddcaae9194bef90d1047a518e45a6" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:22.923120  2067 raft_consensus.cc:2272] T 8bcfe15edc4947b988fd204a12c79094 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:22.922675  3214 leader_election.cc:304] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:22.928988  5524 raft_consensus.cc:2749] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:22.932734  5546 raft_consensus.cc:493] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:22.933337  5546 raft_consensus.cc:515] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:22.937789  5546 leader_election.cc:290] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:22.942277  2067 tablet_replica.cc:333] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:22.952105  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:22.965240  2067 raft_consensus.cc:2243] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:22.953225  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:22.966383  2067 raft_consensus.cc:2272] T 6b70b620cf8243f4b39cf03eaae44881 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:22.959687  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076c82502ea24caf81503a8f13fadf97" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:22.975545  2067 tablet_replica.cc:333] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:22.976547  2067 raft_consensus.cc:2243] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:22.977196  2067 raft_consensus.cc:2272] T a7494971ced1402da04932982df157fd P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:22.980039  2067 tablet_replica.cc:333] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:22.980827  2067 raft_consensus.cc:2243] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:22.993290  5546 raft_consensus.cc:493] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e38cc90af94c47f8a86be67ad610cbd4)
I20260709 12:24:22.994019  5546 raft_consensus.cc:515] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:22.997104  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:22.998530  5546 leader_election.cc:290] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:22.999944  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f37527cd3a34753b502172f81b808e0" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:23.002017  3213 leader_election.cc:304] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:23.004488  5565 raft_consensus.cc:493] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c93989d3dd634ef585d77b03354121a4)
I20260709 12:24:23.006628  5565 raft_consensus.cc:515] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:23.010583  5565 leader_election.cc:290] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:23.011956  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd6cee8d33de46388ec38d683a789e94" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:23.013818  5565 raft_consensus.cc:493] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:23.014448  5565 raft_consensus.cc:515] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:23.016166  5544 raft_consensus.cc:2749] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:23.017874  5565 leader_election.cc:290] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:23.020025  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:23.024097  3992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd6cee8d33de46388ec38d683a789e94" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:23.024698  5524 raft_consensus.cc:493] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:23.025318  5524 raft_consensus.cc:515] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:23.027595  5524 leader_election.cc:290] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:23.028589  3213 leader_election.cc:304] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:23.034161  3215 leader_election.cc:304] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:23.034332  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:23.035475  5565 raft_consensus.cc:2749] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:23.038110  2067 raft_consensus.cc:2272] T 0634624f456740b48b77096645ab6b0b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.039446  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a558d863a564ddf9aa7c954dffaeafe" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:23.044734  2067 tablet_replica.cc:333] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.052862  2067 raft_consensus.cc:2243] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:23.049695  5565 raft_consensus.cc:493] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e38cc90af94c47f8a86be67ad610cbd4)
I20260709 12:24:23.053673  5524 raft_consensus.cc:2749] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:23.053682  5544 raft_consensus.cc:493] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c93989d3dd634ef585d77b03354121a4)
I20260709 12:24:23.057006  5544 raft_consensus.cc:515] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:23.060654  5544 leader_election.cc:290] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:23.061069  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36377e446734493c8f3b1ff53d40d1cc" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:23.061378  5565 raft_consensus.cc:515] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:23.066983  5565 leader_election.cc:290] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:23.073570  5494 raft_consensus.cc:493] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c93989d3dd634ef585d77b03354121a4)
I20260709 12:24:23.075393  5494 raft_consensus.cc:515] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:23.078505  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5d938460364390a17cada19308b314" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:23.078878  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:23.080678  5494 leader_election.cc:290] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:23.082211  5524 raft_consensus.cc:493] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:23.082762  5524 raft_consensus.cc:515] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:23.085057  5524 leader_election.cc:290] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:23.086752  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b0ee883b3374a4eb86de5773524325b" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:23.092216  5524 raft_consensus.cc:493] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:23.092818  5544 raft_consensus.cc:493] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c93989d3dd634ef585d77b03354121a4)
I20260709 12:24:23.093436  5548 raft_consensus.cc:493] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c93989d3dd634ef585d77b03354121a4)
I20260709 12:24:23.093586  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb1e1f3b50843ffbc685b3a019ea8cb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:23.095265  5548 raft_consensus.cc:515] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:23.092820  5524 raft_consensus.cc:515] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:23.097592  5524 leader_election.cc:290] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:23.098048  5548 leader_election.cc:290] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:23.098618  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9b725784044c8da903f092febf7c05" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:23.099334  3213 leader_election.cc:304] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:23.100404  5548 raft_consensus.cc:2749] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:23.099228  5544 raft_consensus.cc:515] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:23.101527  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d639c2cd28f9418682dc27d0b0ba86b2" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:23.103976  5544 leader_election.cc:290] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:23.104844  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:23.108116  5546 raft_consensus.cc:493] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:23.117187  5546 raft_consensus.cc:515] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:23.074447  2067 raft_consensus.cc:2272] T 3098f015fb154cf490e147a8e71f65b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.110970  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9145effc86e4e0db3de0ff579cd939d" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:23.113610  3215 leader_election.cc:304] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:23.123678  5548 raft_consensus.cc:2749] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.125563  3215 leader_election.cc:304] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:23.126526  5548 raft_consensus.cc:2749] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:23.131326  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118e1f84035044e1b2df505c59a79fdb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:23.134069  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:23.129493  2067 tablet_replica.cc:333] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.141041  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8131bbfd8374661bd9731b4a57468ee" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:23.143998  2067 raft_consensus.cc:2243] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:23.145468  2067 raft_consensus.cc:2272] T 00dbb3c83d794b5db75532d598235d9a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.132867  5546 leader_election.cc:290] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:23.133327  3213 leader_election.cc:304] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:23.151224  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36377e446734493c8f3b1ff53d40d1cc" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:23.157074  5546 raft_consensus.cc:2749] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:23.165723  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a5d938460364390a17cada19308b314" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:23.169670  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c9b725784044c8da903f092febf7c05" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:23.172402  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b0ee883b3374a4eb86de5773524325b" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:23.175855  2067 tablet_replica.cc:333] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.176671  2067 raft_consensus.cc:2243] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:23.177606  2067 raft_consensus.cc:2272] T 16f6e94aa295499cb8358daafcb934b6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.179188  5548 raft_consensus.cc:493] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e38cc90af94c47f8a86be67ad610cbd4)
I20260709 12:24:23.179699  5548 raft_consensus.cc:515] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:23.182039  5548 leader_election.cc:290] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:23.182530  5548 raft_consensus.cc:493] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c93989d3dd634ef585d77b03354121a4)
I20260709 12:24:23.183163  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d639c2cd28f9418682dc27d0b0ba86b2" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:23.183380  5548 raft_consensus.cc:515] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:23.185648  5548 leader_election.cc:290] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:23.186968  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d7a917f21741e9827c30e5433a38e3" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:23.187150  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:23.189865  5544 raft_consensus.cc:493] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e38cc90af94c47f8a86be67ad610cbd4)
I20260709 12:24:23.190526  5544 raft_consensus.cc:515] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:23.192927  3215 leader_election.cc:304] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:23.193779  5548 raft_consensus.cc:493] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:23.194334  5548 raft_consensus.cc:515] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:23.196233  5546 raft_consensus.cc:493] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:23.196811  5546 raft_consensus.cc:515] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:23.196485  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412da949752f49049ec6148896078f29" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:23.200824  5546 leader_election.cc:290] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:23.201388  5546 raft_consensus.cc:2749] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:23.198092  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6d2799afd04cbe9bfaa74d467e66cb" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:23.202113  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2be382cd47e41839958b59bc8e3c40a" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:23.202862  5548 leader_election.cc:290] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:23.205286  5544 leader_election.cc:290] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:23.209111  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d7a917f21741e9827c30e5433a38e3" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:23.229167  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba9db97635f4d28bf0f3079edf45a77" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:23.230397  3215 leader_election.cc:304] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:23.231246  5544 raft_consensus.cc:2749] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.231766  3215 leader_election.cc:304] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:23.215282  2067 tablet_replica.cc:333] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.232995  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d97394c304e81affa5ee75b52de28" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:23.233942  2067 raft_consensus.cc:2243] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.234498  5544 raft_consensus.cc:2706] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Received pre-election callback for term 2 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 12:24:23.233013  3215 leader_election.cc:304] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:23.235399  2067 raft_consensus.cc:2272] T 36377e446734493c8f3b1ff53d40d1cc P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.238591  2067 tablet_replica.cc:333] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.239645  2067 raft_consensus.cc:2243] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.240293  2067 raft_consensus.cc:2272] T f85f93dbda424e3aa0102bc19a2ff837 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.243237  2067 tablet_replica.cc:333] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.244083  2067 raft_consensus.cc:2243] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:23.245316  2067 raft_consensus.cc:2272] T e2884104b65f4a589e915fd4328b729c P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.246290  5544 raft_consensus.cc:2749] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:23.246510  3215 leader_election.cc:304] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:23.247457  5544 raft_consensus.cc:2749] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.247828  3215 leader_election.cc:304] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:23.249168  3215 leader_election.cc:304] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:23.253916  5565 raft_consensus.cc:2749] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:23.254498  3215 leader_election.cc:304] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:23.255858  3215 leader_election.cc:304] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:23.256428  5548 raft_consensus.cc:2749] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.257156  5548 raft_consensus.cc:2749] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:23.254678  5565 raft_consensus.cc:2749] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:23.257419  3215 leader_election.cc:304] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:23.269435  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d97394c304e81affa5ee75b52de28" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:23.270309  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2be382cd47e41839958b59bc8e3c40a" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:23.270754  5544 raft_consensus.cc:2749] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:23.278460  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ba9db97635f4d28bf0f3079edf45a77" candidate_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:23.279059  2067 tablet_replica.cc:333] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.292301  3215 leader_election.cc:304] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4
W20260709 12:24:23.293457  3132 catalog_manager.cc:4755] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): ChangeConfig:REMOVE_PEER RPC failed for tablet 84e58c891d3b452eb575499063d5d34f: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:24:23.294471  3215 leader_election.cc:304] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:23.297133  5565 raft_consensus.cc:2749] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:23.298892  2067 raft_consensus.cc:2243] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:23.299070  3215 leader_election.cc:304] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cdaa0234c64450eb4ce4500b51b8a81; no voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:23.293673  5544 raft_consensus.cc:2749] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:23.304507  2067 raft_consensus.cc:2272] T b7048879a65244efa5a140c52d782803 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.300851  5565 raft_consensus.cc:2749] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:23.323941  2067 tablet_replica.cc:333] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.324983  2067 raft_consensus.cc:2243] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.325703  2067 raft_consensus.cc:2272] T c9145effc86e4e0db3de0ff579cd939d P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.339571  2067 tablet_replica.cc:333] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.340521  2067 raft_consensus.cc:2243] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:23.341730  2067 raft_consensus.cc:2272] T d373e8b022274a08b08f00c0057ae3d3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.351809  2067 tablet_replica.cc:333] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.353758  2067 raft_consensus.cc:2243] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.354411  2067 raft_consensus.cc:2272] T 6c9b725784044c8da903f092febf7c05 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.365175  2067 tablet_replica.cc:333] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.366050  2067 raft_consensus.cc:2243] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.366638  2067 raft_consensus.cc:2272] T 572d97394c304e81affa5ee75b52de28 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.369458  2067 tablet_replica.cc:333] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.381131  2067 raft_consensus.cc:2243] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.381884  2067 raft_consensus.cc:2272] T e8d7a917f21741e9827c30e5433a38e3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.385138  2067 tablet_replica.cc:333] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.386919  2067 raft_consensus.cc:2243] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.388113  2067 raft_consensus.cc:2272] T 5ba9db97635f4d28bf0f3079edf45a77 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.416761  2067 tablet_replica.cc:333] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.430266  2067 raft_consensus.cc:2243] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.431318  2067 raft_consensus.cc:2272] T b8131bbfd8374661bd9731b4a57468ee P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.434422  2067 tablet_replica.cc:333] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.435407  2067 raft_consensus.cc:2243] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.436147  2067 raft_consensus.cc:2272] T 076c82502ea24caf81503a8f13fadf97 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.439978  2067 tablet_replica.cc:333] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.440961  2067 raft_consensus.cc:2243] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.441847  2067 raft_consensus.cc:2272] T 0a5d938460364390a17cada19308b314 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.445170  2067 tablet_replica.cc:333] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.446221  2067 raft_consensus.cc:2243] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:23.447992  2067 raft_consensus.cc:2272] T 84e58c891d3b452eb575499063d5d34f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.459297  2067 tablet_replica.cc:333] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.460408  2067 raft_consensus.cc:2243] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.463265  2067 raft_consensus.cc:2272] T d639c2cd28f9418682dc27d0b0ba86b2 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.466611  2067 tablet_replica.cc:333] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.467367  2067 raft_consensus.cc:2243] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.467952  2067 raft_consensus.cc:2272] T 87fddcaae9194bef90d1047a518e45a6 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.470806  2067 tablet_replica.cc:333] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.477350  2067 raft_consensus.cc:2243] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.479310  2067 raft_consensus.cc:2272] T 2a558d863a564ddf9aa7c954dffaeafe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.482549  2067 tablet_replica.cc:333] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.483384  2067 raft_consensus.cc:2243] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:23.484455  2067 raft_consensus.cc:2272] T 054906db16c94967bdef7c9dd5a47ea3 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.487576  2067 tablet_replica.cc:333] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.488184  2067 raft_consensus.cc:2243] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:23.489658  2067 raft_consensus.cc:2272] T dfabf95aa4ee4eae966abca58130bd91 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.493703  2067 tablet_replica.cc:333] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.495326  2067 raft_consensus.cc:2243] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.496107  2067 raft_consensus.cc:2272] T bfb1e1f3b50843ffbc685b3a019ea8cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.499696  2067 tablet_replica.cc:333] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.500553  2067 raft_consensus.cc:2243] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.501149  2067 raft_consensus.cc:2272] T 3af545b01cb24f1cacb0056f804a7692 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.504330  2067 tablet_replica.cc:333] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.505189  2067 raft_consensus.cc:2243] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.505854  2067 raft_consensus.cc:2272] T 9f37527cd3a34753b502172f81b808e0 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.508510  2067 tablet_replica.cc:333] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.509302  2067 raft_consensus.cc:2243] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.509897  2067 raft_consensus.cc:2272] T 118e1f84035044e1b2df505c59a79fdb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.513358  2067 tablet_replica.cc:333] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.514180  2067 raft_consensus.cc:2243] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.514804  2067 raft_consensus.cc:2272] T a2be382cd47e41839958b59bc8e3c40a P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.518541  2067 tablet_replica.cc:333] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.519373  2067 raft_consensus.cc:2243] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.520025  2067 raft_consensus.cc:2272] T 412da949752f49049ec6148896078f29 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.523109  2067 tablet_replica.cc:333] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.523957  2067 raft_consensus.cc:2243] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:23.525092  2067 raft_consensus.cc:2272] T af3368aec5b1412797545abe458a95fe P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.528272  2067 tablet_replica.cc:333] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.529109  2067 raft_consensus.cc:2243] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.529699  2067 raft_consensus.cc:2272] T 1b0ee883b3374a4eb86de5773524325b P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.532136  2067 tablet_replica.cc:333] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.532837  2067 raft_consensus.cc:2243] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:23.534005  2067 raft_consensus.cc:2272] T dbb481337ada4c83945e213fb117ee7f P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.537591  2067 tablet_replica.cc:333] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.538491  2067 raft_consensus.cc:2243] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.539165  2067 raft_consensus.cc:2272] T a9cc3d1278674852b3b6c8c39f4f1b59 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.542092  2067 tablet_replica.cc:333] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.542845  2067 raft_consensus.cc:2243] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.543419  2067 raft_consensus.cc:2272] T fd6cee8d33de46388ec38d683a789e94 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.546554  2067 tablet_replica.cc:333] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.547369  2067 raft_consensus.cc:2243] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:23.548368  2067 raft_consensus.cc:2272] T 2bde98a54e9f47a0b9afe40221bcb336 P 7cdaa0234c64450eb4ce4500b51b8a81 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:23.556018  2067 tablet_replica.cc:333] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81: stopping tablet replica
I20260709 12:24:23.556752  2067 raft_consensus.cc:2243] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:23.557353  2067 raft_consensus.cc:2272] T 7a6d2799afd04cbe9bfaa74d467e66cb P 7cdaa0234c64450eb4ce4500b51b8a81 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:23.686645  3214 connection.cc:466] server connection from 127.0.0.1:36494 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36494 (request call id 2507) could send its response
I20260709 12:24:23.878798  5558 raft_consensus.cc:493] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:23.879490  5558 raft_consensus.cc:515] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:23.882153  5558 leader_election.cc:290] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:23.884897  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
W20260709 12:24:23.891611  3362 leader_election.cc:336] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:23.894050  3362 leader_election.cc:304] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:23.895041  5558 raft_consensus.cc:2749] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:24.030238  3961 consensus_peers.cc:597] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:24.201767  3362 consensus_peers.cc:597] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.221166  5568 raft_consensus.cc:493] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.221951  5568 raft_consensus.cc:515] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.224862  5568 leader_election.cc:290] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:24:24.225193  3288 consensus_peers.cc:597] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.226421  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff1130a2f274bccbc5b15389deacdc1" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:24.239842  5569 raft_consensus.cc:493] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.240550  5569 raft_consensus.cc:515] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:24.242738  3961 consensus_peers.cc:597] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.247195  5569 leader_election.cc:290] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:24.254405  5478 raft_consensus.cc:493] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.255110  5478 raft_consensus.cc:515] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.248739  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcfe15edc4947b988fd204a12c79094" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:24.257323  5479 raft_consensus.cc:493] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.257674  5478 leader_election.cc:290] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:24.258100  5479 raft_consensus.cc:515] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:24.260409  5479 leader_election.cc:290] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:24.262390  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:24.264029  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3098f015fb154cf490e147a8e71f65b6" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
W20260709 12:24:24.269348  3961 leader_election.cc:336] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.269989  3961 leader_election.cc:304] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
W20260709 12:24:24.270128  3288 leader_election.cc:336] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.270773  3288 leader_election.cc:304] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:24.270866  5569 raft_consensus.cc:2749] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:24.272243  3288 leader_election.cc:336] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.272682  3288 leader_election.cc:304] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:24.273842  5478 raft_consensus.cc:2749] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:24.274029  5479 raft_consensus.cc:2749] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:24.280869  3961 leader_election.cc:336] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.281433  3961 leader_election.cc:304] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:24.282228  5569 raft_consensus.cc:2749] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:24.314458  3961 consensus_peers.cc:597] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.315333  5569 raft_consensus.cc:493] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.315905  5569 raft_consensus.cc:515] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.318181  5569 leader_election.cc:290] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:24.319286  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2884104b65f4a589e915fd4328b729c" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
W20260709 12:24:24.324911  3288 consensus_peers.cc:597] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:24.325708  3961 leader_election.cc:336] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.326292  3961 leader_election.cc:304] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:24.330152  5576 raft_consensus.cc:2749] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:24.362725  3288 consensus_peers.cc:597] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:24.365320  3362 consensus_peers.cc:597] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:24.371349  3288 consensus_peers.cc:597] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:24.376209  3961 consensus_peers.cc:597] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.397540  5575 raft_consensus.cc:493] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.398204  5575 raft_consensus.cc:515] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:24.399456  3362 consensus_peers.cc:597] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.400398  5575 leader_election.cc:290] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:24.403280  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff1130a2f274bccbc5b15389deacdc1" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:24.404024  3995 raft_consensus.cc:2468] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:24:24.405339  5575 raft_consensus.cc:493] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.406111  5575 raft_consensus.cc:515] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:24.407850  3362 leader_election.cc:304] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:24:24.408296  5575 leader_election.cc:290] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:24.408828  5558 raft_consensus.cc:2804] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:24.409227  5558 raft_consensus.cc:493] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.409582  5558 raft_consensus.cc:3060] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:24.417913  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:24.418747  3320 raft_consensus.cc:2468] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 2.
I20260709 12:24:24.420253  5478 raft_consensus.cc:493] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.420868  5478 raft_consensus.cc:515] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.421384  3364 leader_election.cc:304] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:24:24.422331  5575 raft_consensus.cc:2804] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:24.422825  5575 raft_consensus.cc:493] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.423202  5575 raft_consensus.cc:3060] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:24.427861  5478 leader_election.cc:290] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
W20260709 12:24:24.430238  3362 leader_election.cc:336] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.430871  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcfe15edc4947b988fd204a12c79094" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:24.431622  3995 raft_consensus.cc:2468] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:24:24.433327  3288 leader_election.cc:304] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
W20260709 12:24:24.436870  3288 leader_election.cc:336] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.437328  5478 raft_consensus.cc:2804] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:24.438745  5478 raft_consensus.cc:493] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.439918  5478 raft_consensus.cc:3060] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:24.442797  5575 raft_consensus.cc:515] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:24.460057  5575 leader_election.cc:290] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:24.473933  5579 raft_consensus.cc:493] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.474673  5579 raft_consensus.cc:515] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.476872  5579 leader_election.cc:290] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:24.482400  5558 raft_consensus.cc:515] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.492703  5575 raft_consensus.cc:493] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.497248  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2884104b65f4a589e915fd4328b729c" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
W20260709 12:24:24.501904  3288 consensus_peers.cc:597] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:24.504729  3289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 81926.5us
I20260709 12:24:24.508157  3995 raft_consensus.cc:2468] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 1.
I20260709 12:24:24.507557  5575 raft_consensus.cc:515] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:24.511641  5575 leader_election.cc:290] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:24.515944  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16f6e94aa295499cb8358daafcb934b6" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:24.516672  3316 raft_consensus.cc:3060] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:24.516085  3288 leader_election.cc:336] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.522917  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:24.526808  3288 leader_election.cc:304] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:24.527784  5577 raft_consensus.cc:2804] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:24.528228  5577 raft_consensus.cc:493] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.528692  5577 raft_consensus.cc:3060] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:24.528487  5576 raft_consensus.cc:493] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.530710  5576 raft_consensus.cc:515] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.533267  5576 leader_election.cc:290] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:24.506687  5479 raft_consensus.cc:493] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.534919  5479 raft_consensus.cc:515] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:24.537209  5479 leader_election.cc:290] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:24.538995  5576 raft_consensus.cc:493] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.539194  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7048879a65244efa5a140c52d782803" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:24.539738  5576 raft_consensus.cc:515] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.544325  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:24.545033  3391 raft_consensus.cc:2468] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:24:24.546221  3962 leader_election.cc:304] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
W20260709 12:24:24.546221  3961 leader_election.cc:336] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.547089  5569 raft_consensus.cc:2804] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:24.547888  5569 raft_consensus.cc:493] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.548251  5569 raft_consensus.cc:3060] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:24.548935  3288 leader_election.cc:336] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.549475  3288 leader_election.cc:304] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:24.551268  5579 raft_consensus.cc:2749] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:24.552417  5576 leader_election.cc:290] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:24.563375  3316 raft_consensus.cc:2468] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
I20260709 12:24:24.564620  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3098f015fb154cf490e147a8e71f65b6" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
W20260709 12:24:24.575872  3961 leader_election.cc:336] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.576066  3320 raft_consensus.cc:2468] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 1.
I20260709 12:24:24.568643  5558 leader_election.cc:290] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
W20260709 12:24:24.569015  3288 consensus_peers.cc:597] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.586184  3364 leader_election.cc:304] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:24:24.587201  5558 raft_consensus.cc:2804] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:24.587783  5558 raft_consensus.cc:697] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
W20260709 12:24:24.588956  3288 consensus_peers.cc:597] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:24.573762  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 67518.1us
W20260709 12:24:24.590152  3362 leader_election.cc:336] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.589731  5558 consensus_queue.cc:237] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:24.599282  5569 raft_consensus.cc:515] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.591358  5478 raft_consensus.cc:515] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.573446  5577 raft_consensus.cc:515] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.605214  3992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff1130a2f274bccbc5b15389deacdc1" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:24:24.617871  3992 raft_consensus.cc:3060] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:24.607254  5503 raft_consensus.cc:493] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.622066  5478 leader_election.cc:290] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:24.621994  5503 raft_consensus.cc:515] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:24:24.608319  3362 leader_election.cc:336] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:24.623414  3362 consensus_peers.cc:597] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.623818  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcfe15edc4947b988fd204a12c79094" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:24:24.624333  5503 leader_election.cc:290] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:24.624461  3995 raft_consensus.cc:3060] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:24.594942  3961 consensus_peers.cc:597] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:24.626583  3961 consensus_peers.cc:597] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.634367  3992 raft_consensus.cc:2468] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 2.
W20260709 12:24:24.636303  3961 consensus_peers.cc:597] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:24.643069  3961 leader_election.cc:336] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.636515  3995 raft_consensus.cc:2468] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:24:24.637252  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d373e8b022274a08b08f00c0057ae3d3" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:24.649677  5569 leader_election.cc:290] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:24.650993  3362 leader_election.cc:304] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:24:24.652213  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00dbb3c83d794b5db75532d598235d9a" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:24.652930  3391 raft_consensus.cc:3060] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:24.655143  5575 raft_consensus.cc:2804] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:24.655777  5575 raft_consensus.cc:697] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
W20260709 12:24:24.655922  3362 consensus_peers.cc:597] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:24.656520  3362 leader_election.cc:336] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.656929  3362 leader_election.cc:304] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:24.656821  5575 consensus_queue.cc:237] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:24.637818  3288 leader_election.cc:336] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.614763  3963 leader_election.cc:304] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:24.657927  5503 raft_consensus.cc:493] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.662200  5503 raft_consensus.cc:515] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.661374  3288 leader_election.cc:304] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:24.665077  5503 leader_election.cc:290] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:24.666177  5569 raft_consensus.cc:2804] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:24.666738  5569 raft_consensus.cc:493] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.666805  5578 raft_consensus.cc:2804] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:24.667292  5569 raft_consensus.cc:3060] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:24.667955  5578 raft_consensus.cc:697] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:24:24.667037  3391 raft_consensus.cc:2468] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 3.
I20260709 12:24:24.668529  5503 raft_consensus.cc:2749] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:24.670389  5578 consensus_queue.cc:237] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:24.666292  3963 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 51632.3us
I20260709 12:24:24.675120  5569 raft_consensus.cc:515] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.681646  5581 raft_consensus.cc:493] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.682292  5581 raft_consensus.cc:515] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:24.682626  3288 leader_election.cc:336] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:24.683449  3288 consensus_peers.cc:597] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.684715  5581 leader_election.cc:290] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:24.686069  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2884104b65f4a589e915fd4328b729c" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
W20260709 12:24:24.686578  3961 leader_election.cc:336] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.686733  3995 raft_consensus.cc:3060] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:24.690390  3961 consensus_peers.cc:597] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.636387  5577 leader_election.cc:290] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
W20260709 12:24:24.694446  3961 leader_election.cc:336] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.698138  5581 raft_consensus.cc:493] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.698266  5569 leader_election.cc:290] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:24.698886  5581 raft_consensus.cc:515] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.700834  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054906db16c94967bdef7c9dd5a47ea3" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:24.713593  5581 leader_election.cc:290] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:24:24.715206  3961 consensus_peers.cc:597] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.719506  3963 leader_election.cc:304] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
W20260709 12:24:24.726025  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 69516.6us
I20260709 12:24:24.726491  3320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
W20260709 12:24:24.726643  3362 consensus_peers.cc:597] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:24.728757  3362 leader_election.cc:336] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.726164  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3098f015fb154cf490e147a8e71f65b6" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:24.732590  3992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:24.732838  3316 raft_consensus.cc:3060] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:24.726655  5581 raft_consensus.cc:2749] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:24.733270  3992 raft_consensus.cc:2468] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 2.
W20260709 12:24:24.738330  3961 leader_election.cc:336] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.739005  3961 leader_election.cc:304] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
W20260709 12:24:24.744438  3362 leader_election.cc:336] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.745118  3362 leader_election.cc:304] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
W20260709 12:24:24.746625  3362 consensus_peers.cc:597] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.748445  5585 raft_consensus.cc:2749] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:24.749504  3362 leader_election.cc:336] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:24.750133  3362 consensus_peers.cc:597] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.756013  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:24.756116  3962 leader_election.cc:304] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
W20260709 12:24:24.757416  3362 consensus_peers.cc:597] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:24.757534  3288 consensus_peers.cc:597] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.760731  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:24.761487  3395 raft_consensus.cc:2468] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:24:24.763532  3364 leader_election.cc:304] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:24.717144  3995 raft_consensus.cc:2468] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:24:24.765090  5581 raft_consensus.cc:2749] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:24.767896  3288 leader_election.cc:304] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:24.769219  5578 raft_consensus.cc:2804] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:24.769821  5578 raft_consensus.cc:697] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:24:24.770663  5578 consensus_queue.cc:237] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.769207  5581 raft_consensus.cc:2804] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:24.773379  5581 raft_consensus.cc:697] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:24:24.779820  5581 consensus_queue.cc:237] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.777105  3141 catalog_manager.cc:5697] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:24.783376  5577 raft_consensus.cc:493] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.784062  5577 raft_consensus.cc:515] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.787611  5577 leader_election.cc:290] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:24.788616  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3368aec5b1412797545abe458a95fe" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
W20260709 12:24:24.791811  3288 leader_election.cc:336] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.792364  3288 leader_election.cc:304] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:24.794013  5478 raft_consensus.cc:2749] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:24.810695  5583 raft_consensus.cc:2749] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:24.811542  5583 raft_consensus.cc:493] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.812007  5583 raft_consensus.cc:515] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:24:24.814224  3364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:33649, user_credentials={real_user=slave}} blocked reactor thread for 50787us
I20260709 12:24:24.814810  5583 leader_election.cc:290] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:24.817099  3145 catalog_manager.cc:5697] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:24.821274  5478 raft_consensus.cc:493] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.822664  5577 raft_consensus.cc:493] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.823218  5577 raft_consensus.cc:515] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:24:24.826332  3362 leader_election.cc:336] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.831280  3322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7048879a65244efa5a140c52d782803" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:24.832013  3322 raft_consensus.cc:2468] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 2.
I20260709 12:24:24.832525  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:24.833160  3391 raft_consensus.cc:2468] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:24:24.835078  3289 leader_election.cc:304] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 
I20260709 12:24:24.835980  5579 raft_consensus.cc:2804] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:24.836428  5579 raft_consensus.cc:493] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.836790  5579 raft_consensus.cc:3060] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:24.829098  3316 raft_consensus.cc:2468] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:24:24.825546  5577 leader_election.cc:290] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:24:24.853327  3288 leader_election.cc:336] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.854579  5579 raft_consensus.cc:515] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:24.861045  3963 leader_election.cc:304] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:24.863099  5568 raft_consensus.cc:2804] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:24.863607  5568 raft_consensus.cc:697] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:24:24.864512  5568 consensus_queue.cc:237] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.822074  5478 raft_consensus.cc:515] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:24.867946  5580 raft_consensus.cc:493] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.868623  5580 raft_consensus.cc:515] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.870998  5580 leader_election.cc:290] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
W20260709 12:24:24.873349  3362 consensus_peers.cc:597] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.874486  3992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054906db16c94967bdef7c9dd5a47ea3" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:24.875208  3992 raft_consensus.cc:2468] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:24:24.882449  5578 raft_consensus.cc:493] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
W20260709 12:24:24.883679  3961 consensus_peers.cc:597] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:24.883203  5578 raft_consensus.cc:515] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.889550  3364 leader_election.cc:304] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:24.887899  5478 leader_election.cc:290] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:24.888334  3288 leader_election.cc:304] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:24:24.897130  5580 raft_consensus.cc:2804] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:24.897585  5580 raft_consensus.cc:493] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.897998  5580 raft_consensus.cc:3060] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:24.898492  5583 raft_consensus.cc:2804] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:24.899019  5583 raft_consensus.cc:493] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.899412  5583 raft_consensus.cc:3060] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:24.906916  3288 consensus_peers.cc:597] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:24.909709  3288 leader_election.cc:336] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.921284  5579 leader_election.cc:290] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:24.906157  5580 raft_consensus.cc:515] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.928295  5580 leader_election.cc:290] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:24.912425  5583 raft_consensus.cc:515] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:24.917876  5503 raft_consensus.cc:493] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.929347  5503 raft_consensus.cc:515] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.931664  5503 leader_election.cc:290] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:24.932916  5568 raft_consensus.cc:493] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.935262  3992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bde98a54e9f47a0b9afe40221bcb336" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:24.938077  3997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054906db16c94967bdef7c9dd5a47ea3" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:24:24.938723  3997 raft_consensus.cc:3060] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:24.940495  5568 raft_consensus.cc:515] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.918496  2067 tablet_server.cc:196] TabletServer@127.2.4.193:0 shutdown complete.
I20260709 12:24:24.917450  3140 catalog_manager.cc:5697] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 3 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:24.938162  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:24.942302  3995 raft_consensus.cc:2468] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:24:24.907889  5578 leader_election.cc:290] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:24.943325  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d373e8b022274a08b08f00c0057ae3d3" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:24.944031  3391 raft_consensus.cc:2468] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:24:24.949371  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:24.950071  3391 raft_consensus.cc:2468] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 2.
I20260709 12:24:24.951239  3391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfabf95aa4ee4eae966abca58130bd91" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
W20260709 12:24:24.951401  3288 leader_election.cc:336] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.951824  3391 raft_consensus.cc:3060] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:24.952667  3997 raft_consensus.cc:2468] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 3.
I20260709 12:24:24.954512  5568 leader_election.cc:290] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:24.961447  3391 raft_consensus.cc:2468] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 3.
W20260709 12:24:24.956190  3362 leader_election.cc:336] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:24.962458  3362 leader_election.cc:336] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.962882  3362 leader_election.cc:304] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:24.963308  3289 leader_election.cc:304] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:24.964224  5586 raft_consensus.cc:2749] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:24.965332  5577 raft_consensus.cc:2804] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:24.965782  5577 raft_consensus.cc:493] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.966187  5577 raft_consensus.cc:3060] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:24.966323  5583 leader_election.cc:290] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:24.967456  3288 leader_election.cc:304] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
W20260709 12:24:24.968593  3961 leader_election.cc:336] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.970271  5479 raft_consensus.cc:2804] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:24:24.971181  3288 leader_election.cc:336] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:24.971783  3288 leader_election.cc:336] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.971889  3289 leader_election.cc:304] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
W20260709 12:24:24.972265  3288 leader_election.cc:336] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:24.976447  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbb481337ada4c83945e213fb117ee7f" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:24.990864  5503 raft_consensus.cc:493] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:24.991609  5503 raft_consensus.cc:515] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:24.995986  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7048879a65244efa5a140c52d782803" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:24.996332  5503 leader_election.cc:290] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:24.996744  3316 raft_consensus.cc:3060] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:24.996963  3997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbb481337ada4c83945e213fb117ee7f" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:24.997646  3997 raft_consensus.cc:2468] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 1.
I20260709 12:24:25.001129  3289 leader_election.cc:304] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:25.005530  3362 leader_election.cc:304] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:24:25.012377  5601 raft_consensus.cc:2804] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:25.005767  5479 raft_consensus.cc:697] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:24:25.017736  5604 raft_consensus.cc:2804] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:25.014235  5586 raft_consensus.cc:2804] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:25.040469  3316 raft_consensus.cc:2468] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
I20260709 12:24:25.034773  5601 raft_consensus.cc:493] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:25.061977  3364 leader_election.cc:304] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:25.041620  5569 raft_consensus.cc:493] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
W20260709 12:24:25.065233  3962 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 78518.7us
I20260709 12:24:25.045179  5586 raft_consensus.cc:493] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:25.067040  5586 raft_consensus.cc:3060] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:25.071117  5503 raft_consensus.cc:2804] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:25.071684  5503 raft_consensus.cc:697] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:25.072531  5503 consensus_queue.cc:237] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:25.073861  5479 consensus_queue.cc:237] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:25.061800  5601 raft_consensus.cc:3060] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:25.079524  3132 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 102312us
I20260709 12:24:25.057010  5604 raft_consensus.cc:697] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:24:25.058512  5577 raft_consensus.cc:515] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:25.100066  5604 consensus_queue.cc:237] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:25.089062  5581 raft_consensus.cc:493] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:25.104084  5581 raft_consensus.cc:515] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:25.083220  5569 raft_consensus.cc:515] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:25.119841  5581 leader_election.cc:290] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:25.123458  5569 leader_election.cc:290] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:25.129489  3132 catalog_manager.cc:4755] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 11921fa3d80340c18f076a4d4499340b: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:25.140930  3961 leader_election.cc:336] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:25.141687  3961 leader_election.cc:336] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:25.132586  5586 raft_consensus.cc:515] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:25.155690  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3368aec5b1412797545abe458a95fe" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4" is_pre_election: true
I20260709 12:24:25.156610  3316 raft_consensus.cc:2468] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 1.
I20260709 12:24:25.156495  5601 raft_consensus.cc:515] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:25.157145  5577 leader_election.cc:290] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:25.158329  3963 leader_election.cc:304] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:25.158985  3395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d373e8b022274a08b08f00c0057ae3d3" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:25.159660  3395 raft_consensus.cc:3060] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:25.169305  5601 leader_election.cc:290] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:25.172149  3140 catalog_manager.cc:5697] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:25.181437  5586 leader_election.cc:290] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:25.184289  3962 leader_election.cc:304] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:25.185586  5605 raft_consensus.cc:2804] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:25.194561  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:25.198840  5605 raft_consensus.cc:493] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:25.199105  3389 raft_consensus.cc:3060] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:25.199487  5605 raft_consensus.cc:3060] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:25.201498  5568 raft_consensus.cc:2749] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:25.183377  3362 leader_election.cc:336] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:25.190970  3395 raft_consensus.cc:2468] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 3.
I20260709 12:24:25.208837  3389 raft_consensus.cc:2468] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 3.
W20260709 12:24:25.210810  3288 consensus_peers.cc:597] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:25.211987  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bde98a54e9f47a0b9afe40221bcb336" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:25.212807  3398 raft_consensus.cc:2468] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
W20260709 12:24:25.214399  3288 leader_election.cc:336] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:25.215058  3288 leader_election.cc:336] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:25.226343  3995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbb481337ada4c83945e213fb117ee7f" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:24:25.227027  3995 raft_consensus.cc:3060] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:25.234759  3995 raft_consensus.cc:2468] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 2.
W20260709 12:24:25.234257  3288 consensus_peers.cc:597] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:25.236919  3997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "8525f77f43714a58bf158d872c1db3e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:24:25.237689  3997 raft_consensus.cc:3060] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:25.248152  3962 leader_election.cc:304] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:25.251484  3995 raft_consensus.cc:1275] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:25.256794  3289 leader_election.cc:304] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
W20260709 12:24:25.254678  3362 consensus_peers.cc:597] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:25.258786  5604 raft_consensus.cc:2804] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:25.259413  5604 raft_consensus.cc:697] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:24:25.260241  5604 consensus_queue.cc:237] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:25.269220  3320 raft_consensus.cc:1275] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:25.255019  3997 raft_consensus.cc:2468] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8525f77f43714a58bf158d872c1db3e4 in term 3.
I20260709 12:24:25.270921  5569 raft_consensus.cc:2804] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:25.271231  3140 catalog_manager.cc:5697] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:25.274437  5569 raft_consensus.cc:493] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7cdaa0234c64450eb4ce4500b51b8a81)
I20260709 12:24:25.275008  5569 raft_consensus.cc:3060] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:25.274868  5608 consensus_queue.cc:1048] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:25.275468  3140 catalog_manager.cc:5697] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 3 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:25.277487  3140 catalog_manager.cc:5697] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 3 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:25.298564  5605 raft_consensus.cc:515] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:25.314594  5479 consensus_queue.cc:1048] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:25.316365  3398 raft_consensus.cc:1275] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:25.317914  3288 leader_election.cc:304] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:25.318823  5479 raft_consensus.cc:2804] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:25.319676  5479 raft_consensus.cc:697] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Becoming Leader. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:24:25.320725  5479 consensus_queue.cc:237] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:25.322304  5605 leader_election.cc:290] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:25.324765  3961 leader_election.cc:336] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:25.324848  3316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af3368aec5b1412797545abe458a95fe" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
I20260709 12:24:25.325829  3316 raft_consensus.cc:3060] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:25.331840  3362 leader_election.cc:304] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:24:25.332918  3990 raft_consensus.cc:1275] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:25.338958  5586 raft_consensus.cc:2804] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:25.339561  5586 raft_consensus.cc:697] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:25.340621  5586 consensus_queue.cc:237] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:25.364148  5479 consensus_queue.cc:1048] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:25.378223  3961 consensus_peers.cc:597] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:25.381091  5581 consensus_queue.cc:1048] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:25.384330  5569 raft_consensus.cc:515] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:25.389503  3316 raft_consensus.cc:2468] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:24:25.390973  3963 leader_election.cc:304] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:25.392618  5605 raft_consensus.cc:2804] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:25.393777  5605 raft_consensus.cc:697] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:24:25.394966  5605 consensus_queue.cc:237] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:25.408775  3961 leader_election.cc:336] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:25.408056  3140 catalog_manager.cc:5697] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 2 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:25.410969  5569 leader_election.cc:290] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:25.416589  3398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bde98a54e9f47a0b9afe40221bcb336" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:25.417302  3398 raft_consensus.cc:3060] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:25.418053  3140 catalog_manager.cc:5697] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:25.432844  3961 consensus_peers.cc:597] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:25.434480  3398 raft_consensus.cc:2468] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 3.
I20260709 12:24:25.427342  3145 catalog_manager.cc:5697] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 3 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:25.441684  3362 leader_election.cc:336] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:25.441713  3962 leader_election.cc:304] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81
I20260709 12:24:25.446662  5606 raft_consensus.cc:2804] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:25.448702  5606 raft_consensus.cc:697] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:24:25.449857  5606 consensus_queue.cc:237] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:25.442040  3145 catalog_manager.cc:5697] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194). New cstate: current_term: 3 leader_uuid: "8525f77f43714a58bf158d872c1db3e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:25.490037  3991 raft_consensus.cc:1275] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:25.490721  3320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:40236
I20260709 12:24:25.491330  3320 raft_consensus.cc:606] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Received request to transfer leadership
I20260709 12:24:25.509581  3389 raft_consensus.cc:1275] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:25.513862  3320 raft_consensus.cc:1275] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:25.517036  5603 consensus_queue.cc:1048] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:24:25.519282  3362 consensus_peers.cc:597] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:25.519824  5569 consensus_queue.cc:1048] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:24:25.522419  3320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:40236
I20260709 12:24:25.523249  3320 raft_consensus.cc:606] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Received request to transfer leadership
I20260709 12:24:25.525139  5583 consensus_queue.cc:1048] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:25.532171  3320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f"
dest_uuid: "8525f77f43714a58bf158d872c1db3e4"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:40236
I20260709 12:24:25.533138  3320 raft_consensus.cc:606] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Received request to transfer leadership
I20260709 12:24:25.561862  3141 catalog_manager.cc:5697] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:25.569089  3145 catalog_manager.cc:5697] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 1 to 2, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 2 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:25.586582  3195 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:24:25.587213  3195 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 84e58c891d3b452eb575499063d5d34f already in progress
W20260709 12:24:25.587518  3195 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 84e58c891d3b452eb575499063d5d34f already in progress
I20260709 12:24:25.645020  3991 raft_consensus.cc:1275] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:25.655421  3316 raft_consensus.cc:1275] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:25.657289  5583 consensus_queue.cc:1048] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:24:25.669219  3288 consensus_peers.cc:597] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:25.677444  3288 consensus_peers.cc:597] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:25.690050  3362 consensus_peers.cc:597] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:25.714370  3962 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:41453, user_credentials={real_user=slave}} blocked reactor thread for 70474.9us
I20260709 12:24:25.717741  3145 catalog_manager.cc:5697] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 2 to 3, leader changed from 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 3 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:25.757941  3389 raft_consensus.cc:1275] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:25.768482  5604 consensus_queue.cc:1048] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:25.771538  5604 consensus_queue.cc:1048] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:25.797739  3389 raft_consensus.cc:1275] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:25.810899  3997 raft_consensus.cc:1275] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Refusing update from remote peer 8525f77f43714a58bf158d872c1db3e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:25.812094  5604 consensus_queue.cc:1048] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:25.823851  2067 tablet_server.cc:179] TabletServer@127.2.4.194:0 shutting down...
W20260709 12:24:25.831310  3963 consensus_peers.cc:597] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:25.838544  5604 consensus_queue.cc:1048] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:25.850069  3364 consensus_peers.cc:597] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:25.863020  3963 consensus_peers.cc:597] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:25.877060  3961 consensus_peers.cc:597] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:25.877095  3288 consensus_peers.cc:597] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:25.881237  3963 consensus_peers.cc:597] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:25.878417  3961 consensus_peers.cc:597] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:25.894099  3963 consensus_peers.cc:597] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:25.904338  3364 consensus_peers.cc:597] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:25.914256  3963 consensus_peers.cc:597] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:25.915603  3362 consensus_peers.cc:597] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:25.917972  3392 raft_consensus.cc:1275] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:25.920341  5629 consensus_queue.cc:1048] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:25.920828  3993 raft_consensus.cc:1275] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:25.923004  5586 consensus_queue.cc:1048] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:24:25.935036  3364 consensus_peers.cc:597] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:25.942068  5479 raft_consensus.cc:993] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4: : Instructing follower e38cc90af94c47f8a86be67ad610cbd4 to start an election
I20260709 12:24:25.952255  5479 raft_consensus.cc:1081] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Signalling peer e38cc90af94c47f8a86be67ad610cbd4 to start an election
W20260709 12:24:25.961802  3963 consensus_peers.cc:597] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:25.971601  3963 consensus_peers.cc:597] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:25.987149  3993 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f"
dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
 from {username='slave'} at 127.0.0.1:35074
I20260709 12:24:25.987865  3993 raft_consensus.cc:493] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:25.988750  3993 raft_consensus.cc:3060] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:26.002233  3364 consensus_peers.cc:597] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:26.013144  3288 consensus_peers.cc:597] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:26.015317  3364 consensus_peers.cc:597] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:26.018671  3993 raft_consensus.cc:515] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:26.021834  3991 raft_consensus.cc:1240] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Rejecting Update request from peer 8525f77f43714a58bf158d872c1db3e4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:26.027803  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "c93989d3dd634ef585d77b03354121a4"
W20260709 12:24:26.028184  3963 leader_election.cc:336] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:24:26.025610  3993 leader_election.cc:290] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 4 election: Requested vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:26.030714  3389 raft_consensus.cc:3060] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:26.040215  5601 consensus_queue.cc:1059] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
W20260709 12:24:26.057544  3961 leader_election.cc:336] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:26.058141  3961 leader_election.cc:304] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:26.058988  5638 raft_consensus.cc:3055] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:26.059478  5576 raft_consensus.cc:2749] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:26.059500  5638 raft_consensus.cc:740] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8525f77f43714a58bf158d872c1db3e4, State: Running, Role: LEADER
I20260709 12:24:26.061131  5638 consensus_queue.cc:260] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:26.063046  5638 raft_consensus.cc:3060] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:26.067476  3389 raft_consensus.cc:2468] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 4.
W20260709 12:24:26.081918  3963 consensus_peers.cc:597] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:26.110889  3963 consensus_peers.cc:597] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:26.113293  3963 consensus_peers.cc:597] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:26.121119  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:26.122951  2067 tablet_replica.cc:333] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.124006  2067 raft_consensus.cc:2243] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.125566  2067 raft_consensus.cc:2272] T f8a494da53f44c1d9780c046a155769a P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.129266  2067 tablet_replica.cc:333] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.130378  2067 raft_consensus.cc:2243] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.131069  2067 raft_consensus.cc:2272] T 7a6d2799afd04cbe9bfaa74d467e66cb P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.135074  2067 tablet_replica.cc:333] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.136046  2067 raft_consensus.cc:2243] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.139050  2067 raft_consensus.cc:2272] T 8bcfe15edc4947b988fd204a12c79094 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.144063  2067 tablet_replica.cc:333] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.145268  2067 raft_consensus.cc:2243] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.146274  2067 raft_consensus.cc:2272] T 11921fa3d80340c18f076a4d4499340b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.149538  2067 tablet_replica.cc:333] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
W20260709 12:24:26.150192  3364 consensus_peers.cc:597] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:26.150704  2067 raft_consensus.cc:2243] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.151541  2067 raft_consensus.cc:2272] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.154604  2067 tablet_replica.cc:333] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.155548  2067 raft_consensus.cc:2243] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.157073  2067 raft_consensus.cc:2272] T 14193bb0c64048cf8ad561f477695970 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:26.161147  3364 consensus_peers.cc:597] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:26.162551  3364 consensus_peers.cc:597] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:26.163843  3963 consensus_peers.cc:597] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:26.164628  2067 tablet_replica.cc:333] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.165675  2067 raft_consensus.cc:2243] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.166405  2067 raft_consensus.cc:2272] T c04a8b7530d5486a880a89a4ba0faac8 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.169883  2067 tablet_replica.cc:333] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.172777  2067 raft_consensus.cc:2243] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.174847  2067 raft_consensus.cc:2272] T 1f3795979cf042008b3447b47c75de09 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.178982  2067 tablet_replica.cc:333] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.180266  2067 raft_consensus.cc:2243] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.181497  2067 raft_consensus.cc:2272] T bb11c9e1f3b048769b242da5c7ae7cd7 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.187856  2067 tablet_replica.cc:333] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.188907  2067 raft_consensus.cc:2243] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.190876  2067 raft_consensus.cc:2272] T 3098f015fb154cf490e147a8e71f65b6 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.194233  2067 tablet_replica.cc:333] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.195158  2067 raft_consensus.cc:2243] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.196007  2067 raft_consensus.cc:2272] T b8f62d0c16794e8c941849d55bb4bb0b P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.199141  2067 tablet_replica.cc:333] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.200043  2067 raft_consensus.cc:2243] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.200704  2067 raft_consensus.cc:2272] T 16f6e94aa295499cb8358daafcb934b6 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.208360  2067 tablet_replica.cc:333] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.209353  2067 raft_consensus.cc:2243] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.212208  2067 raft_consensus.cc:2272] T 36377e446734493c8f3b1ff53d40d1cc P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.217386  2067 tablet_replica.cc:333] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.218393  2067 raft_consensus.cc:2243] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.219820  2067 raft_consensus.cc:2272] T b7048879a65244efa5a140c52d782803 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.225525  2067 tablet_replica.cc:333] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.226768  2067 raft_consensus.cc:2243] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.228300  2067 raft_consensus.cc:2272] T e2884104b65f4a589e915fd4328b729c P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.231792  2067 tablet_replica.cc:333] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.232709  2067 raft_consensus.cc:2243] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.234496  2067 raft_consensus.cc:2272] T d02f86799bc54984b41ede0f7db226e6 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.237591  2067 tablet_replica.cc:333] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.238471  2067 raft_consensus.cc:2243] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.239614  2067 raft_consensus.cc:2272] T fe4a63fc24af42ca862894414f658194 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.243577  2067 tablet_replica.cc:333] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.244455  2067 raft_consensus.cc:2243] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.245592  2067 raft_consensus.cc:2272] T d373e8b022274a08b08f00c0057ae3d3 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.249481  2067 tablet_replica.cc:333] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
W20260709 12:24:26.250289  3364 consensus_peers.cc:597] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:26.250720  2067 raft_consensus.cc:2243] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.251528  2067 raft_consensus.cc:2272] T 6c9b725784044c8da903f092febf7c05 P 8525f77f43714a58bf158d872c1db3e4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.254593  2067 tablet_replica.cc:333] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.256589  2067 raft_consensus.cc:2243] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:24:26.257042  3364 consensus_peers.cc:597] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:26.257212  2067 raft_consensus.cc:2272] T 572d97394c304e81affa5ee75b52de28 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.260556  2067 tablet_replica.cc:333] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.261473  2067 raft_consensus.cc:2243] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.262198  2067 raft_consensus.cc:2272] T e8d7a917f21741e9827c30e5433a38e3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.265115  2067 tablet_replica.cc:333] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.266378  2067 raft_consensus.cc:2243] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.267133  2067 raft_consensus.cc:2272] T c4a1156be8e3418790f41cd380136b93 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.269964  2067 tablet_replica.cc:333] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.270902  2067 raft_consensus.cc:2243] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.272054  2067 raft_consensus.cc:2272] T 076c82502ea24caf81503a8f13fadf97 P 8525f77f43714a58bf158d872c1db3e4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.275257  2067 tablet_replica.cc:333] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.276160  2067 raft_consensus.cc:2243] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.276978  2067 raft_consensus.cc:2272] T 0a5d938460364390a17cada19308b314 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.279886  2067 tablet_replica.cc:333] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.280794  2067 raft_consensus.cc:2243] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.281812  2067 raft_consensus.cc:2272] T 84e58c891d3b452eb575499063d5d34f P 8525f77f43714a58bf158d872c1db3e4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.285218  2067 tablet_replica.cc:333] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.286141  2067 raft_consensus.cc:2243] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.287245  2067 raft_consensus.cc:2272] T d6a9849f17fd4313ab9ddbb06589c46f P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.290488  2067 tablet_replica.cc:333] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.291505  2067 raft_consensus.cc:2243] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.292598  2067 raft_consensus.cc:2272] T d639c2cd28f9418682dc27d0b0ba86b2 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:26.292614  3963 consensus_peers.cc:597] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:26.295781  2067 tablet_replica.cc:333] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.296806  2067 raft_consensus.cc:2243] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.297485  2067 raft_consensus.cc:2272] T 87fddcaae9194bef90d1047a518e45a6 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.300405  2067 tablet_replica.cc:333] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.301307  2067 raft_consensus.cc:2243] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.302534  2067 raft_consensus.cc:2272] T 5ba9db97635f4d28bf0f3079edf45a77 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.305848  2067 tablet_replica.cc:333] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.306727  2067 raft_consensus.cc:2243] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.308044  2067 raft_consensus.cc:2272] T b8131bbfd8374661bd9731b4a57468ee P 8525f77f43714a58bf158d872c1db3e4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.311911  2067 tablet_replica.cc:333] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.312907  2067 raft_consensus.cc:2243] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.314325  2067 raft_consensus.cc:2272] T 054906db16c94967bdef7c9dd5a47ea3 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:26.316088  3963 consensus_peers.cc:597] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:26.319026  2067 tablet_replica.cc:333] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.327857  2067 raft_consensus.cc:2243] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.330624  2067 raft_consensus.cc:2272] T 2a558d863a564ddf9aa7c954dffaeafe P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.333516  2067 tablet_replica.cc:333] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.334362  2067 raft_consensus.cc:2243] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.335209  2067 raft_consensus.cc:2272] T 3015bb6b71f444dcb17062ac96b48cf5 P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.338043  2067 tablet_replica.cc:333] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.338816  2067 raft_consensus.cc:2243] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.339881  2067 raft_consensus.cc:2272] T caed93336fbb415c82acd602adce7969 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.342707  2067 tablet_replica.cc:333] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.343449  2067 raft_consensus.cc:2243] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.344422  2067 raft_consensus.cc:2272] T dfabf95aa4ee4eae966abca58130bd91 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.347749  2067 tablet_replica.cc:333] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.348556  2067 raft_consensus.cc:2243] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.349152  2067 raft_consensus.cc:2272] T 9253ca0fa41f43a7ac13ff17879c2ed3 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.352025  2067 tablet_replica.cc:333] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.352725  2067 raft_consensus.cc:2243] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.353744  2067 raft_consensus.cc:2272] T 118e1f84035044e1b2df505c59a79fdb P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.356297  2067 tablet_replica.cc:333] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.357034  2067 raft_consensus.cc:2243] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.357813  2067 raft_consensus.cc:2272] T 3af545b01cb24f1cacb0056f804a7692 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.360404  2067 tablet_replica.cc:333] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.361104  2067 raft_consensus.cc:2243] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.361775  2067 raft_consensus.cc:2272] T 1d167141acd1450192e10b3c44347860 P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.367903  2067 tablet_replica.cc:333] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.369877  2067 raft_consensus.cc:2243] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.370954  2067 raft_consensus.cc:2272] T a2be382cd47e41839958b59bc8e3c40a P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.373670  2067 tablet_replica.cc:333] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.374369  2067 raft_consensus.cc:2243] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.375376  2067 raft_consensus.cc:2272] T 412da949752f49049ec6148896078f29 P 8525f77f43714a58bf158d872c1db3e4 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:26.380751  3364 consensus_peers.cc:597] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:26.383620  2067 tablet_replica.cc:333] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.384474  2067 raft_consensus.cc:2243] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.385269  2067 raft_consensus.cc:2272] T 34c56ebe809a4c09b75ee18faf980d8c P 8525f77f43714a58bf158d872c1db3e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.388628  2067 tablet_replica.cc:333] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.389369  2067 raft_consensus.cc:2243] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.390110  2067 raft_consensus.cc:2272] T af3368aec5b1412797545abe458a95fe P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.392635  2067 tablet_replica.cc:333] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.393285  2067 raft_consensus.cc:2243] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:26.394070  2067 raft_consensus.cc:2272] T 1f76214f21f642889fe174df15bb7047 P 8525f77f43714a58bf158d872c1db3e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:26.397125  2067 tablet_replica.cc:333] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4: stopping tablet replica
I20260709 12:24:26.397835  2067 raft_consensus.cc:2243] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:26.398841  2067 raft_consensus.cc:2272] T 1b0ee883b3374a4eb86de5773524325b P 8525f77f43714a58bf158d872c1db3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:26.438258  3362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:37779, user_credentials={real_user=slave}} blocked reactor thread for 45551.1us
W20260709 12:24:26.619608  3362 consensus_peers.cc:597] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:26.696799  3963 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111) [suppressed 423 similar messages]
W20260709 12:24:26.732811  3961 consensus_peers.cc:597] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:26.761255  3362 consensus_peers.cc:597] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:26.769608  3362 consensus_peers.cc:597] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:26.834424  3961 consensus_peers.cc:597] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:26.864794  3961 consensus_peers.cc:597] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:26.919806  3961 consensus_peers.cc:597] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:26.956631  3362 consensus_peers.cc:597] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:26.982162  3362 consensus_peers.cc:597] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:26.982656  3362 consensus_peers.cc:597] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:27.025935  3961 consensus_peers.cc:597] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:27.054922  3362 consensus_peers.cc:597] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:27.101280  3362 consensus_peers.cc:597] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:24:27.202301  5586 raft_consensus.cc:493] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.202917  5586 raft_consensus.cc:515] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.205595  5608 raft_consensus.cc:493] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.206256  5608 raft_consensus.cc:515] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:27.210777  5608 leader_election.cc:290] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:27.215200  3961 consensus_peers.cc:597] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:24:27.216884  5586 leader_election.cc:290] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
W20260709 12:24:27.220780  3362 leader_election.cc:336] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:27.220654  3991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a9849f17fd4313ab9ddbb06589c46f" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
W20260709 12:24:27.222290  3364 leader_election.cc:336] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.222803  3364 leader_election.cc:304] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.223605  5586 raft_consensus.cc:2749] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:27.229063  3364 leader_election.cc:336] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.229493  3364 leader_election.cc:304] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:27.230360  5586 raft_consensus.cc:2749] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:27.231376  5586 raft_consensus.cc:493] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.231964  5586 raft_consensus.cc:515] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:24:27.233210  3132 catalog_manager.cc:4755] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 11921fa3d80340c18f076a4d4499340b: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:27.234390  5586 leader_election.cc:290] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:27.240126  3362 leader_election.cc:336] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:27.245697  3362 consensus_peers.cc:597] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:27.246332  3364 leader_election.cc:336] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.256314  3364 leader_election.cc:304] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.257416  5586 raft_consensus.cc:2749] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:27.260723  3961 consensus_peers.cc:597] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:27.284484  3961 consensus_peers.cc:597] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:24:27.297996  5608 raft_consensus.cc:493] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.298753  5608 raft_consensus.cc:515] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.303210  5608 leader_election.cc:290] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:27.304267  3997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4a63fc24af42ca862894414f658194" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
W20260709 12:24:27.308122  3364 leader_election.cc:336] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.308624  3364 leader_election.cc:304] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 8525f77f43714a58bf158d872c1db3e4, e38cc90af94c47f8a86be67ad610cbd4
I20260709 12:24:27.309377  5608 raft_consensus.cc:2749] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:27.355047  3961 consensus_peers.cc:597] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:24:27.371028  5586 raft_consensus.cc:493] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.371807  5586 raft_consensus.cc:515] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:27.374315  5586 leader_election.cc:290] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:27.380407  3362 leader_election.cc:336] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:27.381490  3364 leader_election.cc:336] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.382079  3364 leader_election.cc:304] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.383044  5608 raft_consensus.cc:2749] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:27.408957  5628 raft_consensus.cc:493] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.409718  5628 raft_consensus.cc:515] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.413074  5629 raft_consensus.cc:493] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.413877  5629 raft_consensus.cc:515] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.416442  5628 leader_election.cc:290] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:27.416451  5629 leader_election.cc:290] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:27.417716  5609 raft_consensus.cc:493] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
W20260709 12:24:27.417798  3963 leader_election.cc:336] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.418398  5609 raft_consensus.cc:515] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:27.418677  5586 raft_consensus.cc:493] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.419361  5586 raft_consensus.cc:515] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:27.420608  5609 leader_election.cc:290] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:27.421360  3963 leader_election.cc:336] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.421808  5586 leader_election.cc:290] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:27.424399  3392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a9849f17fd4313ab9ddbb06589c46f" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:27.425160  3392 raft_consensus.cc:2468] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 1.
I20260709 12:24:27.427129  3962 leader_election.cc:304] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.428444  5629 raft_consensus.cc:2804] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:27.428853  5629 raft_consensus.cc:493] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.429240  5629 raft_consensus.cc:3060] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:27.429850  3961 leader_election.cc:336] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:27.430509  3961 leader_election.cc:304] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.431391  5628 raft_consensus.cc:2749] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:27.432454  5576 raft_consensus.cc:493] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.432971  5576 raft_consensus.cc:515] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.435168  5576 leader_election.cc:290] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:27.432317  3362 leader_election.cc:336] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:27.437237  3362 leader_election.cc:336] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:27.442018  5576 raft_consensus.cc:493] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
W20260709 12:24:27.442920  3364 leader_election.cc:336] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.442723  5576 raft_consensus.cc:515] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:27.443238  3963 leader_election.cc:336] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.444146  3364 leader_election.cc:304] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.442960  5629 raft_consensus.cc:515] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.446462  5576 leader_election.cc:290] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:27.447114  3961 leader_election.cc:336] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:27.447896  5586 raft_consensus.cc:2749] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:27.449093  5629 leader_election.cc:290] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:24:27.448122  3364 leader_election.cc:336] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.449554  3961 leader_election.cc:304] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.450366  3364 leader_election.cc:304] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.451221  5586 raft_consensus.cc:2749] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:27.452150  5629 raft_consensus.cc:2749] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:27.452491  3392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a9849f17fd4313ab9ddbb06589c46f" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
W20260709 12:24:27.452987  3963 leader_election.cc:336] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.453176  3392 raft_consensus.cc:3060] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:27.455783  3963 leader_election.cc:336] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
W20260709 12:24:27.461879  3961 leader_election.cc:336] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:27.462478  3961 leader_election.cc:304] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.461524  3392 raft_consensus.cc:2468] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
I20260709 12:24:27.464927  5576 raft_consensus.cc:2749] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:27.466465  3962 leader_election.cc:304] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.468101  5576 raft_consensus.cc:2804] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:27.468684  5576 raft_consensus.cc:697] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:24:27.469808  5576 consensus_queue.cc:237] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.482667  5628 raft_consensus.cc:493] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.483386  5628 raft_consensus.cc:515] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.485232  5576 raft_consensus.cc:493] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.485939  5628 leader_election.cc:290] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:27.487397  3392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11c9e1f3b048769b242da5c7ae7cd7" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:27.488361  5576 raft_consensus.cc:515] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.494102  3392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4a63fc24af42ca862894414f658194" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:27.494297  3140 catalog_manager.cc:5697] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 1 to 2, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:27.494865  3392 raft_consensus.cc:2468] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 1.
W20260709 12:24:27.500300  3963 leader_election.cc:336] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.500892  5576 leader_election.cc:290] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:27.500753  3962 leader_election.cc:304] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
I20260709 12:24:27.500804  3963 leader_election.cc:304] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:27.502406  5628 raft_consensus.cc:2804] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:27.502859  5628 raft_consensus.cc:493] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.503341  5628 raft_consensus.cc:3060] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:27.512866  5608 raft_consensus.cc:493] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.513335  5629 raft_consensus.cc:2749] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:27.513545  5608 raft_consensus.cc:515] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:24:27.517728  3963 leader_election.cc:336] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.518208  5608 leader_election.cc:290] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:27.523620  5628 raft_consensus.cc:515] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:27.525560  3362 consensus_peers.cc:597] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:27.526175  3362 leader_election.cc:336] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:27.529953  5576 raft_consensus.cc:493] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.530719  5576 raft_consensus.cc:515] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.539813  3392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caed93336fbb415c82acd602adce7969" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:27.540199  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4a63fc24af42ca862894414f658194" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4"
I20260709 12:24:27.540892  3389 raft_consensus.cc:3060] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:27.540936  5628 leader_election.cc:290] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
W20260709 12:24:27.541707  3963 leader_election.cc:336] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.542354  5576 leader_election.cc:290] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:27.542410  3963 leader_election.cc:304] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:27.543493  5576 raft_consensus.cc:2749] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:27.541848  3364 leader_election.cc:336] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.544174  3364 leader_election.cc:304] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.545212  5586 raft_consensus.cc:2749] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:27.549701  5576 raft_consensus.cc:493] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.550400  5576 raft_consensus.cc:515] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:27.553694  2067 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 3345)
I20260709 12:24:27.555419  5576 leader_election.cc:290] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:27.557839  3389 raft_consensus.cc:2468] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38cc90af94c47f8a86be67ad610cbd4 in term 2.
W20260709 12:24:27.550495  3963 leader_election.cc:336] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
W20260709 12:24:27.559048  3961 leader_election.cc:336] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:27.560256  3962 leader_election.cc:304] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.562425  5628 raft_consensus.cc:2804] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:27.563208  5628 raft_consensus.cc:697] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Becoming Leader. State: Replica: e38cc90af94c47f8a86be67ad610cbd4, State: Running, Role: LEADER
I20260709 12:24:27.564220  5628 consensus_queue.cc:237] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.565999  5629 raft_consensus.cc:493] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.566805  5629 raft_consensus.cc:515] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.576303  5629 leader_election.cc:290] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:27.578588  3362 consensus_peers.cc:597] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:27.581800  3961 leader_election.cc:336] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:27.582345  5629 raft_consensus.cc:493] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.583021  5629 raft_consensus.cc:515] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.586560  5629 leader_election.cc:290] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:27.586596  5586 raft_consensus.cc:493] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.587178  3389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14193bb0c64048cf8ad561f477695970" candidate_uuid: "e38cc90af94c47f8a86be67ad610cbd4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c93989d3dd634ef585d77b03354121a4" is_pre_election: true
I20260709 12:24:27.587373  5586 raft_consensus.cc:515] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:27.588047  3963 leader_election.cc:336] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.588758  3963 leader_election.cc:304] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.589560  5586 leader_election.cc:290] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:27.592784  5629 raft_consensus.cc:2749] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:27.594065  3997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caed93336fbb415c82acd602adce7969" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:27.594806  3997 raft_consensus.cc:2468] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 2.
I20260709 12:24:27.599375  3362 leader_election.cc:304] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 
W20260709 12:24:27.599557  3961 consensus_peers.cc:597] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:24:27.600322  5586 raft_consensus.cc:2804] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:27.600773  5586 raft_consensus.cc:493] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.601151  5586 raft_consensus.cc:3060] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:27.601805  3364 leader_election.cc:336] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
W20260709 12:24:27.601985  3963 leader_election.cc:336] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.602813  3963 leader_election.cc:304] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.604234  5576 raft_consensus.cc:2749] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:27.608808  3963 leader_election.cc:336] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.609776  3963 leader_election.cc:304] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:27.610848  5576 raft_consensus.cc:2749] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:27.609758  5586 raft_consensus.cc:515] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.613574  5586 leader_election.cc:290] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:27.615701  5586 raft_consensus.cc:493] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.616196  5586 raft_consensus.cc:515] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.617491  5576 raft_consensus.cc:493] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.618094  5576 raft_consensus.cc:515] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:27.618876  3364 leader_election.cc:336] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
W20260709 12:24:27.625463  3364 leader_election.cc:336] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.623118  5586 leader_election.cc:290] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
W20260709 12:24:27.626217  3961 leader_election.cc:336] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:27.627761  5576 leader_election.cc:290] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:27.632766  3991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11c9e1f3b048769b242da5c7ae7cd7" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:27.633889  5609 raft_consensus.cc:493] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.634456  5609 raft_consensus.cc:515] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.636948  5609 leader_election.cc:290] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:27.633497  3991 raft_consensus.cc:2468] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 2.
I20260709 12:24:27.639219  3997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caed93336fbb415c82acd602adce7969" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:24:27.639833  3997 raft_consensus.cc:3060] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:27.640617  3362 leader_election.cc:304] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.642544  3991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14193bb0c64048cf8ad561f477695970" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:27.643191  3991 raft_consensus.cc:2468] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 2.
I20260709 12:24:27.643834  5586 raft_consensus.cc:2804] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:27.644232  5586 raft_consensus.cc:493] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.644620  5586 raft_consensus.cc:3060] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:27.637455  5609 raft_consensus.cc:493] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.646631  5609 raft_consensus.cc:515] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
W20260709 12:24:27.648432  3364 leader_election.cc:336] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.654990  3997 raft_consensus.cc:2468] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
I20260709 12:24:27.655694  5609 leader_election.cc:290] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:27.662099  3364 leader_election.cc:336] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.656545  5586 raft_consensus.cc:515] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.663708  3140 catalog_manager.cc:5697] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 reported cstate change: term changed from 1 to 2, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "e38cc90af94c47f8a86be67ad610cbd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:27.665396  5586 leader_election.cc:290] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
W20260709 12:24:27.665488  3963 leader_election.cc:336] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.667275  3963 leader_election.cc:304] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.672571  5576 raft_consensus.cc:2749] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:27.676513  3364 leader_election.cc:336] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.678074  3997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11c9e1f3b048769b242da5c7ae7cd7" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:24:27.678938  3362 leader_election.cc:304] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.679129  3997 raft_consensus.cc:3060] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:27.680248  3362 leader_election.cc:304] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.681234  5586 raft_consensus.cc:2804] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:27.681613  5586 raft_consensus.cc:493] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.682026  5586 raft_consensus.cc:3060] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:27.687541  3997 raft_consensus.cc:2468] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
I20260709 12:24:27.688398  5652 raft_consensus.cc:2804] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:27.689251  5652 raft_consensus.cc:697] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:27.690369  5652 consensus_queue.cc:237] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:27.690982  3362 consensus_peers.cc:597] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:27.693651  5586 raft_consensus.cc:515] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.695652  3362 leader_election.cc:304] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.696645  5609 raft_consensus.cc:2804] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:27.697229  5609 raft_consensus.cc:697] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
I20260709 12:24:27.698216  5609 consensus_queue.cc:237] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.699579  3991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14193bb0c64048cf8ad561f477695970" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4"
I20260709 12:24:27.700237  3991 raft_consensus.cc:3060] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:27.702848  5586 leader_election.cc:290] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Requested vote from peers 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
W20260709 12:24:27.712767  3362 leader_election.cc:336] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:27.713470  3362 leader_election.cc:304] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.714449  5652 raft_consensus.cc:2749] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:27.715546  5652 raft_consensus.cc:493] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.716073  5652 raft_consensus.cc:515] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:27.718516  5652 leader_election.cc:290] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:27.719839  3140 catalog_manager.cc:5697] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 2 to 3, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:27.721380  3991 raft_consensus.cc:2468] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 3.
W20260709 12:24:27.733554  3364 leader_election.cc:336] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
W20260709 12:24:27.739025  3364 leader_election.cc:336] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.741914  3362 leader_election.cc:304] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93989d3dd634ef585d77b03354121a4, e38cc90af94c47f8a86be67ad610cbd4; no voters: 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.743126  5609 raft_consensus.cc:2804] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:27.743737  5609 raft_consensus.cc:697] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Becoming Leader. State: Replica: c93989d3dd634ef585d77b03354121a4, State: Running, Role: LEADER
W20260709 12:24:27.744817  3362 leader_election.cc:336] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:27.744712  5609 consensus_queue.cc:237] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.745532  3362 leader_election.cc:304] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.746495  5652 raft_consensus.cc:2749] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:27.753983  5652 raft_consensus.cc:493] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:27.754530  5652 raft_consensus.cc:515] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:27.756742  5652 leader_election.cc:290] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:27.763665  3140 catalog_manager.cc:5697] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 2 to 3, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:27.767117  3362 leader_election.cc:336] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:27.765610  3140 catalog_manager.cc:5697] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 reported cstate change: term changed from 2 to 3, leader changed from 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194) to c93989d3dd634ef585d77b03354121a4 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "c93989d3dd634ef585d77b03354121a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:27.773473  3364 leader_election.cc:336] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:27.774173  3364 leader_election.cc:304] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.778652  5652 raft_consensus.cc:2749] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:27.931243  5609 raft_consensus.cc:493] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.931867  5609 raft_consensus.cc:515] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:27.939734  5609 leader_election.cc:290] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), e38cc90af94c47f8a86be67ad610cbd4 (127.2.4.196:45287)
I20260709 12:24:27.944862  3997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84e58c891d3b452eb575499063d5d34f" candidate_uuid: "c93989d3dd634ef585d77b03354121a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e38cc90af94c47f8a86be67ad610cbd4" is_pre_election: true
I20260709 12:24:27.945706  3997 raft_consensus.cc:2468] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93989d3dd634ef585d77b03354121a4 in term 4.
W20260709 12:24:27.947471  3364 leader_election.cc:336] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
W20260709 12:24:27.948325  3362 leader_election.cc:336] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:27.948805  3362 leader_election.cc:304] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:27.949548  5586 raft_consensus.cc:2749] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:27.984894  3392 raft_consensus.cc:1275] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:27.986382  5628 consensus_queue.cc:1048] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:24:28.017061  3963 consensus_peers.cc:597] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:28.036644  3961 consensus_peers.cc:597] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:28.038472  3392 raft_consensus.cc:1275] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Refusing update from remote peer e38cc90af94c47f8a86be67ad610cbd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:28.040205  5576 consensus_queue.cc:1048] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:24:28.044485  3961 consensus_peers.cc:597] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.061756  3963 consensus_peers.cc:597] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:28.132212  3362 consensus_peers.cc:597] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:28.136904  3997 raft_consensus.cc:1275] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:24:28.137876  3364 consensus_peers.cc:597] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:28.138387  5609 consensus_queue.cc:1048] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:24:28.163384  3364 consensus_peers.cc:597] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:28.171715  3364 consensus_peers.cc:597] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:28.255259  3364 consensus_peers.cc:597] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.262151  3963 consensus_peers.cc:597] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:28.285153  3997 raft_consensus.cc:1275] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:28.286881  5652 consensus_queue.cc:1048] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:24:28.305471  3362 consensus_peers.cc:597] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:28.318744  3997 raft_consensus.cc:1275] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Refusing update from remote peer c93989d3dd634ef585d77b03354121a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:28.320747  5586 consensus_queue.cc:1048] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:24:28.375833  3963 consensus_peers.cc:597] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.383950  3963 consensus_peers.cc:597] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.412747  3963 consensus_peers.cc:597] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.418643  3961 consensus_peers.cc:597] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.429536  3961 consensus_peers.cc:597] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.482722  3963 consensus_peers.cc:597] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.495095  3963 consensus_peers.cc:597] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.495262  3364 consensus_peers.cc:597] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.515234  3963 consensus_peers.cc:597] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.538901  3963 consensus_peers.cc:597] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.554240  2067 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 3345)
W20260709 12:24:28.569031  3364 consensus_peers.cc:597] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.577342  3364 consensus_peers.cc:597] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.577926  3364 consensus_peers.cc:597] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.590723  3364 consensus_peers.cc:597] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.659185  3364 consensus_peers.cc:597] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.660058  3963 consensus_peers.cc:597] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.674230  3364 consensus_peers.cc:597] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.691443  3364 consensus_peers.cc:597] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.717003  3963 consensus_peers.cc:597] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.731617  3364 consensus_peers.cc:597] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.752943  3364 consensus_peers.cc:597] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.763548  3963 consensus_peers.cc:597] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.861672  3963 consensus_peers.cc:597] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.865224  3963 consensus_peers.cc:597] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Couldn't send request to peer 8525f77f43714a58bf158d872c1db3e4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:28.865345  3362 consensus_peers.cc:597] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:24:28.972404  5652 raft_consensus.cc:493] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:28.973196  5652 raft_consensus.cc:515] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:28.975728  5652 leader_election.cc:290] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:28.980774  3362 leader_election.cc:336] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:28.981592  3364 leader_election.cc:336] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:28.982252  3364 leader_election.cc:304] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:28.984135  5652 raft_consensus.cc:2749] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:29.033267  2067 tablet_server.cc:196] TabletServer@127.2.4.194:0 shutdown complete.
I20260709 12:24:29.140964  2067 tablet_server.cc:179] TabletServer@127.2.4.195:0 shutting down...
I20260709 12:24:29.154021  5576 raft_consensus.cc:493] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:29.163518  5576 raft_consensus.cc:515] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:29.167565  3962 consensus_peers.cc:597] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Couldn't send request to peer c93989d3dd634ef585d77b03354121a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:29.176046  3961 leader_election.cc:336] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:29.177994  5576 leader_election.cc:290] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:29.178977  5576 raft_consensus.cc:493] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:29.182701  5576 raft_consensus.cc:515] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
W20260709 12:24:29.187649  3963 leader_election.cc:336] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:29.188306  5576 leader_election.cc:290] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:29.193848  3962 consensus_peers.cc:597] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 -> Peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Couldn't send request to peer c93989d3dd634ef585d77b03354121a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:29.188313  3963 leader_election.cc:304] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
W20260709 12:24:29.197393  3961 leader_election.cc:336] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:29.198181  5629 raft_consensus.cc:2749] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:29.204389  3962 consensus_peers.cc:597] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Couldn't send request to peer c93989d3dd634ef585d77b03354121a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:29.208773  3362 consensus_peers.cc:597] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:24:29.207736  3963 leader_election.cc:336] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:29.209648  3963 leader_election.cc:304] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:29.211148  5664 raft_consensus.cc:2749] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:29.225718  3962 consensus_peers.cc:597] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Couldn't send request to peer c93989d3dd634ef585d77b03354121a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:29.227689  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:29.228645  2067 tablet_replica.cc:333] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.229444  2067 raft_consensus.cc:2243] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.230815  2067 raft_consensus.cc:2272] T 14193bb0c64048cf8ad561f477695970 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.233009  2067 tablet_replica.cc:333] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.233728  2067 raft_consensus.cc:2243] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.234169  2067 raft_consensus.cc:2272] T f8a494da53f44c1d9780c046a155769a P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.236747  2067 tablet_replica.cc:333] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.237416  2067 raft_consensus.cc:2243] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.238396  2067 raft_consensus.cc:2272] T 1f3795979cf042008b3447b47c75de09 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.241701  2067 tablet_replica.cc:333] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.242460  2067 raft_consensus.cc:2243] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.243672  2067 raft_consensus.cc:2272] T 7ff1130a2f274bccbc5b15389deacdc1 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.245841  2067 tablet_replica.cc:333] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.246472  2067 raft_consensus.cc:2243] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.247589  2067 raft_consensus.cc:2272] T bb11c9e1f3b048769b242da5c7ae7cd7 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.250442  2067 tablet_replica.cc:333] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.251391  2067 raft_consensus.cc:2243] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.251951  2067 raft_consensus.cc:2272] T 1f76214f21f642889fe174df15bb7047 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.254276  2067 tablet_replica.cc:333] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.254951  2067 raft_consensus.cc:2243] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.255538  2067 raft_consensus.cc:2272] T 73563dc7466d4636a64ebd4d363935b1 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.258209  2067 tablet_replica.cc:333] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.259084  2067 raft_consensus.cc:2243] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.260016  2067 raft_consensus.cc:2272] T 6b70b620cf8243f4b39cf03eaae44881 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.262176  2067 tablet_replica.cc:333] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.262805  2067 raft_consensus.cc:2243] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.263296  2067 raft_consensus.cc:2272] T a7494971ced1402da04932982df157fd P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.265254  2067 tablet_replica.cc:333] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.265941  2067 raft_consensus.cc:2243] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.266786  2067 raft_consensus.cc:2272] T 0634624f456740b48b77096645ab6b0b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.268674  2067 tablet_replica.cc:333] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.269260  2067 raft_consensus.cc:2243] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.270238  2067 raft_consensus.cc:2272] T 16f6e94aa295499cb8358daafcb934b6 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.272109  2067 tablet_replica.cc:333] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.272743  2067 raft_consensus.cc:2243] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.273547  2067 raft_consensus.cc:2272] T 36377e446734493c8f3b1ff53d40d1cc P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.276324  2067 tablet_replica.cc:333] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.277212  2067 raft_consensus.cc:2243] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.277839  2067 raft_consensus.cc:2272] T b8f62d0c16794e8c941849d55bb4bb0b P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.280886  2067 tablet_replica.cc:333] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.281800  2067 raft_consensus.cc:2243] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Raft consensus shutting down.
W20260709 12:24:29.282025  3362 consensus_peers.cc:597] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:24:29.282735  2067 raft_consensus.cc:2272] T f85f93dbda424e3aa0102bc19a2ff837 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.285559  5652 raft_consensus.cc:493] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:29.285790  2067 tablet_replica.cc:333] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.286144  5652 raft_consensus.cc:515] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:29.286706  2067 raft_consensus.cc:2243] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.287869  2067 raft_consensus.cc:2272] T b7048879a65244efa5a140c52d782803 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.288440  5652 leader_election.cc:290] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:29.290781  2067 tablet_replica.cc:333] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.291754  2067 raft_consensus.cc:2243] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.292906  2067 raft_consensus.cc:2272] T d02f86799bc54984b41ede0f7db226e6 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:29.293061  3362 leader_election.cc:336] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:29.295711  3364 leader_election.cc:336] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:29.296011  2067 tablet_replica.cc:333] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.296250  3364 leader_election.cc:304] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:29.296959  2067 raft_consensus.cc:2243] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.297029  5652 raft_consensus.cc:2749] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:29.297935  2067 raft_consensus.cc:2272] T fe4a63fc24af42ca862894414f658194 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.299961  2067 tablet_replica.cc:333] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.300741  2067 raft_consensus.cc:2243] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.301234  2067 raft_consensus.cc:2272] T 00dbb3c83d794b5db75532d598235d9a P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.303344  2067 tablet_replica.cc:333] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.304090  2067 raft_consensus.cc:2243] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.305088  2067 raft_consensus.cc:2272] T c9145effc86e4e0db3de0ff579cd939d P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.306241  5652 raft_consensus.cc:493] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:29.306833  5652 raft_consensus.cc:515] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:29.307821  2067 tablet_replica.cc:333] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.308781  2067 raft_consensus.cc:2243] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.309274  5652 leader_election.cc:290] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:29.309916  2067 raft_consensus.cc:2272] T 6c9b725784044c8da903f092febf7c05 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.313004  2067 tablet_replica.cc:333] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.313854  2067 raft_consensus.cc:2243] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:24:29.314495  3362 leader_election.cc:336] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:29.314929  2067 raft_consensus.cc:2272] T e8d7a917f21741e9827c30e5433a38e3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:29.316634  3364 leader_election.cc:336] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:29.317054  3364 leader_election.cc:304] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:29.318022  5652 raft_consensus.cc:2749] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:29.318374  3961 consensus_peers.cc:597] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:24:29.319113  2067 tablet_replica.cc:333] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.320026  2067 raft_consensus.cc:2243] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.320621  2067 raft_consensus.cc:2272] T d373e8b022274a08b08f00c0057ae3d3 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:29.321854  3962 consensus_peers.cc:597] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Couldn't send request to peer c93989d3dd634ef585d77b03354121a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:29.322813  2067 tablet_replica.cc:333] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.323479  2067 raft_consensus.cc:2243] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.324007  2067 raft_consensus.cc:2272] T 84e58c891d3b452eb575499063d5d34f P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.326223  2067 tablet_replica.cc:333] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.326779  2067 raft_consensus.cc:2243] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.327409  2067 raft_consensus.cc:2272] T d6a9849f17fd4313ab9ddbb06589c46f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.329207  2067 tablet_replica.cc:333] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.329897  2067 raft_consensus.cc:2243] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.330454  2067 raft_consensus.cc:2272] T c4a1156be8e3418790f41cd380136b93 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.332237  2067 tablet_replica.cc:333] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.332834  2067 raft_consensus.cc:2243] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.333313  2067 raft_consensus.cc:2272] T 076c82502ea24caf81503a8f13fadf97 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.335318  2067 tablet_replica.cc:333] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.335968  2067 raft_consensus.cc:2243] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.336463  2067 raft_consensus.cc:2272] T 2a558d863a564ddf9aa7c954dffaeafe P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.338816  2067 tablet_replica.cc:333] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.339493  2067 raft_consensus.cc:2243] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.340040  2067 raft_consensus.cc:2272] T 9253ca0fa41f43a7ac13ff17879c2ed3 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.342163  2067 tablet_replica.cc:333] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.342717  2067 raft_consensus.cc:2243] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.343648  2067 raft_consensus.cc:2272] T caed93336fbb415c82acd602adce7969 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.345613  2067 tablet_replica.cc:333] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.346223  2067 raft_consensus.cc:2243] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.346675  2067 raft_consensus.cc:2272] T dfabf95aa4ee4eae966abca58130bd91 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.347500  5629 raft_consensus.cc:493] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:29.348064  5629 raft_consensus.cc:515] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:29.349332  2067 tablet_replica.cc:333] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.350281  2067 raft_consensus.cc:2243] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.350412  5629 leader_election.cc:290] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:29.351387  2067 raft_consensus.cc:2272] T bfb1e1f3b50843ffbc685b3a019ea8cb P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:29.352607  3962 consensus_peers.cc:597] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 -> Peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Couldn't send request to peer c93989d3dd634ef585d77b03354121a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:29.354135  2067 tablet_replica.cc:333] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.354887  2067 raft_consensus.cc:2243] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.355474  2067 raft_consensus.cc:2272] T 9f37527cd3a34753b502172f81b808e0 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:29.355635  3961 leader_election.cc:336] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:29.356312  3963 leader_election.cc:336] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:29.356770  3963 leader_election.cc:304] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:29.357755  5664 raft_consensus.cc:2749] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:29.358180  2067 tablet_replica.cc:333] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.359097  2067 raft_consensus.cc:2243] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.359620  2067 raft_consensus.cc:2272] T 118e1f84035044e1b2df505c59a79fdb P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.362299  2067 tablet_replica.cc:333] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
W20260709 12:24:29.363749  3961 consensus_peers.cc:597] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:24:29.426599  2067 raft_consensus.cc:2243] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.431450  2067 raft_consensus.cc:2272] T 2bde98a54e9f47a0b9afe40221bcb336 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.436954  2067 tablet_replica.cc:333] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.439453  2067 raft_consensus.cc:2243] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:24:29.441516  3961 consensus_peers.cc:597] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:24:29.443782  2067 raft_consensus.cc:2272] T fd6cee8d33de46388ec38d683a789e94 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:29.447516  3962 consensus_peers.cc:597] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 -> Peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Couldn't send request to peer c93989d3dd634ef585d77b03354121a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:29.451116  3962 consensus_peers.cc:597] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Couldn't send request to peer c93989d3dd634ef585d77b03354121a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:29.451936  2067 tablet_replica.cc:333] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.454298  2067 raft_consensus.cc:2243] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.459743  2067 raft_consensus.cc:2272] T a9cc3d1278674852b3b6c8c39f4f1b59 P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:29.462766  3962 consensus_peers.cc:597] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 -> Peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Couldn't send request to peer c93989d3dd634ef585d77b03354121a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:29.465318  2067 tablet_replica.cc:333] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.467082  2067 raft_consensus.cc:2243] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.469583  2067 raft_consensus.cc:2272] T 1d167141acd1450192e10b3c44347860 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.480059  2067 tablet_replica.cc:333] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.484623  2067 raft_consensus.cc:2243] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 3 LEADER]: Raft consensus shutting down.
W20260709 12:24:29.485180  3961 consensus_peers.cc:597] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:24:29.487807  2067 raft_consensus.cc:2272] T 686eafc8b6bb49fbb5fd3dd0b68a3099 P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:29.487712  3962 consensus_peers.cc:597] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Couldn't send request to peer c93989d3dd634ef585d77b03354121a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:29.492174  2067 tablet_replica.cc:333] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.493049  2067 raft_consensus.cc:2243] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.494931  2067 raft_consensus.cc:2272] T 11921fa3d80340c18f076a4d4499340b P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.497699  2067 tablet_replica.cc:333] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.498383  2067 raft_consensus.cc:2243] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.499073  2067 raft_consensus.cc:2272] T 412da949752f49049ec6148896078f29 P c93989d3dd634ef585d77b03354121a4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.501616  2067 tablet_replica.cc:333] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.502367  2067 raft_consensus.cc:2243] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.503100  2067 raft_consensus.cc:2272] T a2be382cd47e41839958b59bc8e3c40a P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.505832  2067 tablet_replica.cc:333] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.506493  2067 raft_consensus.cc:2243] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.507086  2067 raft_consensus.cc:2272] T 34c56ebe809a4c09b75ee18faf980d8c P c93989d3dd634ef585d77b03354121a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.509786  2067 tablet_replica.cc:333] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.510340  2067 raft_consensus.cc:2243] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.512639  2067 raft_consensus.cc:2272] T dbb481337ada4c83945e213fb117ee7f P c93989d3dd634ef585d77b03354121a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.512722  5652 raft_consensus.cc:493] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:29.513991  5652 raft_consensus.cc:515] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } }
I20260709 12:24:29.516455  2067 tablet_replica.cc:333] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.517009  5652 leader_election.cc:290] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
I20260709 12:24:29.517417  2067 raft_consensus.cc:2243] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:29.520141  2067 raft_consensus.cc:2272] T 3015bb6b71f444dcb17062ac96b48cf5 P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:29.521433  3362 leader_election.cc:336] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:29.523273  3364 leader_election.cc:336] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:29.523342  2067 tablet_replica.cc:333] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4: stopping tablet replica
I20260709 12:24:29.523768  3364 leader_election.cc:304] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93989d3dd634ef585d77b03354121a4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:29.524267  2067 raft_consensus.cc:2243] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:29.524780  5652 raft_consensus.cc:2706] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Received pre-election callback for term 2 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 12:24:29.525354  2067 raft_consensus.cc:2272] T 1b0ee883b3374a4eb86de5773524325b P c93989d3dd634ef585d77b03354121a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:29.549223  5664 raft_consensus.cc:493] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:29.550356  5664 raft_consensus.cc:515] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:29.553329  5664 leader_election.cc:290] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:29.556862  3961 consensus_peers.cc:597] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:24:29.557492  3961 leader_election.cc:336] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:29.558746  3963 leader_election.cc:336] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
I20260709 12:24:29.559195  3963 leader_election.cc:304] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:29.560099  5664 raft_consensus.cc:2749] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:29.655303  3962 consensus_peers.cc:597] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 -> Peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Couldn't send request to peer c93989d3dd634ef585d77b03354121a4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.195:41453: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:29.669102  3962 consensus_peers.cc:597] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Couldn't send request to peer c93989d3dd634ef585d77b03354121a4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.195:41453: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:29.679325  3962 consensus_peers.cc:597] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Couldn't send request to peer c93989d3dd634ef585d77b03354121a4. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.195:41453: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:29.683815  5663 raft_consensus.cc:493] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:29.684774  5663 raft_consensus.cc:515] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:29.687186  5663 leader_election.cc:290] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:29.688907  3961 consensus_peers.cc:597] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:24:29.691526  3963 leader_election.cc:336] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
W20260709 12:24:29.693248  3961 leader_election.cc:336] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:29.693738  3961 leader_election.cc:304] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:29.695017  5663 raft_consensus.cc:2749] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:29.753464  5664 raft_consensus.cc:493] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8525f77f43714a58bf158d872c1db3e4)
I20260709 12:24:29.754235  5664 raft_consensus.cc:515] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:29.756531  5664 leader_election.cc:290] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649)
W20260709 12:24:29.762885  3963 leader_election.cc:336] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
W20260709 12:24:29.763804  3961 leader_election.cc:336] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:29.764976  3961 leader_election.cc:304] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4
I20260709 12:24:29.765842  5663 raft_consensus.cc:2749] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:29.903477  3961 consensus_peers.cc:597] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:24:29.933364  3961 consensus_peers.cc:597] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:24:29.957252  5629 raft_consensus.cc:493] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.958249  5629 raft_consensus.cc:515] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7cdaa0234c64450eb4ce4500b51b8a81" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 37779 } } peers { permanent_uuid: "8525f77f43714a58bf158d872c1db3e4" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33649 } attrs { replace: true } } peers { permanent_uuid: "c93989d3dd634ef585d77b03354121a4" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 41453 } } peers { permanent_uuid: "e38cc90af94c47f8a86be67ad610cbd4" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 45287 } attrs { promote: false } }
I20260709 12:24:29.968227  5629 leader_election.cc:290] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779), 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649), c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453)
I20260709 12:24:29.973364  2067 tablet_server.cc:196] TabletServer@127.2.4.195:0 shutdown complete.
W20260709 12:24:29.969180  3961 leader_election.cc:336] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
W20260709 12:24:29.975101  3963 leader_election.cc:336] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8525f77f43714a58bf158d872c1db3e4 (127.2.4.194:33649): Network error: Client connection negotiation failed: client connection to 127.2.4.194:33649: connect: Connection refused (error 111)
W20260709 12:24:29.976020  3962 leader_election.cc:336] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c93989d3dd634ef585d77b03354121a4 (127.2.4.195:41453): Network error: Client connection negotiation failed: client connection to 127.2.4.195:41453: connect: Connection refused (error 111)
I20260709 12:24:29.978096  3962 leader_election.cc:304] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 1 yes votes; 3 no votes. yes voters: e38cc90af94c47f8a86be67ad610cbd4; no voters: 7cdaa0234c64450eb4ce4500b51b8a81, 8525f77f43714a58bf158d872c1db3e4, c93989d3dd634ef585d77b03354121a4
I20260709 12:24:29.979207  5629 raft_consensus.cc:2749] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:29.983547  3961 consensus_peers.cc:597] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:24:30.115319  2067 tablet_server.cc:179] TabletServer@127.2.4.196:0 shutting down...
I20260709 12:24:30.186373  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:30.187505  2067 tablet_replica.cc:333] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.188930  2067 raft_consensus.cc:2243] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.189903  2067 pending_rounds.cc:70] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4: Trying to abort 1 pending ops.
I20260709 12:24:30.190109  2067 pending_rounds.cc:77] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4: Aborting op as it isn't in flight: id { term: 3 index: 5 } timestamp: 7305625048368185344 op_type: NO_OP noop_request { }
I20260709 12:24:30.190577  2067 raft_consensus.cc:2889] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:24:30.190887  2067 raft_consensus.cc:2272] T 84e58c891d3b452eb575499063d5d34f P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.193929  2067 tablet_replica.cc:333] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.194666  2067 raft_consensus.cc:2243] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.195402  2067 raft_consensus.cc:2272] T bfb1e1f3b50843ffbc685b3a019ea8cb P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.198046  2067 tablet_replica.cc:333] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.198734  2067 raft_consensus.cc:2243] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.200266  2067 raft_consensus.cc:2272] T 2bde98a54e9f47a0b9afe40221bcb336 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.203058  2067 tablet_replica.cc:333] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.203827  2067 raft_consensus.cc:2243] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.205037  2067 raft_consensus.cc:2272] T fe4a63fc24af42ca862894414f658194 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.207538  2067 tablet_replica.cc:333] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.208132  2067 raft_consensus.cc:2243] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.208693  2067 raft_consensus.cc:2272] T b8131bbfd8374661bd9731b4a57468ee P e38cc90af94c47f8a86be67ad610cbd4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.211086  2067 tablet_replica.cc:333] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.211733  2067 raft_consensus.cc:2243] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.212500  2067 raft_consensus.cc:2272] T 5ba9db97635f4d28bf0f3079edf45a77 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.214768  2067 tablet_replica.cc:333] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.215369  2067 raft_consensus.cc:2243] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.215935  2067 raft_consensus.cc:2272] T 054906db16c94967bdef7c9dd5a47ea3 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.218245  2067 tablet_replica.cc:333] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.218796  2067 raft_consensus.cc:2243] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.219946  2067 raft_consensus.cc:2272] T 00dbb3c83d794b5db75532d598235d9a P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.222370  2067 tablet_replica.cc:333] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.223008  2067 raft_consensus.cc:2243] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.223747  2067 raft_consensus.cc:2272] T 14193bb0c64048cf8ad561f477695970 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.225893  2067 tablet_replica.cc:333] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.226672  2067 raft_consensus.cc:2243] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.228166  2067 raft_consensus.cc:2272] T 3098f015fb154cf490e147a8e71f65b6 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.231437  2067 tablet_replica.cc:333] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.232280  2067 raft_consensus.cc:2243] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.233224  2067 raft_consensus.cc:2272] T d639c2cd28f9418682dc27d0b0ba86b2 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.235967  2067 tablet_replica.cc:333] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.236578  2067 raft_consensus.cc:2243] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.237569  2067 raft_consensus.cc:2272] T d6a9849f17fd4313ab9ddbb06589c46f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.239938  2067 tablet_replica.cc:333] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.240507  2067 raft_consensus.cc:2243] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.241369  2067 raft_consensus.cc:2272] T 87fddcaae9194bef90d1047a518e45a6 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.243633  2067 tablet_replica.cc:333] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.244216  2067 raft_consensus.cc:2243] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.245061  2067 raft_consensus.cc:2272] T 9253ca0fa41f43a7ac13ff17879c2ed3 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.247188  2067 tablet_replica.cc:333] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.247785  2067 raft_consensus.cc:2243] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.248445  2067 raft_consensus.cc:2272] T 1f3795979cf042008b3447b47c75de09 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.250545  2067 tablet_replica.cc:333] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.251118  2067 raft_consensus.cc:2243] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.252053  2067 raft_consensus.cc:2272] T a7494971ced1402da04932982df157fd P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.254246  2067 tablet_replica.cc:333] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.254902  2067 raft_consensus.cc:2243] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.255399  2067 raft_consensus.cc:2272] T 0634624f456740b48b77096645ab6b0b P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.257491  2067 tablet_replica.cc:333] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.258213  2067 raft_consensus.cc:2243] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.259270  2067 raft_consensus.cc:2272] T 9f37527cd3a34753b502172f81b808e0 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.261884  2067 tablet_replica.cc:333] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
W20260709 12:24:30.262462  3961 consensus_peers.cc:597] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 -> Peer 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): Couldn't send request to peer 7cdaa0234c64450eb4ce4500b51b8a81. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:24:30.262620  2067 raft_consensus.cc:2243] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.263420  2067 raft_consensus.cc:2272] T bb11c9e1f3b048769b242da5c7ae7cd7 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.266247  2067 tablet_replica.cc:333] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.266903  2067 raft_consensus.cc:2243] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.268033  2067 raft_consensus.cc:2272] T c04a8b7530d5486a880a89a4ba0faac8 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.270751  2067 tablet_replica.cc:333] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.271554  2067 raft_consensus.cc:2243] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.272258  2067 raft_consensus.cc:2272] T fd6cee8d33de46388ec38d683a789e94 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.274403  2067 tablet_replica.cc:333] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.274981  2067 raft_consensus.cc:2243] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.275552  2067 raft_consensus.cc:2272] T 3015bb6b71f444dcb17062ac96b48cf5 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.278918  2067 tablet_replica.cc:333] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.279886  2067 raft_consensus.cc:2243] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.281090  2067 raft_consensus.cc:2272] T 572d97394c304e81affa5ee75b52de28 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.283721  2067 tablet_replica.cc:333] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.284376  2067 raft_consensus.cc:2243] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.285213  2067 raft_consensus.cc:2272] T d02f86799bc54984b41ede0f7db226e6 P e38cc90af94c47f8a86be67ad610cbd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.287494  2067 tablet_replica.cc:333] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.288113  2067 raft_consensus.cc:2243] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.288635  2067 raft_consensus.cc:2272] T f85f93dbda424e3aa0102bc19a2ff837 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.290908  2067 tablet_replica.cc:333] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.291587  2067 raft_consensus.cc:2243] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.292281  2067 raft_consensus.cc:2272] T e2884104b65f4a589e915fd4328b729c P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.294597  2067 tablet_replica.cc:333] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.295307  2067 raft_consensus.cc:2243] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.296001  2067 raft_consensus.cc:2272] T caed93336fbb415c82acd602adce7969 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.298208  2067 tablet_replica.cc:333] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.298878  2067 raft_consensus.cc:2243] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.299561  2067 raft_consensus.cc:2272] T 7ff1130a2f274bccbc5b15389deacdc1 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.302059  2067 tablet_replica.cc:333] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.302721  2067 raft_consensus.cc:2243] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.303807  2067 pending_rounds.cc:70] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4: Trying to abort 1 pending ops.
I20260709 12:24:30.304039  2067 pending_rounds.cc:77] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4: Aborting op as it isn't in flight: id { term: 2 index: 5 } timestamp: 7305625048709591040 op_type: NO_OP noop_request { }
I20260709 12:24:30.304364  2067 raft_consensus.cc:2889] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:24:30.304708  2067 raft_consensus.cc:2272] T af3368aec5b1412797545abe458a95fe P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.306998  2067 tablet_replica.cc:333] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.307617  2067 raft_consensus.cc:2243] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.308468  2067 raft_consensus.cc:2272] T 34c56ebe809a4c09b75ee18faf980d8c P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.310627  2067 tablet_replica.cc:333] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.311232  2067 raft_consensus.cc:2243] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.311772  2067 raft_consensus.cc:2272] T 1d167141acd1450192e10b3c44347860 P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.314155  2067 tablet_replica.cc:333] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.314762  2067 raft_consensus.cc:2243] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.315850  2067 raft_consensus.cc:2272] T 7a6d2799afd04cbe9bfaa74d467e66cb P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.318084  2067 tablet_replica.cc:333] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.318673  2067 raft_consensus.cc:2243] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.319553  2067 raft_consensus.cc:2272] T dbb481337ada4c83945e213fb117ee7f P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.322013  2067 tablet_replica.cc:333] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.322635  2067 raft_consensus.cc:2243] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.323565  2067 raft_consensus.cc:2272] T b8f62d0c16794e8c941849d55bb4bb0b P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.325829  2067 tablet_replica.cc:333] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.326467  2067 raft_consensus.cc:2243] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.327401  2067 raft_consensus.cc:2272] T a9cc3d1278674852b3b6c8c39f4f1b59 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.330266  2067 tablet_replica.cc:333] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.330862  2067 raft_consensus.cc:2243] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.331771  2067 raft_consensus.cc:2272] T 1f76214f21f642889fe174df15bb7047 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.333850  2067 tablet_replica.cc:333] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.334434  2067 raft_consensus.cc:2243] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.334978  2067 raft_consensus.cc:2272] T c9145effc86e4e0db3de0ff579cd939d P e38cc90af94c47f8a86be67ad610cbd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.336916  2067 tablet_replica.cc:333] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.337486  2067 raft_consensus.cc:2243] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.338636  2067 raft_consensus.cc:2272] T 3af545b01cb24f1cacb0056f804a7692 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.340581  2067 tablet_replica.cc:333] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.341181  2067 raft_consensus.cc:2243] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.341971  2067 raft_consensus.cc:2272] T 6b70b620cf8243f4b39cf03eaae44881 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.344077  2067 tablet_replica.cc:333] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.344671  2067 raft_consensus.cc:2243] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.345513  2067 raft_consensus.cc:2272] T 73563dc7466d4636a64ebd4d363935b1 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.347523  2067 tablet_replica.cc:333] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.348073  2067 raft_consensus.cc:2243] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.348809  2067 raft_consensus.cc:2272] T 11921fa3d80340c18f076a4d4499340b P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.350991  2067 tablet_replica.cc:333] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.351531  2067 raft_consensus.cc:2243] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.352401  2067 raft_consensus.cc:2272] T c4a1156be8e3418790f41cd380136b93 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.354430  2067 tablet_replica.cc:333] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.354955  2067 raft_consensus.cc:2243] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:30.355657  2067 raft_consensus.cc:2272] T 8bcfe15edc4947b988fd204a12c79094 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.357931  2067 tablet_replica.cc:333] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4: stopping tablet replica
I20260709 12:24:30.358412  2067 raft_consensus.cc:2243] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:30.359251  2067 raft_consensus.cc:2272] T 0a5d938460364390a17cada19308b314 P e38cc90af94c47f8a86be67ad610cbd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:30.590525  2067 tablet_server.cc:196] TabletServer@127.2.4.196:0 shutdown complete.
I20260709 12:24:30.669481  2067 master.cc:562] Master@127.2.4.254:39037 shutting down...
W20260709 12:24:31.365449  3132 catalog_manager.cc:4755] TS 7cdaa0234c64450eb4ce4500b51b8a81 (127.2.4.193:37779): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 11921fa3d80340c18f076a4d4499340b: Network error: Client connection negotiation failed: client connection to 127.2.4.193:37779: connect: Connection refused (error 111)
I20260709 12:24:31.480607  2067 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:31.481482  2067 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:31.481899  2067 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6777153c831e452da47938f652649392: stopping tablet replica
I20260709 12:24:31.594748  2067 master.cc:584] Master@127.2.4.254:39037 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (83524 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:24:31.713209  2067 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.4.254:41049
I20260709 12:24:31.714665  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:31.721581  5671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:31.721587  5672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:31.723984  5674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:31.725955  2067 server_base.cc:1061] running on GCE node
I20260709 12:24:31.728153  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:31.728412  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:31.728581  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599871728562 us; error 0 us; skew 500 ppm
I20260709 12:24:31.729254  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:31.732390  2067 webserver.cc:533] Webserver started at http://127.2.4.254:41905/ using document root <none> and password file <none>
I20260709 12:24:31.732995  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:31.733214  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:31.733523  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:31.735496  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/master-0-root/instance:
uuid: "03f70467f9c34de887520aec76068d6f"
format_stamp: "Formatted at 2026-07-09 12:24:31 on dist-test-slave-4qvp"
I20260709 12:24:31.740859  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 12:24:31.745298  5680 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:31.746464  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:24:31.746956  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/master-0-root
uuid: "03f70467f9c34de887520aec76068d6f"
format_stamp: "Formatted at 2026-07-09 12:24:31 on dist-test-slave-4qvp"
I20260709 12:24:31.747335  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:31.761930  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:31.763408  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:31.812399  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.254:41049
I20260709 12:24:31.812549  5731 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.254:41049 every 8 connection(s)
I20260709 12:24:31.817226  5732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:31.828724  5732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Bootstrap starting.
I20260709 12:24:31.833902  5732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.839125  5732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: No bootstrap required, opened a new log
I20260709 12:24:31.841523  5732 raft_consensus.cc:359] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f70467f9c34de887520aec76068d6f" member_type: VOTER }
I20260709 12:24:31.842027  5732 raft_consensus.cc:385] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.842280  5732 raft_consensus.cc:740] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03f70467f9c34de887520aec76068d6f, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.843005  5732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f70467f9c34de887520aec76068d6f" member_type: VOTER }
I20260709 12:24:31.843493  5732 raft_consensus.cc:399] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:24:31.843717  5732 raft_consensus.cc:493] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:24:31.843961  5732 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.848821  5732 raft_consensus.cc:515] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f70467f9c34de887520aec76068d6f" member_type: VOTER }
I20260709 12:24:31.849606  5732 leader_election.cc:304] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03f70467f9c34de887520aec76068d6f; no voters: 
I20260709 12:24:31.850962  5732 leader_election.cc:290] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:24:31.851446  5735 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:31.853220  5735 raft_consensus.cc:697] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [term 1 LEADER]: Becoming Leader. State: Replica: 03f70467f9c34de887520aec76068d6f, State: Running, Role: LEADER
I20260709 12:24:31.854249  5732 sys_catalog.cc:565] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:24:31.854184  5735 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f70467f9c34de887520aec76068d6f" member_type: VOTER }
I20260709 12:24:31.860934  5737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03f70467f9c34de887520aec76068d6f. Latest consensus state: current_term: 1 leader_uuid: "03f70467f9c34de887520aec76068d6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f70467f9c34de887520aec76068d6f" member_type: VOTER } }
I20260709 12:24:31.861573  5737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [sys.catalog]: This master's current role is: LEADER
I20260709 12:24:31.864630  5743 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:24:31.864740  5736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03f70467f9c34de887520aec76068d6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f70467f9c34de887520aec76068d6f" member_type: VOTER } }
I20260709 12:24:31.865667  5736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [sys.catalog]: This master's current role is: LEADER
I20260709 12:24:31.874346  5743 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:24:31.875672  2067 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:24:31.886302  5743 catalog_manager.cc:1383] Generated new cluster ID: 10324fd216fa422eaee29cab9ba1f2db
I20260709 12:24:31.886664  5743 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:24:31.901887  5743 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:24:31.903308  5743 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:24:31.919611  5743 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Generated new TSK 0
I20260709 12:24:31.920408  5743 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:24:31.943476  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:31.951826  5753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:31.951907  5754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:31.955236  5756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:31.956107  2067 server_base.cc:1061] running on GCE node
I20260709 12:24:31.958360  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:31.958624  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:31.958856  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599871958831 us; error 0 us; skew 500 ppm
I20260709 12:24:31.959437  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:31.962435  2067 webserver.cc:533] Webserver started at http://127.2.4.193:35095/ using document root <none> and password file <none>
I20260709 12:24:31.963047  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:31.963224  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:31.963477  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:31.964859  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/instance:
uuid: "1fc57407d56241b38a94d543e073f8af"
format_stamp: "Formatted at 2026-07-09 12:24:31 on dist-test-slave-4qvp"
I20260709 12:24:31.970268  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 12:24:31.974597  5761 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:31.975601  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.001s
I20260709 12:24:31.976008  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-0-root
uuid: "1fc57407d56241b38a94d543e073f8af"
format_stamp: "Formatted at 2026-07-09 12:24:31 on dist-test-slave-4qvp"
I20260709 12:24:31.976338  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:31.999410  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:32.001008  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:32.002985  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:32.006104  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:32.006376  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:32.006654  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:32.006886  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:32.062207  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.193:38701
I20260709 12:24:32.062803  5823 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.193:38701 every 8 connection(s)
I20260709 12:24:32.068883  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:32.078011  5828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:32.080186  5829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:32.083009  5831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:32.084810  2067 server_base.cc:1061] running on GCE node
I20260709 12:24:32.087131  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:24:32.087181  5824 heartbeater.cc:344] Connected to a master server at 127.2.4.254:41049
W20260709 12:24:32.087416  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:32.087709  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599872087689 us; error 0 us; skew 500 ppm
I20260709 12:24:32.087764  5824 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:32.088402  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:32.088744  5824 heartbeater.cc:507] Master 127.2.4.254:41049 requested a full tablet report, sending...
I20260709 12:24:32.091372  5697 ts_manager.cc:194] Registered new tserver with Master: 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:32.091953  2067 webserver.cc:533] Webserver started at http://127.2.4.194:35135/ using document root <none> and password file <none>
I20260709 12:24:32.092705  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:32.092957  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:32.093308  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:32.094312  5697 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40810
I20260709 12:24:32.094909  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/instance:
uuid: "7a0204b931394fc48ffc503716e0d111"
format_stamp: "Formatted at 2026-07-09 12:24:32 on dist-test-slave-4qvp"
I20260709 12:24:32.100277  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:24:32.104287  5836 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:32.105311  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:24:32.105926  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-1-root
uuid: "7a0204b931394fc48ffc503716e0d111"
format_stamp: "Formatted at 2026-07-09 12:24:32 on dist-test-slave-4qvp"
I20260709 12:24:32.106282  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:32.139513  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:32.140996  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:32.142796  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:32.145524  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:32.145805  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:24:32.146070  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:32.146255  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:32.197240  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.194:38005
I20260709 12:24:32.197412  5899 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.194:38005 every 8 connection(s)
I20260709 12:24:32.202526  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:32.211236  5903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:32.213225  5904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:32.216089  5906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:32.216970  2067 server_base.cc:1061] running on GCE node
I20260709 12:24:32.218510  5900 heartbeater.cc:344] Connected to a master server at 127.2.4.254:41049
I20260709 12:24:32.218958  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:24:32.219007  5900 heartbeater.cc:461] Registering TS with master...
W20260709 12:24:32.219347  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:32.219630  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599872219609 us; error 0 us; skew 500 ppm
I20260709 12:24:32.220314  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:32.220283  5900 heartbeater.cc:507] Master 127.2.4.254:41049 requested a full tablet report, sending...
I20260709 12:24:32.223258  5697 ts_manager.cc:194] Registered new tserver with Master: 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:32.223984  2067 webserver.cc:533] Webserver started at http://127.2.4.195:41053/ using document root <none> and password file <none>
I20260709 12:24:32.224619  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:32.224851  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:32.224967  5697 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40812
I20260709 12:24:32.225140  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:32.227157  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/instance:
uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
format_stamp: "Formatted at 2026-07-09 12:24:32 on dist-test-slave-4qvp"
I20260709 12:24:32.232892  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:24:32.237362  5911 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:32.238433  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:24:32.238855  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-2-root
uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
format_stamp: "Formatted at 2026-07-09 12:24:32 on dist-test-slave-4qvp"
I20260709 12:24:32.239212  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:32.262159  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:32.263703  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:32.265401  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:32.268410  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:32.268728  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:32.269060  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:32.269227  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:32.324347  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.195:43329
I20260709 12:24:32.324446  5973 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.195:43329 every 8 connection(s)
I20260709 12:24:32.330870  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:32.340633  5978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:32.341673  5977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:32.345109  5980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:32.345979  5974 heartbeater.cc:344] Connected to a master server at 127.2.4.254:41049
I20260709 12:24:32.346551  5974 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:32.347461  5974 heartbeater.cc:507] Master 127.2.4.254:41049 requested a full tablet report, sending...
I20260709 12:24:32.348556  2067 server_base.cc:1061] running on GCE node
I20260709 12:24:32.349915  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:32.350193  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:32.350414  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599872350393 us; error 0 us; skew 500 ppm
I20260709 12:24:32.350593  5697 ts_manager.cc:194] Registered new tserver with Master: 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:32.351255  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:32.352509  5697 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40818
I20260709 12:24:32.354699  2067 webserver.cc:533] Webserver started at http://127.2.4.196:39689/ using document root <none> and password file <none>
I20260709 12:24:32.355525  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:32.355800  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:32.356127  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:32.357334  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-3-root/instance:
uuid: "6add331326ca47038308a447bf798fa7"
format_stamp: "Formatted at 2026-07-09 12:24:32 on dist-test-slave-4qvp"
I20260709 12:24:32.362501  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.001s
I20260709 12:24:32.366986  5985 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:32.368151  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:24:32.368567  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-3-root
uuid: "6add331326ca47038308a447bf798fa7"
format_stamp: "Formatted at 2026-07-09 12:24:32 on dist-test-slave-4qvp"
I20260709 12:24:32.368878  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:32.386076  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:32.387441  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:32.389014  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:32.391758  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:32.392041  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:32.392326  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:32.392518  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:32.443229  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.196:34835
I20260709 12:24:32.443394  6048 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.196:34835 every 8 connection(s)
I20260709 12:24:32.460531  6049 heartbeater.cc:344] Connected to a master server at 127.2.4.254:41049
I20260709 12:24:32.461056  6049 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:32.462010  6049 heartbeater.cc:507] Master 127.2.4.254:41049 requested a full tablet report, sending...
I20260709 12:24:32.464398  5697 ts_manager.cc:194] Registered new tserver with Master: 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:32.464957  2067 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.018026038s
I20260709 12:24:32.466531  5697 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40830
I20260709 12:24:32.510547  5697 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40844:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:24:32.774042  5864 tablet_service.cc:1511] Processing CreateTablet for tablet 926e6a2e59d94f64ba48bae2ca5f3f55 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:24:32.774575  5863 tablet_service.cc:1511] Processing CreateTablet for tablet 7a3145d5d96e41518c7bf41999289ac3 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:24:32.774763  5789 tablet_service.cc:1511] Processing CreateTablet for tablet 926e6a2e59d94f64ba48bae2ca5f3f55 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:24:32.775211  5786 tablet_service.cc:1511] Processing CreateTablet for tablet 18a324a652924631995e78ee3fda8422 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:24:32.775883  5864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 926e6a2e59d94f64ba48bae2ca5f3f55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.776496  5789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 926e6a2e59d94f64ba48bae2ca5f3f55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.776948  5862 tablet_service.cc:1511] Processing CreateTablet for tablet ebb084ec08f44cb2a935c184afafadb0 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:24:32.777194  5787 tablet_service.cc:1511] Processing CreateTablet for tablet 1ab14137002240c2909503f8b4e55c79 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:24:32.778471  5862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebb084ec08f44cb2a935c184afafadb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.779228  5787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ab14137002240c2909503f8b4e55c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.779294  5788 tablet_service.cc:1511] Processing CreateTablet for tablet 7a3145d5d96e41518c7bf41999289ac3 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:24:32.781334  5863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a3145d5d96e41518c7bf41999289ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.782131  5788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a3145d5d96e41518c7bf41999289ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.786779  5786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a324a652924631995e78ee3fda8422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.798487  5861 tablet_service.cc:1511] Processing CreateTablet for tablet 18a324a652924631995e78ee3fda8422 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:24:32.800055  5861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a324a652924631995e78ee3fda8422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.808864  5938 tablet_service.cc:1511] Processing CreateTablet for tablet 1ab14137002240c2909503f8b4e55c79 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:24:32.810281  5938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ab14137002240c2909503f8b4e55c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.812666  5785 tablet_service.cc:1511] Processing CreateTablet for tablet 9bede7b5a3c84fe28a073542219f3fe4 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:24:32.814344  6014 tablet_service.cc:1511] Processing CreateTablet for tablet 926e6a2e59d94f64ba48bae2ca5f3f55 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:24:32.815002  5937 tablet_service.cc:1511] Processing CreateTablet for tablet ebb084ec08f44cb2a935c184afafadb0 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:24:32.815915  6014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 926e6a2e59d94f64ba48bae2ca5f3f55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.816542  5937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebb084ec08f44cb2a935c184afafadb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.816306  6012 tablet_service.cc:1511] Processing CreateTablet for tablet ebb084ec08f44cb2a935c184afafadb0 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:24:32.818931  6011 tablet_service.cc:1511] Processing CreateTablet for tablet 9bede7b5a3c84fe28a073542219f3fe4 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:24:32.819151  5936 tablet_service.cc:1511] Processing CreateTablet for tablet 18a324a652924631995e78ee3fda8422 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:24:32.820684  5936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a324a652924631995e78ee3fda8422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.820922  5785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bede7b5a3c84fe28a073542219f3fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.815004  6013 tablet_service.cc:1511] Processing CreateTablet for tablet 1ab14137002240c2909503f8b4e55c79 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:24:32.808621  5939 tablet_service.cc:1511] Processing CreateTablet for tablet 7a3145d5d96e41518c7bf41999289ac3 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:24:32.824755  5939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a3145d5d96e41518c7bf41999289ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.825919  6012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebb084ec08f44cb2a935c184afafadb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.834081  6011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bede7b5a3c84fe28a073542219f3fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.834980  6013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ab14137002240c2909503f8b4e55c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.841877  5860 tablet_service.cc:1511] Processing CreateTablet for tablet 9bede7b5a3c84fe28a073542219f3fe4 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:24:32.843528  5860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bede7b5a3c84fe28a073542219f3fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.848461  5935 tablet_service.cc:1511] Processing CreateTablet for tablet 75e8568a9892438c91e30ddc62fef67c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:24:32.850117  5935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e8568a9892438c91e30ddc62fef67c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.857527  6010 tablet_service.cc:1511] Processing CreateTablet for tablet 75e8568a9892438c91e30ddc62fef67c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:24:32.859390  6010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e8568a9892438c91e30ddc62fef67c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.863654  5784 tablet_service.cc:1511] Processing CreateTablet for tablet 75e8568a9892438c91e30ddc62fef67c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:24:32.865542  5784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e8568a9892438c91e30ddc62fef67c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:32.869805  5745 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:32.870361  5859 tablet_service.cc:1511] Processing CreateTablet for tablet 187e00b075464d9c855ce2b87615ac70 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:24:32.871944  5859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 187e00b075464d9c855ce2b87615ac70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.888144  5934 tablet_service.cc:1511] Processing CreateTablet for tablet 187e00b075464d9c855ce2b87615ac70 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:24:32.890329  5934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 187e00b075464d9c855ce2b87615ac70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.899068  6071 tablet_bootstrap.cc:492] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:32.903412  5863 tablet_service.cc:1511] Processing CreateTablet for tablet 45edb6642b84468aa6fc19a7fd2ddf6c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:24:32.905063  5863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45edb6642b84468aa6fc19a7fd2ddf6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.925788  6069 tablet_bootstrap.cc:492] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:32.925853  5936 tablet_service.cc:1511] Processing CreateTablet for tablet 45edb6642b84468aa6fc19a7fd2ddf6c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:24:32.933550  6009 tablet_service.cc:1511] Processing CreateTablet for tablet 187e00b075464d9c855ce2b87615ac70 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:24:32.934333  5788 tablet_service.cc:1511] Processing CreateTablet for tablet 45edb6642b84468aa6fc19a7fd2ddf6c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:24:32.935230  6009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 187e00b075464d9c855ce2b87615ac70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.935781  5788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45edb6642b84468aa6fc19a7fd2ddf6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.937942  6070 tablet_bootstrap.cc:492] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:32.942157  6069 tablet_bootstrap.cc:654] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:32.948565  5936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45edb6642b84468aa6fc19a7fd2ddf6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.953539  6070 tablet_bootstrap.cc:654] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:32.954173  6071 tablet_bootstrap.cc:654] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:32.964108  6010 tablet_service.cc:1511] Processing CreateTablet for tablet e9bb01883059454e80b19d5333aad71e (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:24:32.965662  6010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9bb01883059454e80b19d5333aad71e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.972581  5785 tablet_service.cc:1511] Processing CreateTablet for tablet e9bb01883059454e80b19d5333aad71e (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:24:32.972920  5863 tablet_service.cc:1511] Processing CreateTablet for tablet e9bb01883059454e80b19d5333aad71e (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:24:32.974124  5785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9bb01883059454e80b19d5333aad71e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.975060  5863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9bb01883059454e80b19d5333aad71e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.982076  5935 tablet_service.cc:1511] Processing CreateTablet for tablet 681f64c726104caf875ae7d4329d0676 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:24:32.983520  5935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 681f64c726104caf875ae7d4329d0676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:32.987150  6074 tablet_bootstrap.cc:492] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:32.993889  6011 tablet_service.cc:1511] Processing CreateTablet for tablet 681f64c726104caf875ae7d4329d0676 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:24:32.997057  6011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 681f64c726104caf875ae7d4329d0676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.005041  6069 tablet_bootstrap.cc:492] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:33.005911  6069 ts_tablet_manager.cc:1403] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.081s	user 0.006s	sys 0.009s
I20260709 12:24:33.006361  6074 tablet_bootstrap.cc:654] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.020890  5784 tablet_service.cc:1511] Processing CreateTablet for tablet 681f64c726104caf875ae7d4329d0676 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:24:33.022455  5784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 681f64c726104caf875ae7d4329d0676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.016675  6069 raft_consensus.cc:359] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:33.034045  6069 raft_consensus.cc:385] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.033855  5859 tablet_service.cc:1511] Processing CreateTablet for tablet 30da8f1ded30432a87f01684d88a9969 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:24:33.034560  6069 raft_consensus.cc:740] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.035579  5859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30da8f1ded30432a87f01684d88a9969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.035540  6069 consensus_queue.cc:260] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:33.047941  5939 tablet_service.cc:1511] Processing CreateTablet for tablet 30da8f1ded30432a87f01684d88a9969 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:24:33.049510  5939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30da8f1ded30432a87f01684d88a9969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.061206  6069 ts_tablet_manager.cc:1434] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.055s	user 0.011s	sys 0.018s
I20260709 12:24:33.062150  5900 heartbeater.cc:499] Master 127.2.4.254:41049 was elected leader, sending a full tablet report...
I20260709 12:24:33.062492  6070 tablet_bootstrap.cc:492] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:33.063016  6070 ts_tablet_manager.cc:1403] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.126s	user 0.032s	sys 0.009s
I20260709 12:24:33.064155  6071 tablet_bootstrap.cc:492] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:33.064868  6071 ts_tablet_manager.cc:1403] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.166s	user 0.024s	sys 0.022s
I20260709 12:24:33.066784  6014 tablet_service.cc:1511] Processing CreateTablet for tablet 30da8f1ded30432a87f01684d88a9969 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:24:33.068336  6014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30da8f1ded30432a87f01684d88a9969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.068958  6071 raft_consensus.cc:359] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:33.069939  6071 raft_consensus.cc:385] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.070325  6071 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.071813  6071 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:33.076670  6069 tablet_bootstrap.cc:492] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:33.082830  5785 tablet_service.cc:1511] Processing CreateTablet for tablet d001526e46b24f429f57c91e4a270036 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:24:33.084736  5785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d001526e46b24f429f57c91e4a270036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.081277  6070 raft_consensus.cc:359] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:33.092473  6070 raft_consensus.cc:385] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.093211  6070 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.099413  6070 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:33.097585  5824 heartbeater.cc:499] Master 127.2.4.254:41049 was elected leader, sending a full tablet report...
I20260709 12:24:33.098444  6074 tablet_bootstrap.cc:492] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:33.095885  5861 tablet_service.cc:1511] Processing CreateTablet for tablet d001526e46b24f429f57c91e4a270036 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:24:33.104436  6074 ts_tablet_manager.cc:1403] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.118s	user 0.023s	sys 0.004s
I20260709 12:24:33.114132  6009 tablet_service.cc:1511] Processing CreateTablet for tablet 685933c4b2364e1c90d92edb4c091799 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:24:33.101401  5935 tablet_service.cc:1511] Processing CreateTablet for tablet d001526e46b24f429f57c91e4a270036 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:24:33.115720  6009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 685933c4b2364e1c90d92edb4c091799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.111701  6079 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.105377  5861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d001526e46b24f429f57c91e4a270036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.119329  5935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d001526e46b24f429f57c91e4a270036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.119315  6079 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:33.113168  6071 ts_tablet_manager.cc:1434] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.006s
I20260709 12:24:33.123908  5974 heartbeater.cc:499] Master 127.2.4.254:41049 was elected leader, sending a full tablet report...
I20260709 12:24:33.124260  6071 tablet_bootstrap.cc:492] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:33.145826  6079 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:33.150521  6080 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.151175  6080 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:33.154270  6069 tablet_bootstrap.cc:654] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.157253  6071 tablet_bootstrap.cc:654] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.118371  6074 raft_consensus.cc:359] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:33.158015  6074 raft_consensus.cc:385] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.158378  6070 ts_tablet_manager.cc:1434] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.095s	user 0.009s	sys 0.004s
I20260709 12:24:33.164526  5859 tablet_service.cc:1511] Processing CreateTablet for tablet 685933c4b2364e1c90d92edb4c091799 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:24:33.161299  6074 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.171725  5859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 685933c4b2364e1c90d92edb4c091799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.185422  5784 tablet_service.cc:1511] Processing CreateTablet for tablet 685933c4b2364e1c90d92edb4c091799 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:24:33.187784  5784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 685933c4b2364e1c90d92edb4c091799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.192412  5934 tablet_service.cc:1511] Processing CreateTablet for tablet 238b79ac4a244a85a83ab0d01b84e117 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:24:33.194802  5934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 238b79ac4a244a85a83ab0d01b84e117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.196219  6074 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:33.202735  5901 tablet.cc:2406] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:24:33.176717  6080 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:33.216288  6070 tablet_bootstrap.cc:492] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:33.238415  5788 tablet_service.cc:1511] Processing CreateTablet for tablet 238b79ac4a244a85a83ab0d01b84e117 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:24:33.240236  5788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 238b79ac4a244a85a83ab0d01b84e117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.246454  6074 ts_tablet_manager.cc:1434] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.132s	user 0.007s	sys 0.009s
I20260709 12:24:33.247766  6011 tablet_service.cc:1511] Processing CreateTablet for tablet 238b79ac4a244a85a83ab0d01b84e117 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:24:33.250967  6074 tablet_bootstrap.cc:492] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:33.251585  6049 heartbeater.cc:499] Master 127.2.4.254:41049 was elected leader, sending a full tablet report...
I20260709 12:24:33.251044  6011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 238b79ac4a244a85a83ab0d01b84e117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.248396  6069 tablet_bootstrap.cc:492] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:33.252430  5935 tablet_service.cc:1511] Processing CreateTablet for tablet d4765e1fc00c485086dc883bea4b87f7 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:24:33.258087  6069 ts_tablet_manager.cc:1403] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.182s	user 0.020s	sys 0.014s
I20260709 12:24:33.258301  6077 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.259109  6077 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:33.263451  6077 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:33.263589  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:24:33.265781  5913 leader_election.cc:343] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.295385  5935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4765e1fc00c485086dc883bea4b87f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.296437  5939 tablet_service.cc:1511] Processing CreateTablet for tablet 07d59a4ebda543dfbfa20847a8e9e4a7 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:24:33.261510  6069 raft_consensus.cc:359] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:33.305982  6071 tablet_bootstrap.cc:492] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:33.310686  5936 tablet_service.cc:1511] Processing CreateTablet for tablet 7d95abc0e3a9452eb13b2625af34008e (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:24:33.330338  5861 tablet_service.cc:1511] Processing CreateTablet for tablet d4765e1fc00c485086dc883bea4b87f7 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:24:33.330788  6011 tablet_service.cc:1511] Processing CreateTablet for tablet d4765e1fc00c485086dc883bea4b87f7 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:24:33.332124  6009 tablet_service.cc:1511] Processing CreateTablet for tablet e55fd31b2e414885a9bd5c9b40e04e19 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:24:33.336174  6010 tablet_service.cc:1511] Processing CreateTablet for tablet 7d95abc0e3a9452eb13b2625af34008e (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:24:33.332351  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:33.336009  5789 tablet_service.cc:1511] Processing CreateTablet for tablet e55fd31b2e414885a9bd5c9b40e04e19 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:24:33.339373  6071 ts_tablet_manager.cc:1403] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.215s	user 0.017s	sys 0.004s
I20260709 12:24:33.337494  6069 raft_consensus.cc:385] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.343281  6069 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.344197  6069 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:33.345505  5786 tablet_service.cc:1511] Processing CreateTablet for tablet 7d95abc0e3a9452eb13b2625af34008e (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:24:33.346611  6069 ts_tablet_manager.cc:1434] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.001s
I20260709 12:24:33.336088  5787 tablet_service.cc:1511] Processing CreateTablet for tablet 07d59a4ebda543dfbfa20847a8e9e4a7 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:24:33.348563  6009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e55fd31b2e414885a9bd5c9b40e04e19. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:33.351743  5915 leader_election.cc:343] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.352380  5915 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:33.353686  6079 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.355176  5789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e55fd31b2e414885a9bd5c9b40e04e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.356329  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:33.341521  5939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07d59a4ebda543dfbfa20847a8e9e4a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.363212  5936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d95abc0e3a9452eb13b2625af34008e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:33.359203  5763 leader_election.cc:343] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.359398  6071 raft_consensus.cc:359] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:33.363910  5938 tablet_service.cc:1511] Processing CreateTablet for tablet 6ad5821a38aa41d7abd76509a12a97df (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:24:33.364565  6071 raft_consensus.cc:385] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.365074  6071 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.365455  5938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ad5821a38aa41d7abd76509a12a97df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.366158  6071 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:33.368846  6071 ts_tablet_manager.cc:1434] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 12:24:33.370138  6071 tablet_bootstrap.cc:492] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:33.379374  6010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d95abc0e3a9452eb13b2625af34008e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.381021  5786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d95abc0e3a9452eb13b2625af34008e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.382929  5787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07d59a4ebda543dfbfa20847a8e9e4a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.385977  6088 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.386592  6088 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:33.386662  6014 tablet_service.cc:1511] Processing CreateTablet for tablet 6ad5821a38aa41d7abd76509a12a97df (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:24:33.388909  6014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ad5821a38aa41d7abd76509a12a97df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.400040  6069 tablet_bootstrap.cc:492] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:33.404073  6071 tablet_bootstrap.cc:654] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.414382  5783 tablet_service.cc:1511] Processing CreateTablet for tablet 860d4f6b513041829ed9cd83880edc5c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:24:33.419019  5783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 860d4f6b513041829ed9cd83880edc5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.423410  5937 tablet_service.cc:1511] Processing CreateTablet for tablet 860d4f6b513041829ed9cd83880edc5c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:24:33.425240  5937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 860d4f6b513041829ed9cd83880edc5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.429939  5861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4765e1fc00c485086dc883bea4b87f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.423331  6088 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:33.442477  5782 tablet_service.cc:1511] Processing CreateTablet for tablet 402bd52b71ce4044b0ec86bf6f8d73fc (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:24:33.444186  5782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402bd52b71ce4044b0ec86bf6f8d73fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.454919  6011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4765e1fc00c485086dc883bea4b87f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.456873  6070 tablet_bootstrap.cc:654] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.498049  6013 tablet_service.cc:1511] Processing CreateTablet for tablet 402bd52b71ce4044b0ec86bf6f8d73fc (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:24:33.501381  6013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402bd52b71ce4044b0ec86bf6f8d73fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.497992  6074 tablet_bootstrap.cc:654] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.509449  5939 tablet_service.cc:1511] Processing CreateTablet for tablet 41bc2065e43e40c98ee821925758a3f3 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:24:33.511194  5939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41bc2065e43e40c98ee821925758a3f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.539036  6069 tablet_bootstrap.cc:654] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.541525  5785 tablet_service.cc:1511] Processing CreateTablet for tablet 41bc2065e43e40c98ee821925758a3f3 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:24:33.543148  5785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41bc2065e43e40c98ee821925758a3f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.553555  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:33.560808  5863 tablet_service.cc:1511] Processing CreateTablet for tablet 07d59a4ebda543dfbfa20847a8e9e4a7 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
W20260709 12:24:33.564738  5987 leader_election.cc:343] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.579182  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:33.580065  6024 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:33.575012  5862 tablet_service.cc:1511] Processing CreateTablet for tablet e55fd31b2e414885a9bd5c9b40e04e19 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:24:33.580667  5934 tablet_service.cc:1511] Processing CreateTablet for tablet a80a3917fd9748bc8ae4817757081f09 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:24:33.588358  5781 tablet_service.cc:1511] Processing CreateTablet for tablet edbd0e138da244d48f133d23b4aa1eda (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:24:33.590233  5934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80a3917fd9748bc8ae4817757081f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.590080  6012 tablet_service.cc:1511] Processing CreateTablet for tablet 41bc2065e43e40c98ee821925758a3f3 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:24:33.591454  5781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edbd0e138da244d48f133d23b4aa1eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.593712  6008 tablet_service.cc:1511] Processing CreateTablet for tablet a80a3917fd9748bc8ae4817757081f09 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:24:33.591593  6012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41bc2065e43e40c98ee821925758a3f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.594808  5863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07d59a4ebda543dfbfa20847a8e9e4a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.595319  6008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80a3917fd9748bc8ae4817757081f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.599994  5862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e55fd31b2e414885a9bd5c9b40e04e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.603253  5860 tablet_service.cc:1511] Processing CreateTablet for tablet 6ad5821a38aa41d7abd76509a12a97df (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:24:33.604102  5858 tablet_service.cc:1511] Processing CreateTablet for tablet 402bd52b71ce4044b0ec86bf6f8d73fc (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:24:33.604930  5860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ad5821a38aa41d7abd76509a12a97df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.606832  5864 tablet_service.cc:1511] Processing CreateTablet for tablet 860d4f6b513041829ed9cd83880edc5c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:24:33.607450  5857 tablet_service.cc:1511] Processing CreateTablet for tablet a80a3917fd9748bc8ae4817757081f09 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:24:33.610741  5856 tablet_service.cc:1511] Processing CreateTablet for tablet edbd0e138da244d48f133d23b4aa1eda (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:24:33.612635  5858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402bd52b71ce4044b0ec86bf6f8d73fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.614068  5864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 860d4f6b513041829ed9cd83880edc5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.614276  5765 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:33.616456  6080 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:33.616956  6080 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.617353  6080 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.618822  5856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edbd0e138da244d48f133d23b4aa1eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.622087  5857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80a3917fd9748bc8ae4817757081f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.624440  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:33.632889  6071 tablet_bootstrap.cc:492] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:33.634411  6070 tablet_bootstrap.cc:492] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:33.635048  6070 ts_tablet_manager.cc:1403] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.435s	user 0.020s	sys 0.009s
I20260709 12:24:33.633544  6071 ts_tablet_manager.cc:1403] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.264s	user 0.026s	sys 0.016s
I20260709 12:24:33.640605  5933 tablet_service.cc:1511] Processing CreateTablet for tablet edbd0e138da244d48f133d23b4aa1eda (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:24:33.642824  5933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edbd0e138da244d48f133d23b4aa1eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.641413  6071 raft_consensus.cc:359] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:33.650638  6071 raft_consensus.cc:385] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.651141  6071 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.652211  6071 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:33.653215  6079 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.653829  6079 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:33.658141  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:33.662624  6079 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:24:33.664111  5913 leader_election.cc:343] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.669173  6080 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:33.644888  6070 raft_consensus.cc:359] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:33.670594  6070 raft_consensus.cc:385] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.671008  6070 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.672039  6070 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:33.673362  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:33.674870  6070 ts_tablet_manager.cc:1434] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.039s	user 0.005s	sys 0.002s
W20260709 12:24:33.675345  5763 leader_election.cc:343] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:33.650749  5840 leader_election.cc:343] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.692261  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:33.693431  5798 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:24:33.692319  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:33.711181  5682 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 63972.8us
W20260709 12:24:33.723716  5840 leader_election.cc:343] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.730420  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:33.732733  6079 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.733363  6079 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:33.737063  6077 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.721717  6080 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:33.744439  6070 tablet_bootstrap.cc:492] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:33.750480  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:33.750604  6079 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:33.752513  5798 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:33.754762  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:33.754819  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:33.757695  6071 ts_tablet_manager.cc:1434] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.120s	user 0.010s	sys 0.007s
I20260709 12:24:33.758622  6069 tablet_bootstrap.cc:492] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:33.759016  6070 tablet_bootstrap.cc:654] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:33.771855  5915 leader_election.cc:343] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.773335  6069 ts_tablet_manager.cc:1403] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.415s	user 0.029s	sys 0.011s
I20260709 12:24:33.775472  6077 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.775787  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:24:33.777251  5915 leader_election.cc:343] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.777769  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:33.776238  6077 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:33.779856  6079 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.779377  6100 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:33.780704  6100 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.781183  6100 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.784729  6069 raft_consensus.cc:359] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:33.787711  6071 tablet_bootstrap.cc:492] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:33.788043  6069 raft_consensus.cc:385] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.788446  6069 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.789300  6069 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:33.800678  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:33.801574  6077 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:24:33.808058  5838 leader_election.cc:343] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.810210  6074 tablet_bootstrap.cc:492] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:33.820307  6074 ts_tablet_manager.cc:1403] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.570s	user 0.024s	sys 0.009s
I20260709 12:24:33.821611  6069 ts_tablet_manager.cc:1434] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.048s	user 0.020s	sys 0.015s
I20260709 12:24:33.823091  6069 tablet_bootstrap.cc:492] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:33.831077  6074 raft_consensus.cc:359] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:33.831743  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:33.831864  6074 raft_consensus.cc:385] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:24:33.838743  5840 leader_election.cc:343] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.839416  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:33.839917  5783 tablet_service.cc:1511] Processing CreateTablet for tablet aa54a35cf4c54bb1a1c28658c11d174c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:24:33.840574  6101 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.841041  6074 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.842254  6074 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:33.844348  5862 tablet_service.cc:1511] Processing CreateTablet for tablet e657475654794984a13c42a668f3158b (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:24:33.841426  5783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa54a35cf4c54bb1a1c28658c11d174c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.845837  5862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e657475654794984a13c42a668f3158b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.850898  6074 ts_tablet_manager.cc:1434] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 12:24:33.852139  6074 tablet_bootstrap.cc:492] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:33.854604  5781 tablet_service.cc:1511] Processing CreateTablet for tablet e657475654794984a13c42a668f3158b (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:24:33.836320  6069 tablet_bootstrap.cc:654] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.856359  5781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e657475654794984a13c42a668f3158b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.860721  6074 tablet_bootstrap.cc:654] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.864156  5864 tablet_service.cc:1511] Processing CreateTablet for tablet 9a7ca03d5d3d4b13bee5481a90dc7bdc (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:24:33.865795  5864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a7ca03d5d3d4b13bee5481a90dc7bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.872412  6012 tablet_service.cc:1511] Processing CreateTablet for tablet e657475654794984a13c42a668f3158b (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
W20260709 12:24:33.873039  5745 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:33.873999  5937 tablet_service.cc:1511] Processing CreateTablet for tablet 9a7ca03d5d3d4b13bee5481a90dc7bdc (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:24:33.875034  5990 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:33.875449  5937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a7ca03d5d3d4b13bee5481a90dc7bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.876034  6088 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.874003  6012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e657475654794984a13c42a668f3158b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.893927  5856 tablet_service.cc:1511] Processing CreateTablet for tablet f3c3647dbbb74fbd86870b12d1409976 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:24:33.896246  5856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3c3647dbbb74fbd86870b12d1409976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.901041  5785 tablet_service.cc:1511] Processing CreateTablet for tablet f3c3647dbbb74fbd86870b12d1409976 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:24:33.903295  5785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3c3647dbbb74fbd86870b12d1409976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.903105  6008 tablet_service.cc:1511] Processing CreateTablet for tablet aa54a35cf4c54bb1a1c28658c11d174c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:24:33.870203  5933 tablet_service.cc:1511] Processing CreateTablet for tablet aa54a35cf4c54bb1a1c28658c11d174c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:24:33.909885  5933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa54a35cf4c54bb1a1c28658c11d174c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.911706  5939 tablet_service.cc:1511] Processing CreateTablet for tablet f3c3647dbbb74fbd86870b12d1409976 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:24:33.912739  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:33.913194  5939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3c3647dbbb74fbd86870b12d1409976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.915627  6011 tablet_service.cc:1511] Processing CreateTablet for tablet 9a7ca03d5d3d4b13bee5481a90dc7bdc (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:24:33.919348  6071 tablet_bootstrap.cc:700] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent opening tablet: real 0.126s	user 0.001s	sys 0.003s
I20260709 12:24:33.920163  6071 tablet_bootstrap.cc:654] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.905591  6008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa54a35cf4c54bb1a1c28658c11d174c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.936398  6011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a7ca03d5d3d4b13bee5481a90dc7bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.945245  6088 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.944885  5787 tablet_service.cc:1511] Processing CreateTablet for tablet bc2de53e733442a69e97d39affab60ba (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:24:33.961802  5861 tablet_service.cc:1511] Processing CreateTablet for tablet bc2de53e733442a69e97d39affab60ba (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:24:33.971246  5861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc2de53e733442a69e97d39affab60ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.971673  5787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc2de53e733442a69e97d39affab60ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.976612  5939 tablet_service.cc:1511] Processing CreateTablet for tablet 4fa58bdc1b834feda10d16a00cd81355 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:24:33.977969  6010 tablet_service.cc:1511] Processing CreateTablet for tablet bc2de53e733442a69e97d39affab60ba (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:24:33.987556  6010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc2de53e733442a69e97d39affab60ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.989193  5939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fa58bdc1b834feda10d16a00cd81355. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.954275  6100 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:33.989418  6009 tablet_service.cc:1511] Processing CreateTablet for tablet 4fa58bdc1b834feda10d16a00cd81355 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:24:33.990789  6009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fa58bdc1b834feda10d16a00cd81355. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.982928  6070 tablet_bootstrap.cc:492] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:33.994406  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:33.995256  5799 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.996706  6069 tablet_bootstrap.cc:492] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:33.996992  6071 tablet_bootstrap.cc:492] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:33.997311  6069 ts_tablet_manager.cc:1403] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.174s	user 0.029s	sys 0.051s
I20260709 12:24:33.997670  6071 ts_tablet_manager.cc:1403] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.210s	user 0.015s	sys 0.014s
I20260709 12:24:34.001773  6100 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:34.001902  6024 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:33.999995  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:24:34.015539  5913 leader_election.cc:343] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.007952  6070 ts_tablet_manager.cc:1403] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.272s	user 0.022s	sys 0.030s
I20260709 12:24:34.015933  6013 tablet_service.cc:1511] Processing CreateTablet for tablet 256f8c690fb2456f92b4035fc3bf5e23 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:24:34.017383  6013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 256f8c690fb2456f92b4035fc3bf5e23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.019483  6071 raft_consensus.cc:359] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:34.006572  5862 tablet_service.cc:1511] Processing CreateTablet for tablet 256f8c690fb2456f92b4035fc3bf5e23 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:24:34.033766  5799 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:34.034793  5862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 256f8c690fb2456f92b4035fc3bf5e23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.035518  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:34.036891  6100 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:34.037431  6100 raft_consensus.cc:697] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:34.038316  6100 consensus_queue.cc:237] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:34.007548  5783 tablet_service.cc:1511] Processing CreateTablet for tablet 4fa58bdc1b834feda10d16a00cd81355 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:24:34.011564  5935 tablet_service.cc:1511] Processing CreateTablet for tablet 256f8c690fb2456f92b4035fc3bf5e23 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:24:34.042902  5935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 256f8c690fb2456f92b4035fc3bf5e23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.008862  6069 raft_consensus.cc:359] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:34.026559  5765 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:34.033999  6071 raft_consensus.cc:385] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.019767  6070 raft_consensus.cc:359] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:34.010895  6074 tablet_bootstrap.cc:492] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:34.044215  6069 raft_consensus.cc:385] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.051367  6074 ts_tablet_manager.cc:1403] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.199s	user 0.022s	sys 0.043s
I20260709 12:24:34.044958  6071 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.045843  6069 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.058182  6071 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:34.059194  6069 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:34.062528  5783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fa58bdc1b834feda10d16a00cd81355. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.063606  5788 tablet_service.cc:1511] Processing CreateTablet for tablet 52af756fdd9b411c8c4b494e2552a3fb (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:24:34.044718  6070 raft_consensus.cc:385] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.066061  6070 raft_consensus.cc:740] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.067586  6070 consensus_queue.cc:260] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:34.047134  6080 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:34.065521  5788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52af756fdd9b411c8c4b494e2552a3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.060930  6071 ts_tablet_manager.cc:1434] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.049s	user 0.004s	sys 0.003s
I20260709 12:24:34.101840  5937 tablet_service.cc:1511] Processing CreateTablet for tablet 52af756fdd9b411c8c4b494e2552a3fb (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:24:34.102607  6071 tablet_bootstrap.cc:492] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:34.088991  6011 tablet_service.cc:1511] Processing CreateTablet for tablet 048b40ab425f451c983713b7df96c732 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:24:34.104368  5939 tablet_service.cc:1511] Processing CreateTablet for tablet d6803fb2b32049f3935eeb6331dcde39 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:24:34.104403  6011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 048b40ab425f451c983713b7df96c732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.085415  6080 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:34.106355  6080 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:34.108042  6074 raft_consensus.cc:359] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:34.096808  5782 tablet_service.cc:1511] Processing CreateTablet for tablet 048b40ab425f451c983713b7df96c732 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:24:34.098680  5858 tablet_service.cc:1511] Processing CreateTablet for tablet 048b40ab425f451c983713b7df96c732 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:24:34.109984  5782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 048b40ab425f451c983713b7df96c732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.110236  5858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 048b40ab425f451c983713b7df96c732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.091202  5864 tablet_service.cc:1511] Processing CreateTablet for tablet 52af756fdd9b411c8c4b494e2552a3fb (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:24:34.115061  5864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52af756fdd9b411c8c4b494e2552a3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.108886  6074 raft_consensus.cc:385] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.124151  6009 tablet_service.cc:1511] Processing CreateTablet for tablet d6803fb2b32049f3935eeb6331dcde39 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:24:34.126286  5786 tablet_service.cc:1511] Processing CreateTablet for tablet d6803fb2b32049f3935eeb6331dcde39 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:24:34.127533  6088 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.128110  6088 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:34.128279  5861 tablet_service.cc:1511] Processing CreateTablet for tablet c0e0b097c72c4c748e146f8303c90f0c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:24:34.129436  5934 tablet_service.cc:1511] Processing CreateTablet for tablet c0e0b097c72c4c748e146f8303c90f0c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:24:34.127985  5786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6803fb2b32049f3935eeb6331dcde39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.118997  5937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52af756fdd9b411c8c4b494e2552a3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.130455  6088 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:34.117966  6074 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.132910  6070 ts_tablet_manager.cc:1434] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.116s	user 0.005s	sys 0.005s
I20260709 12:24:34.136777  6009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6803fb2b32049f3935eeb6331dcde39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.146142  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:34.146947  5949 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:34.148103  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:34.148674  6074 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:34.150213  5863 tablet_service.cc:1511] Processing CreateTablet for tablet d04ce59a08f3427581e5e6b5d87ad757 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:24:34.151206  6074 ts_tablet_manager.cc:1434] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.077s	user 0.008s	sys 0.000s
I20260709 12:24:34.152334  6074 tablet_bootstrap.cc:492] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:34.153116  6101 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.153667  6101 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:34.155866  6101 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:34.157435  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:34.157824  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:34.158427  6088 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:34.158918  6088 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.159324  6088 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:34.159817  5840 leader_election.cc:343] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.160518  6100 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.161038  6100 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:34.161960  5697 catalog_manager.cc:5697] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 1 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:34.164346  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:34.165341  6024 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 1: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:24:34.166155  5934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0e0b097c72c4c748e146f8303c90f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.167078  5939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6803fb2b32049f3935eeb6331dcde39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.167443  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:34.168789  5990 leader_election.cc:343] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.169209  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:34.170239  6079 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.171329  6012 tablet_service.cc:1511] Processing CreateTablet for tablet c0e0b097c72c4c748e146f8303c90f0c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:24:34.164381  6100 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:34.137953  6069 ts_tablet_manager.cc:1434] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.132s	user 0.010s	sys 0.000s
I20260709 12:24:34.179917  6012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0e0b097c72c4c748e146f8303c90f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.180027  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:34.141196  5861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0e0b097c72c4c748e146f8303c90f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.186467  6071 tablet_bootstrap.cc:654] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.190012  5863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d04ce59a08f3427581e5e6b5d87ad757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.193133  6069 tablet_bootstrap.cc:492] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
W20260709 12:24:34.199589  5840 leader_election.cc:343] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.200202  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:34.201584  6101 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:34.203001  6070 tablet_bootstrap.cc:492] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:34.214715  5787 tablet_service.cc:1511] Processing CreateTablet for tablet d04ce59a08f3427581e5e6b5d87ad757 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:24:34.215402  6088 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:34.216816  6079 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:34.225919  6069 tablet_bootstrap.cc:654] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.233316  6088 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:34.234268  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:34.234913  5949 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.239271  5937 tablet_service.cc:1511] Processing CreateTablet for tablet d04ce59a08f3427581e5e6b5d87ad757 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:24:34.239822  5787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d04ce59a08f3427581e5e6b5d87ad757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.239766  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:34.241847  5937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d04ce59a08f3427581e5e6b5d87ad757. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:34.242527  5990 leader_election.cc:343] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.254631  5696 catalog_manager.cc:5697] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:34.245003  6088 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.258287  6088 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:34.262774  6088 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:34.275223  6080 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.275938  6080 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:34.295902  6101 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.296808  6101 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:34.297415  6070 tablet_bootstrap.cc:654] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.304832  6101 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:34.306816  6010 tablet_service.cc:1511] Processing CreateTablet for tablet 1ba7af1a4a744df6bc8c20ec72e21519 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:24:34.307264  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:34.308336  6010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ba7af1a4a744df6bc8c20ec72e21519. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.310771  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:34.311801  5948 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:34.315536  6071 tablet_bootstrap.cc:492] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:34.316182  6071 ts_tablet_manager.cc:1403] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.214s	user 0.012s	sys 0.025s
W20260709 12:24:34.316876  5765 leader_election.cc:343] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.315295  5838 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:34.318862  6101 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:34.319434  6101 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.322552  6101 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:34.321051  6110 log.cc:927] Time spent T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:24:34.323096  5782 tablet_service.cc:1511] Processing CreateTablet for tablet 1ba7af1a4a744df6bc8c20ec72e21519 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:24:34.290761  6080 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:34.321442  5935 tablet_service.cc:1511] Processing CreateTablet for tablet fee70b2d75314f788ad873b1950fb890 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:24:34.320114  6074 tablet_bootstrap.cc:654] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.337777  5782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ba7af1a4a744df6bc8c20ec72e21519. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.353381  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:34.354346  5948 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:34.362351  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:34.363436  6088 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:34.363978  6088 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.364342  6088 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.367264  5939 tablet_service.cc:1511] Processing CreateTablet for tablet 6ed8b895df664a3b8f7934c37fd8f181 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:24:34.369925  5856 tablet_service.cc:1511] Processing CreateTablet for tablet 1ba7af1a4a744df6bc8c20ec72e21519 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:24:34.371438  5856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ba7af1a4a744df6bc8c20ec72e21519. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.371654  5783 tablet_service.cc:1511] Processing CreateTablet for tablet fee70b2d75314f788ad873b1950fb890 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:24:34.372406  6088 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:34.373171  5783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fee70b2d75314f788ad873b1950fb890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.373797  5935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fee70b2d75314f788ad873b1950fb890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.374377  5949 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:34.374576  6088 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:34.374789  5862 tablet_service.cc:1511] Processing CreateTablet for tablet 6ed8b895df664a3b8f7934c37fd8f181 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:24:34.376161  5862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed8b895df664a3b8f7934c37fd8f181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.379341  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:34.380287  6024 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 1: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:24:34.380704  6008 tablet_service.cc:1511] Processing CreateTablet for tablet fee70b2d75314f788ad873b1950fb890 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:24:34.382296  6008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fee70b2d75314f788ad873b1950fb890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.384967  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:34.385502  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:34.365406  6069 tablet_bootstrap.cc:492] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:34.386256  5949 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.386476  5939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed8b895df664a3b8f7934c37fd8f181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.386741  6069 ts_tablet_manager.cc:1403] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.194s	user 0.012s	sys 0.007s
I20260709 12:24:34.390782  6071 raft_consensus.cc:359] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:34.402808  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:34.404345  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:34.392187  6088 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:34.409883  6071 raft_consensus.cc:385] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.410288  6071 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.411158  6071 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:34.413987  6071 ts_tablet_manager.cc:1434] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.097s	user 0.006s	sys 0.005s
I20260709 12:24:34.413823  6069 raft_consensus.cc:359] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:34.414847  6069 raft_consensus.cc:385] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.415270  6071 tablet_bootstrap.cc:492] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:34.410414  6070 tablet_bootstrap.cc:492] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:34.428268  5949 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:34.428710  6011 tablet_service.cc:1511] Processing CreateTablet for tablet 6ed8b895df664a3b8f7934c37fd8f181 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:24:34.429062  5786 tablet_service.cc:1511] Processing CreateTablet for tablet d45107f59eee48f79ae2b6006319cd62 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:24:34.429668  6088 raft_consensus.cc:697] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:34.430506  6011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed8b895df664a3b8f7934c37fd8f181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.430599  5786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d45107f59eee48f79ae2b6006319cd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.431725  6074 tablet_bootstrap.cc:492] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:34.432246  6074 ts_tablet_manager.cc:1403] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.280s	user 0.015s	sys 0.001s
I20260709 12:24:34.423755  6070 ts_tablet_manager.cc:1403] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.221s	user 0.012s	sys 0.011s
I20260709 12:24:34.430771  6088 consensus_queue.cc:237] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:34.434391  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:34.435667  6074 raft_consensus.cc:359] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:34.436513  6074 raft_consensus.cc:385] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.436854  6074 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.437449  6070 raft_consensus.cc:359] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:34.446817  5864 tablet_service.cc:1511] Processing CreateTablet for tablet d45107f59eee48f79ae2b6006319cd62 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 12:24:34.441540  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 126303us
I20260709 12:24:34.448418  5864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d45107f59eee48f79ae2b6006319cd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.415287  6069 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.456187  6070 raft_consensus.cc:385] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.479494  5934 tablet_service.cc:1511] Processing CreateTablet for tablet d45107f59eee48f79ae2b6006319cd62 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:24:34.480969  5934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d45107f59eee48f79ae2b6006319cd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.487082  5861 tablet_service.cc:1511] Processing CreateTablet for tablet 428a50885681465a98089cb853b233b9 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:24:34.488700  5861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 428a50885681465a98089cb853b233b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.488277  5933 tablet_service.cc:1511] Processing CreateTablet for tablet 2dcbedb48e03420dbf7f8646e65b1886 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:24:34.489833  6079 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:34.489785  6074 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:34.491206  6012 tablet_service.cc:1511] Processing CreateTablet for tablet 428a50885681465a98089cb853b233b9 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:24:34.492552  6074 ts_tablet_manager.cc:1434] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.002s
I20260709 12:24:34.492619  6012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 428a50885681465a98089cb853b233b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.463794  6120 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:34.502242  6120 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:34.503314  6120 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:34.508980  5933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dcbedb48e03420dbf7f8646e65b1886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.512482  6074 tablet_bootstrap.cc:492] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:34.495095  6070 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.522635  5787 tablet_service.cc:1511] Processing CreateTablet for tablet 428a50885681465a98089cb853b233b9 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
W20260709 12:24:34.501528  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 67255us
I20260709 12:24:34.527997  6013 tablet_service.cc:1511] Processing CreateTablet for tablet 2dcbedb48e03420dbf7f8646e65b1886 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:24:34.530373  5788 tablet_service.cc:1511] Processing CreateTablet for tablet 2dcbedb48e03420dbf7f8646e65b1886 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:24:34.532392  5863 tablet_service.cc:1511] Processing CreateTablet for tablet 8e301d9d5b914f3b9ec07dea273d5126 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:24:34.565119  5788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dcbedb48e03420dbf7f8646e65b1886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.566838  6013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dcbedb48e03420dbf7f8646e65b1886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.495591  6101 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:34.550072  6080 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:34.584805  5874 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:24:34.585681  5863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e301d9d5b914f3b9ec07dea273d5126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.587220  5697 catalog_manager.cc:5697] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 1 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:34.600596  5787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 428a50885681465a98089cb853b233b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.600759  5783 tablet_service.cc:1511] Processing CreateTablet for tablet 4ef7a99e80494a989eda0b1a37482656 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:24:34.602412  5783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ef7a99e80494a989eda0b1a37482656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.608680  5858 tablet_service.cc:1511] Processing CreateTablet for tablet 4ef7a99e80494a989eda0b1a37482656 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:24:34.609176  6101 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:34.610162  5858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ef7a99e80494a989eda0b1a37482656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.610715  6014 tablet_service.cc:1511] Processing CreateTablet for tablet c3c69fa4d49547f8a9e76c055c3ae7be (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:24:34.630005  5781 tablet_service.cc:1511] Processing CreateTablet for tablet 85d9b48bdd4046d788b310a73871c31c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:24:34.630540  6014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3c69fa4d49547f8a9e76c055c3ae7be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.631413  5781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d9b48bdd4046d788b310a73871c31c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.611748  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:34.633517  5857 tablet_service.cc:1511] Processing CreateTablet for tablet 60934281a651439a87cd8c11702f67d2 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:24:34.634436  5948 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 1: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:24:34.635079  5857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60934281a651439a87cd8c11702f67d2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:34.637526  5913 consensus_peers.cc:597] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:34.638657  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:34.639626  5874 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:34.641099  5763 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:34.642179  6080 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:34.642694  6080 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.643035  6080 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.517880  6069 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:34.612531  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:34.612926  5937 tablet_service.cc:1511] Processing CreateTablet for tablet 8e301d9d5b914f3b9ec07dea273d5126 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:24:34.613972  5782 tablet_service.cc:1511] Processing CreateTablet for tablet c3c69fa4d49547f8a9e76c055c3ae7be (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:24:34.617502  5860 tablet_service.cc:1511] Processing CreateTablet for tablet c3c69fa4d49547f8a9e76c055c3ae7be (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:24:34.620903  5936 tablet_service.cc:1511] Processing CreateTablet for tablet 85d9b48bdd4046d788b310a73871c31c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:24:34.652514  5936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d9b48bdd4046d788b310a73871c31c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.652761  6117 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.653373  6117 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:34.656106  5860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3c69fa4d49547f8a9e76c055c3ae7be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.660334  5938 tablet_service.cc:1511] Processing CreateTablet for tablet 60934281a651439a87cd8c11702f67d2 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:24:34.661859  5938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60934281a651439a87cd8c11702f67d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.661677  6070 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:34.662652  5782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3c69fa4d49547f8a9e76c055c3ae7be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.622912  5939 tablet_service.cc:1511] Processing CreateTablet for tablet 4ef7a99e80494a989eda0b1a37482656 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:24:34.623029  6009 tablet_service.cc:1511] Processing CreateTablet for tablet 8e301d9d5b914f3b9ec07dea273d5126 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:24:34.625397  6007 tablet_service.cc:1511] Processing CreateTablet for tablet 85d9b48bdd4046d788b310a73871c31c (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:24:34.628208  6071 tablet_bootstrap.cc:654] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.678485  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:34.679888  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:34.680881  6101 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:34.690192  5785 tablet_service.cc:1511] Processing CreateTablet for tablet 6a1efdcc52fa4f98acdbe3c7f5c38570 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:24:34.692375  5785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a1efdcc52fa4f98acdbe3c7f5c38570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.681970  6088 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.694541  6008 tablet_service.cc:1511] Processing CreateTablet for tablet 60934281a651439a87cd8c11702f67d2 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:24:34.694399  6088 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:34.710242  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:34.713236  5838 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:34.714509  6101 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:34.715553  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:34.716367  5874 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:34.716933  6069 ts_tablet_manager.cc:1434] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.330s	user 0.006s	sys 0.009s
I20260709 12:24:34.717190  6009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e301d9d5b914f3b9ec07dea273d5126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.719753  5939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ef7a99e80494a989eda0b1a37482656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.721892  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:34.722215  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:34.707347  6117 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:24:34.723495  5990 leader_election.cc:343] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.706257  6088 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:34.727952  5937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e301d9d5b914f3b9ec07dea273d5126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.730422  5697 catalog_manager.cc:5697] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 1 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:34.735093  5932 tablet_service.cc:1511] Processing CreateTablet for tablet 6a1efdcc52fa4f98acdbe3c7f5c38570 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:24:34.736809  5932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a1efdcc52fa4f98acdbe3c7f5c38570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.736989  6074 tablet_bootstrap.cc:654] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.740906  6006 tablet_service.cc:1511] Processing CreateTablet for tablet 48c0a9da01454a2794160d9c0ff08ea9 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:24:34.713073  5861 tablet_service.cc:1511] Processing CreateTablet for tablet 6a1efdcc52fa4f98acdbe3c7f5c38570 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:24:34.721149  6080 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:34.740896  5862 tablet_service.cc:1511] Processing CreateTablet for tablet 48c0a9da01454a2794160d9c0ff08ea9 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:24:34.776335  5931 tablet_service.cc:1511] Processing CreateTablet for tablet 02e8ec653e484ba58498cc78f412de75 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:24:34.777833  6008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60934281a651439a87cd8c11702f67d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.790349  5862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48c0a9da01454a2794160d9c0ff08ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.791913  6007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d9b48bdd4046d788b310a73871c31c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.792186  6011 tablet_service.cc:1511] Processing CreateTablet for tablet 02e8ec653e484ba58498cc78f412de75 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:24:34.756023  6120 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:34.794127  6120 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.794727  6120 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.795270  6129 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.795886  6129 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:34.801841  5931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e8ec653e484ba58498cc78f412de75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.811299  6129 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:34.783392  6069 tablet_bootstrap.cc:492] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:34.825865  6011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e8ec653e484ba58498cc78f412de75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.828999  5786 tablet_service.cc:1511] Processing CreateTablet for tablet 48c0a9da01454a2794160d9c0ff08ea9 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:24:34.830574  5786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48c0a9da01454a2794160d9c0ff08ea9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:34.765607  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 43528.9us
I20260709 12:24:34.854384  5861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a1efdcc52fa4f98acdbe3c7f5c38570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.846581  6079 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.874110  6079 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:34.876634  6079 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:34.873347  6006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48c0a9da01454a2794160d9c0ff08ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.889460  5783 tablet_service.cc:1511] Processing CreateTablet for tablet 02e8ec653e484ba58498cc78f412de75 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:24:34.891254  5783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e8ec653e484ba58498cc78f412de75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.894680  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:34.895565  5798 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:34.897724  5915 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:34.898667  6079 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:34.899130  6079 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.899474  6079 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.900054  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:34.901902  6070 ts_tablet_manager.cc:1434] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.468s	user 0.034s	sys 0.052s
I20260709 12:24:34.903326  6070 tablet_bootstrap.cc:492] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
W20260709 12:24:34.904065  5765 leader_election.cc:343] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.904559  6080 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:34.907749  6079 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:24:34.911783  5745 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:34.917683  5860 tablet_service.cc:1511] Processing CreateTablet for tablet 8d0c096d28fc46d3be987a091fcb5533 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:24:34.923749  5933 tablet_service.cc:1511] Processing CreateTablet for tablet 8d0c096d28fc46d3be987a091fcb5533 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:24:34.933817  6069 tablet_bootstrap.cc:654] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.973338  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:34.974179  5874 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:34.975999  6079 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:34.979197  6074 tablet_bootstrap.cc:492] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:34.978798  6013 tablet_service.cc:1511] Processing CreateTablet for tablet 8d0c096d28fc46d3be987a091fcb5533 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:24:34.979898  6074 ts_tablet_manager.cc:1403] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.484s	user 0.021s	sys 0.007s
I20260709 12:24:34.980691  6013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d0c096d28fc46d3be987a091fcb5533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.966691  5860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d0c096d28fc46d3be987a091fcb5533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.983376  6088 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:34.969542  5933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d0c096d28fc46d3be987a091fcb5533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:34.988830  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:34.989609  5798 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.003994  5788 tablet_service.cc:1511] Processing CreateTablet for tablet cfa3fcaa83734d509f6b31c3874873be (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:24:35.004336  6120 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:35.040895  5782 tablet_service.cc:1511] Processing CreateTablet for tablet e42697dffe5a495ea1ba4a111df5ba39 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:24:35.042626  6120 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:35.006973  6074 raft_consensus.cc:359] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.043169  6008 tablet_service.cc:1511] Processing CreateTablet for tablet e42697dffe5a495ea1ba4a111df5ba39 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:24:35.015825  6071 tablet_bootstrap.cc:492] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:35.044358  6071 ts_tablet_manager.cc:1403] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.629s	user 0.016s	sys 0.004s
I20260709 12:24:35.044840  6008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e42697dffe5a495ea1ba4a111df5ba39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.048170  6071 raft_consensus.cc:359] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:35.049237  6071 raft_consensus.cc:385] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.049718  6071 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.051352  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:35.051462  6071 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:35.052301  5873 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.054375  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:35.055274  5872 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.056862  6071 ts_tablet_manager.cc:1434] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.004s
I20260709 12:24:35.058216  6071 tablet_bootstrap.cc:492] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
W20260709 12:24:35.025949  5763 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 284018us
I20260709 12:24:35.026089  5936 tablet_service.cc:1511] Processing CreateTablet for tablet cfa3fcaa83734d509f6b31c3874873be (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:24:35.061141  5936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfa3fcaa83734d509f6b31c3874873be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.064491  5873 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:35.043629  6074 raft_consensus.cc:385] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.020455  6132 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:35.060671  5856 tablet_service.cc:1511] Processing CreateTablet for tablet e42697dffe5a495ea1ba4a111df5ba39 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:24:35.066859  5935 tablet_service.cc:1511] Processing CreateTablet for tablet ba269f0998844ab5b461587eaa90df32 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:24:35.067488  5788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfa3fcaa83734d509f6b31c3874873be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.034067  5864 tablet_service.cc:1511] Processing CreateTablet for tablet cfa3fcaa83734d509f6b31c3874873be (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:24:35.030339  6070 tablet_bootstrap.cc:654] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.070353  5872 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:35.089200  5864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfa3fcaa83734d509f6b31c3874873be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.090353  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:35.091527  6120 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:35.091832  5935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba269f0998844ab5b461587eaa90df32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.092188  6120 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:35.071899  6009 tablet_service.cc:1511] Processing CreateTablet for tablet ba269f0998844ab5b461587eaa90df32 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:24:35.079389  6074 raft_consensus.cc:740] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.093144  6120 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:35.100040  6074 consensus_queue.cc:260] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.099892  5862 tablet_service.cc:1511] Processing CreateTablet for tablet 0006f949032c4034b4f4872ee332386e (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:24:35.103868  6074 ts_tablet_manager.cc:1434] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.124s	user 0.012s	sys 0.000s
I20260709 12:24:35.107127  5782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e42697dffe5a495ea1ba4a111df5ba39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.108273  6009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba269f0998844ab5b461587eaa90df32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.108844  5862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0006f949032c4034b4f4872ee332386e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.110391  6074 tablet_bootstrap.cc:492] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:35.118728  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:35.119522  5873 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.119880  5798 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:35.121342  5915 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:35.122468  6136 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:35.123304  6136 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:35.124266  6136 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:35.127030  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:35.137190  5872 raft_consensus.cc:2393] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 1: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:24:35.142357  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:35.165712  5937 tablet_service.cc:1511] Processing CreateTablet for tablet 0006f949032c4034b4f4872ee332386e (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:24:35.167433  5856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e42697dffe5a495ea1ba4a111df5ba39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.168349  5937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0006f949032c4034b4f4872ee332386e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.130988  6088 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 12:24:35.174059  5861 tablet_service.cc:1511] Processing CreateTablet for tablet 59cab2d7e5234efd9a97b8377f1919f7 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:24:35.155112  6074 tablet_bootstrap.cc:654] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.191311  5861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59cab2d7e5234efd9a97b8377f1919f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.192929  5785 tablet_service.cc:1511] Processing CreateTablet for tablet 59cab2d7e5234efd9a97b8377f1919f7 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:24:35.194531  5785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59cab2d7e5234efd9a97b8377f1919f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.200381  6069 tablet_bootstrap.cc:492] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:35.201010  6069 ts_tablet_manager.cc:1403] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.438s	user 0.014s	sys 0.006s
I20260709 12:24:35.202253  5872 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.192497  5695 catalog_manager.cc:5697] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 1 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:35.237875  5933 tablet_service.cc:1511] Processing CreateTablet for tablet 59cab2d7e5234efd9a97b8377f1919f7 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:24:35.239319  5933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59cab2d7e5234efd9a97b8377f1919f7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:35.227079  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 99433.4us
I20260709 12:24:35.251076  5873 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:35.253157  5786 tablet_service.cc:1511] Processing CreateTablet for tablet ba269f0998844ab5b461587eaa90df32 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:24:35.261391  5786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba269f0998844ab5b461587eaa90df32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.223018  6120 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260709 12:24:35.269309  6006 tablet_service.cc:1511] Processing CreateTablet for tablet 595c6563cdd74f47bda68ead57106a8b (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:24:35.270795  6006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 595c6563cdd74f47bda68ead57106a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.273068  5763 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:35.258592  6070 tablet_bootstrap.cc:492] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:35.253809  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:35.274518  6070 ts_tablet_manager.cc:1403] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.371s	user 0.016s	sys 0.003s
I20260709 12:24:35.289642  6131 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:35.291831  6070 raft_consensus.cc:359] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:35.292870  6070 raft_consensus.cc:385] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.293182  6070 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.231802  6069 raft_consensus.cc:359] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.298859  6069 raft_consensus.cc:385] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.299377  6069 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.306257  6071 tablet_bootstrap.cc:654] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.317198  5697 catalog_manager.cc:5697] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 1 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:35.337420  6074 tablet_bootstrap.cc:492] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:35.367822  6136 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.370276  6136 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:35.372764  6136 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:35.374900  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:35.324007  6069 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.365101  6070 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:35.377275  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:35.377475  6129 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.378146  6129 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:24:35.378652  5990 consensus_peers.cc:597] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:35.379737  5915 leader_election.cc:343] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.381075  6013 tablet_service.cc:1511] Processing CreateTablet for tablet 99b9212afb8b459bb96908f8259bb8c0 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:24:35.381822  5864 tablet_service.cc:1511] Processing CreateTablet for tablet 595c6563cdd74f47bda68ead57106a8b (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
W20260709 12:24:35.382226  5913 leader_election.cc:343] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.382544  6013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99b9212afb8b459bb96908f8259bb8c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.382777  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:35.383291  5864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 595c6563cdd74f47bda68ead57106a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.384259  6136 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:35.386756  5935 tablet_service.cc:1511] Processing CreateTablet for tablet 99b9212afb8b459bb96908f8259bb8c0 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:24:35.388166  5935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99b9212afb8b459bb96908f8259bb8c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.360008  6007 tablet_service.cc:1511] Processing CreateTablet for tablet 0006f949032c4034b4f4872ee332386e (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:24:35.389397  6007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0006f949032c4034b4f4872ee332386e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.393095  6129 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:35.365705  6074 ts_tablet_manager.cc:1403] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.261s	user 0.011s	sys 0.026s
I20260709 12:24:35.406488  6146 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.407214  6146 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.411134  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:35.412014  5873 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:24:35.413730  6146 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:35.370316  5785 tablet_service.cc:1511] Processing CreateTablet for tablet 595c6563cdd74f47bda68ead57106a8b (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:24:35.458981  5785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 595c6563cdd74f47bda68ead57106a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.420750  6074 raft_consensus.cc:359] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.463052  6074 raft_consensus.cc:385] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.463527  6074 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.424075  5763 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:35.427776  6069 ts_tablet_manager.cc:1434] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.226s	user 0.005s	sys 0.004s
I20260709 12:24:35.430441  5783 tablet_service.cc:1511] Processing CreateTablet for tablet 99b9212afb8b459bb96908f8259bb8c0 (DEFAULT_TABLE table=test-workload [id=aca28e3b970a4c5cae0071b6cf1cc837]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:24:35.432232  6070 ts_tablet_manager.cc:1434] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.157s	user 0.007s	sys 0.001s
W20260709 12:24:35.405922  5990 leader_election.cc:343] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:35.471351  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 66651.2us
I20260709 12:24:35.472991  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:24:35.468546  5763 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 63032.4us
I20260709 12:24:35.469969  6129 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:35.476774  6129 raft_consensus.cc:697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:35.478086  6129 consensus_queue.cc:237] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.471553  6070 tablet_bootstrap.cc:492] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:35.469344  6069 tablet_bootstrap.cc:492] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:35.485085  6071 tablet_bootstrap.cc:492] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:35.485862  6071 ts_tablet_manager.cc:1403] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.428s	user 0.014s	sys 0.007s
I20260709 12:24:35.464847  6074 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.495921  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:35.497000  5872 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 1: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:24:35.499790  5783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99b9212afb8b459bb96908f8259bb8c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:35.501365  6074 ts_tablet_manager.cc:1434] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.100s	user 0.006s	sys 0.003s
I20260709 12:24:35.502532  6071 raft_consensus.cc:359] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.504913  6071 raft_consensus.cc:385] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.505412  6071 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.506487  6071 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.508540  5765 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:35.510128  6130 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.517289  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:35.520015  5990 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:35.521576  6146 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.502755  6074 tablet_bootstrap.cc:492] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:35.522701  6070 tablet_bootstrap.cc:654] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.525771  6069 tablet_bootstrap.cc:654] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.534924  6130 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:35.548648  6074 tablet_bootstrap.cc:654] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.536424  5697 catalog_manager.cc:5697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:35.563001  6155 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.555883  6069 tablet_bootstrap.cc:492] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:35.556381  6071 ts_tablet_manager.cc:1434] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.002s
I20260709 12:24:35.555313  6146 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:35.565979  6069 ts_tablet_manager.cc:1403] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.097s	user 0.016s	sys 0.000s
I20260709 12:24:35.565382  6155 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.570062  6069 raft_consensus.cc:359] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:35.570101  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:35.570919  6069 raft_consensus.cc:385] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.571168  6069 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.571316  5948 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:35.571875  6069 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:35.574512  6071 tablet_bootstrap.cc:492] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:35.577152  5838 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:35.578689  6117 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:35.579101  6117 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.579433  6117 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.581176  6069 ts_tablet_manager.cc:1434] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:24:35.582561  6069 tablet_bootstrap.cc:492] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:35.586910  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:35.587756  6155 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:35.587872  6023 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:35.594496  6069 tablet_bootstrap.cc:654] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.598214  6071 tablet_bootstrap.cc:654] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.598718  6074 tablet_bootstrap.cc:492] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:35.598726  6136 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:35.599560  6074 ts_tablet_manager.cc:1403] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.097s	user 0.012s	sys 0.003s
I20260709 12:24:35.593093  6117 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.603173  6074 raft_consensus.cc:359] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.605654  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:35.607234  6023 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.609743  6071 tablet_bootstrap.cc:492] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:35.610242  6071 ts_tablet_manager.cc:1403] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.002s
I20260709 12:24:35.603945  6117 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:35.611901  6074 raft_consensus.cc:385] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.612766  6074 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.613282  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:35.614018  5948 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.614909  6074 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.617995  6074 ts_tablet_manager.cc:1434] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.001s
I20260709 12:24:35.619400  6023 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:35.620811  6146 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:35.622830  6071 raft_consensus.cc:359] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:35.623313  6074 tablet_bootstrap.cc:492] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:35.623621  6071 raft_consensus.cc:385] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.623968  6071 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.620826  5840 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:35.628105  6117 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:35.628625  6117 raft_consensus.cc:697] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:35.624788  6071 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:35.632789  6117 consensus_queue.cc:237] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.637094  6071 ts_tablet_manager.cc:1434] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:24:35.637861  6069 tablet_bootstrap.cc:492] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:35.638281  6071 tablet_bootstrap.cc:492] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:35.638409  6069 ts_tablet_manager.cc:1403] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.056s	user 0.010s	sys 0.005s
I20260709 12:24:35.644471  6069 raft_consensus.cc:359] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:35.645246  6069 raft_consensus.cc:385] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.645498  6069 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.647617  6070 tablet_bootstrap.cc:492] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:35.648343  6070 ts_tablet_manager.cc:1403] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.177s	user 0.016s	sys 0.008s
I20260709 12:24:35.650583  5948 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:35.653702  6069 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:35.658537  6070 raft_consensus.cc:359] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:35.659677  6070 raft_consensus.cc:385] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.660061  6070 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.662166  6074 tablet_bootstrap.cc:654] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.667088  6069 ts_tablet_manager.cc:1434] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:24:35.668260  6069 tablet_bootstrap.cc:492] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:35.671308  6155 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.671887  6155 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:35.676884  6155 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:35.681823  6155 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.682559  6155 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:35.678843  6071 tablet_bootstrap.cc:654] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.679734  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:35.680629  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:35.685245  5840 leader_election.cc:343] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.686911  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:35.688424  5840 leader_election.cc:343] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.688459  6079 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:35.688926  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:24:35.690299  5840 leader_election.cc:343] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.692718  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:35.693586  5948 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:35.697193  6117 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:35.698942  5838 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:35.700161  6117 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:35.696740  6155 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:35.710027  6146 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:35.713709  5695 catalog_manager.cc:5697] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:35.702721  6117 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.717872  6117 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.674337  6070 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:35.713466  6069 tablet_bootstrap.cc:654] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.730206  6070 ts_tablet_manager.cc:1434] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.081s	user 0.007s	sys 0.006s
I20260709 12:24:35.736608  6117 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:35.758317  6070 tablet_bootstrap.cc:492] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:35.762136  5797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:35.764570  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:35.765441  5948 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:35.770342  5840 leader_election.cc:343] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.759310  6117 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:35.775826  6071 tablet_bootstrap.cc:492] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:35.776511  6071 ts_tablet_manager.cc:1403] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.138s	user 0.016s	sys 0.013s
I20260709 12:24:35.779835  6070 tablet_bootstrap.cc:654] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.779799  6071 raft_consensus.cc:359] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.780926  6071 raft_consensus.cc:385] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.781222  6071 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.782092  6071 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.784677  6071 ts_tablet_manager.cc:1434] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:24:35.785825  6071 tablet_bootstrap.cc:492] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:35.792014  6074 tablet_bootstrap.cc:492] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:35.792619  6074 ts_tablet_manager.cc:1403] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.170s	user 0.021s	sys 0.021s
I20260709 12:24:35.794727  5948 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:35.810527  5838 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:35.811509  6117 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:35.810520  6069 tablet_bootstrap.cc:492] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:35.812106  6117 raft_consensus.cc:697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:35.799738  6074 raft_consensus.cc:359] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:35.812904  6074 raft_consensus.cc:385] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.813215  6069 ts_tablet_manager.cc:1403] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.145s	user 0.007s	sys 0.009s
I20260709 12:24:35.813090  6117 consensus_queue.cc:237] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:35.816828  6069 raft_consensus.cc:359] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.813323  6074 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.818593  6074 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:35.821758  6074 ts_tablet_manager.cc:1434] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.002s
I20260709 12:24:35.822139  6070 tablet_bootstrap.cc:492] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:35.822767  6070 ts_tablet_manager.cc:1403] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.065s	user 0.014s	sys 0.000s
I20260709 12:24:35.822989  6074 tablet_bootstrap.cc:492] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:35.803313  6071 tablet_bootstrap.cc:654] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.822144  6069 raft_consensus.cc:385] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.825763  6069 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.825917  6070 raft_consensus.cc:359] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:35.826784  6070 raft_consensus.cc:385] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.827262  6070 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.828270  6070 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:35.826545  6069 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:35.831665  6069 ts_tablet_manager.cc:1434] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.001s
I20260709 12:24:35.832801  6069 tablet_bootstrap.cc:492] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:35.840000  6074 tablet_bootstrap.cc:654] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.844260  6070 ts_tablet_manager.cc:1434] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.004s
I20260709 12:24:35.846351  6070 tablet_bootstrap.cc:492] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:35.854153  6070 tablet_bootstrap.cc:654] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.846304  6069 tablet_bootstrap.cc:654] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.865061  6074 tablet_bootstrap.cc:492] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:35.865515  6069 tablet_bootstrap.cc:492] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:35.865723  6074 ts_tablet_manager.cc:1403] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.005s
I20260709 12:24:35.866144  6069 ts_tablet_manager.cc:1403] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.000s
I20260709 12:24:35.867352  6131 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:35.869958  6074 raft_consensus.cc:359] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:35.872552  6074 raft_consensus.cc:385] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.872936  6074 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.874234  6074 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:35.877735  6070 tablet_bootstrap.cc:492] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:35.878510  6070 ts_tablet_manager.cc:1403] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.002s
I20260709 12:24:35.882068  6074 ts_tablet_manager.cc:1434] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:24:35.883391  6074 tablet_bootstrap.cc:492] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:35.883442  6071 tablet_bootstrap.cc:492] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:35.884440  6071 ts_tablet_manager.cc:1403] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.099s	user 0.030s	sys 0.021s
I20260709 12:24:35.887734  6070 raft_consensus.cc:359] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:35.887861  6071 raft_consensus.cc:359] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:35.888161  6146 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.888567  6070 raft_consensus.cc:385] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.888628  6071 raft_consensus.cc:385] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.889076  6070 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.889130  6071 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.890273  6069 raft_consensus.cc:359] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:35.890276  6071 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:35.891201  6069 raft_consensus.cc:385] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.891494  6069 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.893384  6071 ts_tablet_manager.cc:1434] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.003s
I20260709 12:24:35.894711  6071 tablet_bootstrap.cc:492] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:35.894711  6069 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:35.888859  6146 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:35.898687  6146 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:35.899044  6069 ts_tablet_manager.cc:1434] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
I20260709 12:24:35.900153  6069 tablet_bootstrap.cc:492] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:35.900703  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:35.905099  6071 tablet_bootstrap.cc:654] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:35.906381  5990 leader_election.cc:343] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.907646  6074 tablet_bootstrap.cc:654] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.907639  6070 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:35.909200  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:35.910018  5948 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:35.905215  5696 catalog_manager.cc:5697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:35.913697  6069 tablet_bootstrap.cc:654] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.917536  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:35.919414  6070 ts_tablet_manager.cc:1434] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.041s	user 0.009s	sys 0.000s
I20260709 12:24:35.920408  6070 tablet_bootstrap.cc:492] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:35.923633  6079 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.924263  6079 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:35.926776  6079 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:35.928045  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:35.928833  5874 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:35.930316  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:35.930248  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:35.931326  6088 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.932224  6079 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:35.931933  6088 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:24:35.932567  5915 leader_election.cc:343] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.922160  6146 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:35.934453  6146 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.934985  6146 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.949924  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:35.954901  5990 leader_election.cc:343] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.955883  6088 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:35.952162  6079 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.956785  6079 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.957433  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:24:35.959260  5987 leader_election.cc:343] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.959780  5987 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:35.961052  6088 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:35.961732  6129 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:35.972882  6070 tablet_bootstrap.cc:654] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:35.986841  5745 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:35.993196  6146 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.024292  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:36.024952  5948 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.024791  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:24:36.026635  5990 leader_election.cc:343] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.027779  6146 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:36.004746  6079 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:36.011894  5799 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.013660  6074 tablet_bootstrap.cc:492] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:36.028980  6074 ts_tablet_manager.cc:1403] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.146s	user 0.013s	sys 0.023s
I20260709 12:24:36.031059  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:36.031860  6074 raft_consensus.cc:359] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.032660  6074 raft_consensus.cc:385] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.032910  6074 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.033690  6074 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.035305  6074 ts_tablet_manager.cc:1434] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:24:36.013763  6071 tablet_bootstrap.cc:492] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:36.035868  6136 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.036531  6136 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.011879  6069 tablet_bootstrap.cc:492] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:36.020303  6070 tablet_bootstrap.cc:492] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:36.043923  6074 tablet_bootstrap.cc:492] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:36.044437  6071 ts_tablet_manager.cc:1403] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.150s	user 0.022s	sys 0.030s
I20260709 12:24:36.046977  6088 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.047617  6117 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:36.049376  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:36.050278  5872 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:36.047645  6088 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.051890  5913 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:36.053058  6164 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:36.053464  6164 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.053818  6164 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.054363  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:36.054883  6136 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:36.055366  5872 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:36.055493  6071 raft_consensus.cc:359] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.056368  6071 raft_consensus.cc:385] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.056628  6071 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.056699  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:36.057334  6071 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.057785  6121 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:36.058254  6121 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.058600  6121 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.060317  6088 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:36.062966  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:24:36.063886  5915 leader_election.cc:343] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.065837  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:36.069119  6079 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:36.069815  6071 ts_tablet_manager.cc:1434] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.003s
I20260709 12:24:36.070868  6071 tablet_bootstrap.cc:492] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:36.072314  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:36.073019  5872 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.078576  6164 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.079795  6069 ts_tablet_manager.cc:1403] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.180s	user 0.029s	sys 0.017s
I20260709 12:24:36.080047  6071 tablet_bootstrap.cc:654] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.080897  6070 ts_tablet_manager.cc:1403] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.161s	user 0.018s	sys 0.000s
I20260709 12:24:36.080318  5947 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:24:36.085074  5948 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:36.088291  6164 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:36.089231  6074 tablet_bootstrap.cc:654] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.089516  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:36.089600  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:36.090194  5874 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.091764  6069 raft_consensus.cc:359] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.092545  6069 raft_consensus.cc:385] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.092911  6069 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.094130  6069 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.099265  6069 ts_tablet_manager.cc:1434] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:24:36.093091  6121 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.105113  6121 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:36.105926  6023 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 1: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:24:36.093156  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:36.124406  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:36.129660  6071 tablet_bootstrap.cc:492] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:36.128459  6121 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:36.130614  6071 ts_tablet_manager.cc:1403] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.060s	user 0.017s	sys 0.005s
I20260709 12:24:36.130898  6121 raft_consensus.cc:697] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:36.131893  6121 consensus_queue.cc:237] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.135076  6071 raft_consensus.cc:359] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:36.136307  6071 raft_consensus.cc:385] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.136716  6071 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.138563  6071 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:36.136637  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:36.142436  6071 ts_tablet_manager.cc:1434] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 12:24:36.144294  6071 tablet_bootstrap.cc:492] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:36.142166  5947 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:24:36.148327  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:36.149958  6146 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:36.152787  5874 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:36.153702  5872 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:36.156278  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:36.157536  5913 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:36.158363  6079 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:36.158706  6164 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:36.159071  6079 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:36.159602  6164 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:36.160079  6079 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:36.160655  6074 tablet_bootstrap.cc:492] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:36.160552  6164 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.161339  6074 ts_tablet_manager.cc:1403] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.118s	user 0.010s	sys 0.018s
I20260709 12:24:36.164173  6074 raft_consensus.cc:359] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.165060  6074 raft_consensus.cc:385] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.165460  6074 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.166613  6074 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.169250  6074 ts_tablet_manager.cc:1434] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:24:36.170367  6074 tablet_bootstrap.cc:492] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:36.170734  6071 tablet_bootstrap.cc:654] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.173959  6069 tablet_bootstrap.cc:492] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:36.178169  6070 raft_consensus.cc:359] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.179771  6070 raft_consensus.cc:385] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.180084  6070 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.180812  6070 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.182869  6074 tablet_bootstrap.cc:654] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.186003  6130 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.185695  5697 catalog_manager.cc:5697] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 1 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:36.187291  6130 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.191774  6130 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:36.193607  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:36.194620  5949 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 1: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:24:36.197428  6070 ts_tablet_manager.cc:1434] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.103s	user 0.009s	sys 0.000s
I20260709 12:24:36.200484  6070 tablet_bootstrap.cc:492] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:36.201396  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:36.206983  6071 tablet_bootstrap.cc:492] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:36.207605  6071 ts_tablet_manager.cc:1403] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.064s	user 0.018s	sys 0.006s
I20260709 12:24:36.213527  5765 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:36.213423  6071 raft_consensus.cc:359] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.214232  6071 raft_consensus.cc:385] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.214514  6071 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.217248  6166 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:36.215374  6071 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.220121  6071 ts_tablet_manager.cc:1434] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:24:36.221124  6071 tablet_bootstrap.cc:492] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:36.224697  6129 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.225303  6129 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.227463  6129 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:36.222999  6130 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.235478  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:36.236320  5874 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:36.236946  5695 catalog_manager.cc:5697] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 1 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:36.238919  5695 catalog_manager.cc:5697] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 1 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:36.240290  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:36.242969  6023 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:36.244122  5763 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:36.246029  6070 tablet_bootstrap.cc:654] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.249969  6129 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:36.250475  6129 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.250799  6129 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.262599  6129 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.265300  6129 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:36.275811  6069 tablet_bootstrap.cc:654] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.276386  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:36.277154  5874 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.277434  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:36.278158  6023 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.280268  6074 tablet_bootstrap.cc:492] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:36.280876  6071 tablet_bootstrap.cc:654] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.281039  6074 ts_tablet_manager.cc:1403] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.111s	user 0.013s	sys 0.020s
I20260709 12:24:36.294488  6074 raft_consensus.cc:359] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.295537  6074 raft_consensus.cc:385] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.295936  6074 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.298026  6074 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.299211  6023 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:36.301154  6129 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.301877  6129 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:36.303339  6074 ts_tablet_manager.cc:1434] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.002s
I20260709 12:24:36.303588  6130 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:36.304253  6129 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:36.305461  5874 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:36.306516  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:36.307037  5763 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:36.307467  5949 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 1: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:24:36.307803  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:36.314635  5763 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:36.315724  6129 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:36.316699  6131 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.322139  6074 tablet_bootstrap.cc:492] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:36.331152  6074 tablet_bootstrap.cc:654] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.321199  6129 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:36.343310  6121 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.344192  6121 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.344522  6129 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.348691  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:24:36.352849  5987 leader_election.cc:343] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.354878  6069 tablet_bootstrap.cc:492] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:36.355490  6069 ts_tablet_manager.cc:1403] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.182s	user 0.019s	sys 0.040s
I20260709 12:24:36.356755  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:36.357435  5949 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:36.358445  6069 raft_consensus.cc:359] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.359326  6069 raft_consensus.cc:385] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.359658  6069 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.360021  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:36.360412  6069 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.360926  6146 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:36.361296  6146 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.361609  6146 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.365104  6069 ts_tablet_manager.cc:1434] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:24:36.359493  6121 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:36.368510  6074 tablet_bootstrap.cc:492] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:36.369201  6074 ts_tablet_manager.cc:1403] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.008s
I20260709 12:24:36.372910  6074 raft_consensus.cc:359] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.373602  6074 raft_consensus.cc:385] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.373940  6074 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.374578  6074 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.376492  6074 ts_tablet_manager.cc:1434] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.004s
I20260709 12:24:36.380815  6131 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:36.377556  6074 tablet_bootstrap.cc:492] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:36.383229  6069 tablet_bootstrap.cc:492] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:36.380959  6146 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.387099  6071 tablet_bootstrap.cc:492] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:36.396303  6071 ts_tablet_manager.cc:1403] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.175s	user 0.021s	sys 0.027s
I20260709 12:24:36.392849  6069 tablet_bootstrap.cc:654] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.397794  6146 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:36.398933  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:36.399641  5949 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.403374  6069 tablet_bootstrap.cc:492] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:36.403343  6071 raft_consensus.cc:359] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.395485  6070 tablet_bootstrap.cc:492] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:36.404393  6071 raft_consensus.cc:385] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.404747  6070 ts_tablet_manager.cc:1403] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.204s	user 0.020s	sys 0.022s
I20260709 12:24:36.404901  6071 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.403349  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:36.406085  6071 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:36.407682  5987 leader_election.cc:343] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.407944  6070 raft_consensus.cc:359] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:36.408851  6070 raft_consensus.cc:385] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.406051  6166 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:24:36.409166  6070 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.410013  6071 ts_tablet_manager.cc:1434] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:24:36.409885  6070 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:36.412199  6071 tablet_bootstrap.cc:492] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:36.412334  6070 ts_tablet_manager.cc:1434] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:24:36.393595  5696 catalog_manager.cc:5697] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:36.413463  6070 tablet_bootstrap.cc:492] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:36.419620  6071 tablet_bootstrap.cc:654] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.420035  6070 tablet_bootstrap.cc:654] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.404045  6069 ts_tablet_manager.cc:1403] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.003s
I20260709 12:24:36.426263  6069 raft_consensus.cc:359] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.426981  6069 raft_consensus.cc:385] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.427230  6069 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.428081  6069 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.430200  6069 ts_tablet_manager.cc:1434] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:24:36.431174  6069 tablet_bootstrap.cc:492] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:36.431893  6155 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:36.437139  6069 tablet_bootstrap.cc:654] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.443378  6071 tablet_bootstrap.cc:492] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:36.445022  6071 ts_tablet_manager.cc:1403] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.011s
I20260709 12:24:36.447317  6070 tablet_bootstrap.cc:492] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:36.448042  6070 ts_tablet_manager.cc:1403] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.035s	user 0.006s	sys 0.006s
I20260709 12:24:36.449811  6071 raft_consensus.cc:359] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.450572  6071 raft_consensus.cc:385] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.450824  6071 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.451015  6070 raft_consensus.cc:359] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.451741  6070 raft_consensus.cc:385] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.451594  6071 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.454265  6071 ts_tablet_manager.cc:1434] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:24:36.455457  6071 tablet_bootstrap.cc:492] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:36.452021  6070 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.460152  6155 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.460830  6155 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.463093  6155 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:36.458846  6070 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.464174  6146 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.465615  6071 tablet_bootstrap.cc:654] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.466205  6129 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.466885  6129 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.466279  6121 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.471287  6129 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:36.471166  6121 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.472545  6070 ts_tablet_manager.cc:1434] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 12:24:36.474098  6070 tablet_bootstrap.cc:492] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:36.475462  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:36.476215  6023 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:36.465412  6146 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.479022  6146 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:36.480775  6070 tablet_bootstrap.cc:654] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.481127  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:36.482904  6164 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.483466  6164 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.485567  5765 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:24:36.491822  6129 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:36.492388  6129 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.492823  6129 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.494879  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:36.495939  5947 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:36.496235  6155 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.496726  6155 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.497386  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:36.498883  6155 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:36.500573  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:36.471191  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:36.509519  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:36.512670  6164 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:36.513000  5949 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:36.515439  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:36.486214  5990 leader_election.cc:343] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.504070  6146 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:36.501240  5947 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:36.490137  6121 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:36.531704  6069 tablet_bootstrap.cc:492] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:36.532523  6069 ts_tablet_manager.cc:1403] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.102s	user 0.011s	sys 0.000s
I20260709 12:24:36.526260  5874 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:36.530615  6023 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:36.536425  6146 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.536834  6146 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:36.535799  5915 leader_election.cc:343] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.539482  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:36.540666  6164 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:36.541080  6164 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.541417  6164 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:36.530037  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 43881.1us
I20260709 12:24:36.545518  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:36.546749  6155 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:36.547158  6155 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.547498  6155 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.549403  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:36.550125  6023 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:36.550226  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:36.550879  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:24:36.552173  5990 leader_election.cc:343] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.558082  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:36.559599  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:36.552904  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:36.556146  6069 raft_consensus.cc:359] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:36.562924  6069 raft_consensus.cc:385] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.552795  6120 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:36.563820  6120 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:36.563292  6069 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
W20260709 12:24:36.565160  6206 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:24:36.564874  6120 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.569422  6186 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:36.538074  6129 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.571991  5799 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 1: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:24:36.575266  6074 tablet_bootstrap.cc:654] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.575783  6129 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:36.576916  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:36.577760  6023 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.580358  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:36.581091  5872 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.558024  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:24:36.584923  5987 leader_election.cc:343] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.585747  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:36.587844  6121 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:36.589421  6186 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:36.591734  5840 leader_election.cc:343] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.602231  6164 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.594043  6069 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:36.603020  6070 tablet_bootstrap.cc:492] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:36.607535  6070 ts_tablet_manager.cc:1403] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.134s	user 0.027s	sys 0.021s
I20260709 12:24:36.608367  6071 tablet_bootstrap.cc:492] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:36.608989  6071 ts_tablet_manager.cc:1403] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.154s	user 0.061s	sys 0.051s
I20260709 12:24:36.609126  6069 ts_tablet_manager.cc:1434] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.076s	user 0.004s	sys 0.005s
I20260709 12:24:36.611112  6155 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.612124  6071 raft_consensus.cc:359] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.622219  6071 raft_consensus.cc:385] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.622125  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:36.622637  6071 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.612396  6146 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.616452  6186 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.623641  6186 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:36.623653  6071 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:24:36.624783  5915 leader_election.cc:343] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.626291  6071 ts_tablet_manager.cc:1434] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.005s
I20260709 12:24:36.619236  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:36.627714  6071 tablet_bootstrap.cc:492] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:36.626184  6186 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:36.629065  6069 tablet_bootstrap.cc:492] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:36.620857  6023 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:36.627874  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:36.633790  5871 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:36.636649  5765 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:24:36.638563  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:36.616808  5872 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:36.639734  6071 tablet_bootstrap.cc:654] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.640342  6129 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.640626  6130 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:36.631567  6070 raft_consensus.cc:359] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:24:36.640601  5915 leader_election.cc:343] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.643002  6186 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:36.644675  6024 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 1: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:24:36.632288  6164 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:36.647281  6155 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:36.642575  6070 raft_consensus.cc:385] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.643144  6130 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:36.648492  6130 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.652061  6070 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.652928  6070 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.642428  6166 raft_consensus.cc:2764] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:24:36.659893  6070 ts_tablet_manager.cc:1434] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.000s
I20260709 12:24:36.665103  6074 tablet_bootstrap.cc:492] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:36.665895  6070 tablet_bootstrap.cc:492] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:36.666206  6074 ts_tablet_manager.cc:1403] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.289s	user 0.027s	sys 0.011s
I20260709 12:24:36.641009  6129 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:36.659178  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:36.675017  6069 tablet_bootstrap.cc:654] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.666910  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:36.673952  6146 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:36.667081  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:36.682420  6164 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:36.682849  6164 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.683164  6164 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.683601  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:36.684552  5947 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:24:36.684662  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:24:36.690977  5840 leader_election.cc:343] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.691043  6079 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.678555  5949 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:36.698349  5990 leader_election.cc:343] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.699178  5990 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:36.700416  6146 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:36.701472  6079 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.703780  6079 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:36.707489  6079 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.709360  6079 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.711635  6079 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:36.712703  6146 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:36.677179  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:36.715662  6079 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:36.718683  5799 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.718799  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:36.721585  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:36.722172  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:36.724249  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:36.726199  6146 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:36.727324  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:36.729244  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:36.734767  6074 raft_consensus.cc:359] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.735591  6074 raft_consensus.cc:385] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.735882  6074 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.736605  6074 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.744199  6074 ts_tablet_manager.cc:1434] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.078s	user 0.007s	sys 0.000s
I20260709 12:24:36.694196  6129 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:24:36.746836  5915 leader_election.cc:343] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:36.747682  5915 leader_election.cc:343] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:36.748347  5915 leader_election.cc:343] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.748770  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:36.749583  6079 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:24:36.754287  5913 leader_election.cc:343] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.770737  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:36.770802  6079 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:36.755363  5763 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:36.763780  6071 tablet_bootstrap.cc:492] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:36.760540  6131 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:24:36.773427  6079 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:36.779433  6071 ts_tablet_manager.cc:1403] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.152s	user 0.018s	sys 0.021s
I20260709 12:24:36.777878  5697 catalog_manager.cc:5697] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 1 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:36.805078  6164 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:36.824864  6164 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:36.811535  5949 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:36.815640  6155 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.803133  6070 tablet_bootstrap.cc:654] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.821393  6071 raft_consensus.cc:359] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.826972  5797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:36.827108  6155 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:24:36.829030  5915 leader_election.cc:343] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.827167  6071 raft_consensus.cc:385] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.832958  6071 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.829569  6155 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:36.832319  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:36.834954  5948 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 1: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:24:36.831483  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:36.837738  6155 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:36.838498  6155 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:36.839752  6155 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:36.830925  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:36.843894  5838 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:36.845350  6117 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.860755  5696 catalog_manager.cc:5697] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:36.864391  6069 tablet_bootstrap.cc:492] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:36.865051  6069 ts_tablet_manager.cc:1403] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.236s	user 0.014s	sys 0.004s
I20260709 12:24:36.835573  6071 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.822110  6074 tablet_bootstrap.cc:492] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:36.908114  6071 ts_tablet_manager.cc:1434] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.122s	user 0.005s	sys 0.010s
I20260709 12:24:36.910132  6071 tablet_bootstrap.cc:492] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
W20260709 12:24:36.878988  5763 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 123715us
I20260709 12:24:36.891122  6131 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:36.916442  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:36.971158  5872 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.887666  6117 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:36.920213  6069 raft_consensus.cc:359] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.972462  6069 raft_consensus.cc:385] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.972831  6069 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.973273  6222 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.973652  6069 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.973970  6222 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:36.976172  6069 ts_tablet_manager.cc:1434] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.111s	user 0.008s	sys 0.008s
I20260709 12:24:36.926059  6074 tablet_bootstrap.cc:654] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.947074  6070 tablet_bootstrap.cc:492] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:36.977779  6070 ts_tablet_manager.cc:1403] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.312s	user 0.020s	sys 0.000s
I20260709 12:24:36.980971  6070 raft_consensus.cc:359] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.981648  6069 tablet_bootstrap.cc:492] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:36.990886  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:36.992069  6022 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:36.994629  5697 catalog_manager.cc:5697] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:36.996567  5872 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:36.983796  6222 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:36.987030  6131 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:36.988940  6070 raft_consensus.cc:385] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.998484  6070 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.999389  6070 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:36.989475  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:24:37.001891  5763 leader_election.cc:343] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:37.004403  6070 ts_tablet_manager.cc:1434] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.003s
I20260709 12:24:37.006651  6070 tablet_bootstrap.cc:492] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:37.014441  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:37.015625  6070 tablet_bootstrap.cc:654] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.015637  6221 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:37.024178  6071 tablet_bootstrap.cc:654] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.026667  6221 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:37.027268  6070 tablet_bootstrap.cc:492] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:37.027957  6070 ts_tablet_manager.cc:1403] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.004s
I20260709 12:24:37.027815  6221 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.034325  5765 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:37.035451  6222 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:37.035920  6222 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.036324  6222 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.037421  6074 tablet_bootstrap.cc:492] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:37.038214  6074 ts_tablet_manager.cc:1403] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.216s	user 0.013s	sys 0.007s
I20260709 12:24:37.034684  6069 tablet_bootstrap.cc:654] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.060103  6222 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.066326  6131 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:37.058679  6070 raft_consensus.cc:359] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.067611  6070 raft_consensus.cc:385] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.067483  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:37.068063  6070 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.068320  6022 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.068881  6070 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.071553  6222 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:37.077037  6022 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:37.079591  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:37.081755  6120 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:37.082284  5763 leader_election.cc:343] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:37.086669  5765 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:37.087925  6131 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:37.089092  6131 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:37.089946  6069 tablet_bootstrap.cc:492] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:37.090692  6069 ts_tablet_manager.cc:1403] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.114s	user 0.014s	sys 0.000s
I20260709 12:24:37.090544  6131 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.070662  6070 ts_tablet_manager.cc:1434] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.042s	user 0.004s	sys 0.006s
I20260709 12:24:37.099397  6070 tablet_bootstrap.cc:492] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:37.103731  6074 raft_consensus.cc:359] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.106120  6069 raft_consensus.cc:359] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.111730  6069 raft_consensus.cc:385] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.112084  6069 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.112901  6070 tablet_bootstrap.cc:654] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.112869  6069 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.114691  6117 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.115463  6069 ts_tablet_manager.cc:1434] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:24:37.115185  6117 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.116662  6069 tablet_bootstrap.cc:492] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:37.110318  6071 tablet_bootstrap.cc:492] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:37.117600  6117 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:37.118021  6071 ts_tablet_manager.cc:1403] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.208s	user 0.030s	sys 0.043s
I20260709 12:24:37.119536  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:37.120208  5948 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:37.121098  6071 raft_consensus.cc:359] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.121816  6071 raft_consensus.cc:385] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.122067  6071 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.122968  6071 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.124296  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:37.125277  6071 ts_tablet_manager.cc:1434] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:24:37.126089  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:37.127707  6146 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:37.128667  5840 leader_election.cc:343] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:37.131814  6117 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:37.133220  6117 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.133718  6117 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.134486  6071 tablet_bootstrap.cc:492] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
W20260709 12:24:37.142794  5745 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:37.143833  5693 catalog_manager.cc:5697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 1 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:37.126107  6074 raft_consensus.cc:385] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.149787  6074 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.151131  6074 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.153522  6222 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:37.154753  6074 ts_tablet_manager.cc:1434] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.116s	user 0.009s	sys 0.002s
I20260709 12:24:37.156292  6074 tablet_bootstrap.cc:492] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:37.146278  6071 tablet_bootstrap.cc:654] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.147817  6069 tablet_bootstrap.cc:654] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.171898  5695 catalog_manager.cc:5697] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:37.175843  6074 tablet_bootstrap.cc:654] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.174623  6117 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.214614  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:37.216082  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:37.214339  6117 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:37.216917  5949 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:37.229871  5840 leader_election.cc:343] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:37.236223  5949 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:37.241041  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:37.242278  6117 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:37.243067  6117 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:37.244174  6117 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.246840  6070 tablet_bootstrap.cc:492] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:37.247452  6070 ts_tablet_manager.cc:1403] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.148s	user 0.044s	sys 0.060s
I20260709 12:24:37.250542  6070 raft_consensus.cc:359] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.251444  6070 raft_consensus.cc:385] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.251776  6070 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.252661  6070 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.255580  6070 ts_tablet_manager.cc:1434] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 12:24:37.256835  6070 tablet_bootstrap.cc:492] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:37.260524  6071 tablet_bootstrap.cc:492] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:37.261214  6071 ts_tablet_manager.cc:1403] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.127s	user 0.013s	sys 0.016s
I20260709 12:24:37.262413  6074 tablet_bootstrap.cc:492] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:37.262885  6070 tablet_bootstrap.cc:654] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.263018  6074 ts_tablet_manager.cc:1403] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.107s	user 0.020s	sys 0.018s
I20260709 12:24:37.264628  6071 raft_consensus.cc:359] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.265446  6071 raft_consensus.cc:385] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.265910  6071 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.266875  6071 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.260581  6069 tablet_bootstrap.cc:492] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:37.268522  6069 ts_tablet_manager.cc:1403] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.152s	user 0.020s	sys 0.019s
I20260709 12:24:37.269109  6071 ts_tablet_manager.cc:1434] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:24:37.266677  6074 raft_consensus.cc:359] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.269860  6074 raft_consensus.cc:385] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.274192  6074 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.275336  6074 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.278813  6074 ts_tablet_manager.cc:1434] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 12:24:37.270207  6071 tablet_bootstrap.cc:492] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:37.287411  6071 tablet_bootstrap.cc:654] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.273171  6069 raft_consensus.cc:359] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.289564  6069 raft_consensus.cc:385] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.290074  6069 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.291177  6221 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.291167  6069 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.288904  6074 tablet_bootstrap.cc:492] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:37.292068  6221 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.296931  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:24:37.298921  5915 leader_election.cc:343] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:37.301316  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:37.302587  6069 ts_tablet_manager.cc:1434] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
W20260709 12:24:37.303110  5915 leader_election.cc:343] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:37.303622  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:37.304157  6069 tablet_bootstrap.cc:492] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:37.304572  6164 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:37.306675  6070 tablet_bootstrap.cc:492] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:37.306802  6146 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.308713  6070 ts_tablet_manager.cc:1403] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.052s	user 0.032s	sys 0.018s
I20260709 12:24:37.308597  6146 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.310498  6074 tablet_bootstrap.cc:654] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.310874  6146 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:37.315028  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:37.315881  5949 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:24:37.316391  6069 tablet_bootstrap.cc:654] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.316439  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:37.313027  6221 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:37.320551  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:37.322877  6146 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.322285  5695 catalog_manager.cc:5697] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:37.325950  6131 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.326663  6131 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.329119  6131 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:37.331133  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:37.332532  5949 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 1: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:24:37.332783  6070 raft_consensus.cc:359] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.333564  6070 raft_consensus.cc:385] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.333854  6070 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.335807  6070 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.338156  6131 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.338622  6131 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.339188  6071 tablet_bootstrap.cc:492] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:37.339540  6146 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:37.339778  6071 ts_tablet_manager.cc:1403] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.070s	user 0.018s	sys 0.016s
I20260709 12:24:37.340901  6070 ts_tablet_manager.cc:1434] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:24:37.342129  6070 tablet_bootstrap.cc:492] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:37.342392  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:37.342950  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:37.343314  6024 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 1: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:24:37.343762  6022 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:37.340780  6131 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:37.345105  5765 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:37.346570  5765 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:24:37.347838  6223 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.346963  6069 tablet_bootstrap.cc:492] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:37.354563  6222 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:37.355108  6222 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.355834  6222 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.354627  6069 ts_tablet_manager.cc:1403] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.051s	user 0.007s	sys 0.005s
I20260709 12:24:37.359491  6069 raft_consensus.cc:359] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.360255  6069 raft_consensus.cc:385] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.355232  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:37.360561  6069 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.360965  5948 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:37.363829  6117 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.364599  6117 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.367895  6117 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:37.368520  6069 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.377034  6223 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:37.374111  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:37.380069  6022 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:37.374905  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:37.381695  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:37.382669  6117 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:37.370082  6071 raft_consensus.cc:359] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.383271  6117 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.383459  6071 raft_consensus.cc:385] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.382067  5798 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:37.377281  6069 ts_tablet_manager.cc:1434] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.001s
I20260709 12:24:37.378881  6222 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.354967  6070 tablet_bootstrap.cc:654] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.386790  6069 tablet_bootstrap.cc:492] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:37.383809  6117 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.393113  6069 tablet_bootstrap.cc:654] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.393859  6071 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.394840  6071 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.397747  6071 ts_tablet_manager.cc:1434] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 12:24:37.398342  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:37.398985  5948 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.399171  6071 tablet_bootstrap.cc:492] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:37.401986  6155 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.402527  6155 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.404659  6155 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:37.406476  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:37.407200  5949 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:37.408507  5838 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:37.409475  6155 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:37.409873  6155 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.410204  6155 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.411872  6071 tablet_bootstrap.cc:654] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.422206  6166 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:37.430862  6074 tablet_bootstrap.cc:492] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:37.431468  6074 ts_tablet_manager.cc:1403] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.143s	user 0.018s	sys 0.021s
I20260709 12:24:37.434949  6074 raft_consensus.cc:359] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.435667  6074 raft_consensus.cc:385] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.435935  6074 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.436642  6074 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.438845  6164 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:37.442005  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:37.452047  5840 leader_election.cc:343] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:37.455111  6222 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:37.459535  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:37.460251  6022 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.460568  6155 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.470163  5948 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:37.475553  6074 ts_tablet_manager.cc:1434] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.001s
I20260709 12:24:37.491581  5763 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:37.491881  6074 tablet_bootstrap.cc:492] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:37.493659  6131 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:37.494236  6131 raft_consensus.cc:697] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:37.495139  6131 consensus_queue.cc:237] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.498302  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:37.477671  6117 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.482827  6022 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:37.485085  6024 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.487486  6069 tablet_bootstrap.cc:492] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:37.499789  6069 ts_tablet_manager.cc:1403] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.113s	user 0.041s	sys 0.037s
I20260709 12:24:37.502549  6069 raft_consensus.cc:359] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.503258  6069 raft_consensus.cc:385] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.503518  6069 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.504192  6069 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.505431  6070 tablet_bootstrap.cc:492] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:37.506132  6070 ts_tablet_manager.cc:1403] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.164s	user 0.012s	sys 0.007s
I20260709 12:24:37.510761  5798 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:37.511713  5840 leader_election.cc:343] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:37.527720  6155 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:37.528410  6155 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.528909  6155 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.530990  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:37.531391  6221 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:37.532166  6256 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:37.531693  5949 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.541608  6069 ts_tablet_manager.cc:1434] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.041s	user 0.019s	sys 0.007s
I20260709 12:24:37.543109  6069 tablet_bootstrap.cc:492] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:37.544792  6117 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:37.546888  6221 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.547561  6221 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.550274  6069 tablet_bootstrap.cc:654] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.551035  5797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:37.550422  5693 catalog_manager.cc:5697] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:37.550052  6070 raft_consensus.cc:359] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:37.552783  5915 leader_election.cc:343] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:37.568552  6070 raft_consensus.cc:385] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.558898  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:37.567641  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:37.570624  5798 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.570539  5947 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:37.572777  6221 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:37.556595  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:37.573803  6023 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.574134  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:37.574625  5838 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:37.575060  5874 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:37.575542  6260 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:37.575906  6260 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.576228  6260 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.576539  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:37.573244  6070 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.577605  5949 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:37.578277  6070 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.579991  5838 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:37.577936  6221 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:37.580741  6221 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.581226  6221 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.581430  6070 ts_tablet_manager.cc:1434] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.075s	user 0.007s	sys 0.001s
I20260709 12:24:37.582906  6070 tablet_bootstrap.cc:492] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:37.569762  6155 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:37.589076  6070 tablet_bootstrap.cc:654] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.592908  5798 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:37.594091  5797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:37.595320  6221 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.596587  6258 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:37.596071  6023 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:37.607957  6258 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:37.608830  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:37.609113  6258 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.610132  6166 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:37.610674  6166 raft_consensus.cc:697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:37.611542  5797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:24:37.612936  5915 leader_election.cc:343] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:37.611562  6166 consensus_queue.cc:237] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.615236  6070 tablet_bootstrap.cc:492] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:37.615892  6070 ts_tablet_manager.cc:1403] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.033s	user 0.006s	sys 0.006s
I20260709 12:24:37.616878  6069 tablet_bootstrap.cc:492] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:37.617575  6069 ts_tablet_manager.cc:1403] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.075s	user 0.014s	sys 0.006s
I20260709 12:24:37.620900  6069 raft_consensus.cc:359] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.621348  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:37.621745  6069 raft_consensus.cc:385] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.622087  6069 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.622260  6221 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:37.624155  6069 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.626365  6069 ts_tablet_manager.cc:1434] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:24:37.628394  6069 tablet_bootstrap.cc:492] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
W20260709 12:24:37.628414  5840 leader_election.cc:343] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:37.632978  6131 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:37.635550  6069 tablet_bootstrap.cc:654] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.637159  6070 raft_consensus.cc:359] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.638001  6070 raft_consensus.cc:385] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.638262  6070 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.638932  6070 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.641470  6070 ts_tablet_manager.cc:1434] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.002s
I20260709 12:24:37.642550  6070 tablet_bootstrap.cc:492] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:37.598984  6074 tablet_bootstrap.cc:654] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.599961  6071 tablet_bootstrap.cc:492] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:37.643684  6071 ts_tablet_manager.cc:1403] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.245s	user 0.030s	sys 0.020s
I20260709 12:24:37.605964  6260 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.642740  6069 tablet_bootstrap.cc:492] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:37.646359  6069 ts_tablet_manager.cc:1403] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.000s
I20260709 12:24:37.647166  5874 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 1: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:24:37.647644  6260 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:37.649091  6070 tablet_bootstrap.cc:654] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.650149  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:37.650406  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:37.651087  5949 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 1: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:24:37.651454  6221 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:37.651405  6071 raft_consensus.cc:359] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.652595  6071 raft_consensus.cc:385] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.653080  6071 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.654045  6071 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.656540  6071 ts_tablet_manager.cc:1434] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:24:37.657758  6071 tablet_bootstrap.cc:492] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:37.658617  6069 raft_consensus.cc:359] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.666914  6069 raft_consensus.cc:385] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.667261  6069 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.668012  6069 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.660064  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:37.666218  5874 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.678149  5799 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.678153  6069 ts_tablet_manager.cc:1434] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 12:24:37.680285  6069 tablet_bootstrap.cc:492] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:37.687002  5799 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:37.688339  5840 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:37.689815  6166 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:37.690608  6166 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:37.691215  6260 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.691752  6260 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.692675  6131 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:37.695577  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:37.695523  6166 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.697777  6069 tablet_bootstrap.cc:654] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.699726  6260 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:37.701048  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:37.703027  6071 tablet_bootstrap.cc:654] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.704258  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:37.712292  6074 tablet_bootstrap.cc:492] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:37.722227  6074 ts_tablet_manager.cc:1403] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.231s	user 0.019s	sys 0.012s
I20260709 12:24:37.724440  6069 tablet_bootstrap.cc:492] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:37.724686  6071 tablet_bootstrap.cc:492] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:37.725076  6069 ts_tablet_manager.cc:1403] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.001s
I20260709 12:24:37.725613  6071 ts_tablet_manager.cc:1403] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.068s	user 0.011s	sys 0.014s
I20260709 12:24:37.725342  6074 raft_consensus.cc:359] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.727687  6074 raft_consensus.cc:385] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.728031  6074 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.728816  6074 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.730482  6071 raft_consensus.cc:359] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.731093  6260 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:37.735496  6074 ts_tablet_manager.cc:1434] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:24:37.736430  6069 raft_consensus.cc:359] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.739243  6069 raft_consensus.cc:385] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.739372  6071 raft_consensus.cc:385] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.739645  6069 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.736755  6166 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:37.736930  5693 catalog_manager.cc:5697] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:37.739874  6074 tablet_bootstrap.cc:492] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:37.742321  6070 tablet_bootstrap.cc:492] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:37.742877  6070 ts_tablet_manager.cc:1403] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.101s	user 0.035s	sys 0.009s
I20260709 12:24:37.745891  6070 raft_consensus.cc:359] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.739778  6071 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.746660  6070 raft_consensus.cc:385] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.747040  6070 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.747287  6071 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.740708  6069 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.751076  6069 ts_tablet_manager.cc:1434] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.003s
I20260709 12:24:37.738549  6221 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.748315  5693 catalog_manager.cc:5697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:37.751655  6221 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.753860  6221 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:37.747820  6070 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.755726  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:37.757002  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:37.757648  5913 leader_election.cc:343] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:37.759162  5915 leader_election.cc:343] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:37.759770  5915 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:37.760651  6221 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:37.764370  6069 tablet_bootstrap.cc:492] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:37.769819  6155 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:37.770985  6071 ts_tablet_manager.cc:1434] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 12:24:37.772121  6071 tablet_bootstrap.cc:492] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:37.772539  6070 ts_tablet_manager.cc:1434] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.001s
I20260709 12:24:37.773368  6069 tablet_bootstrap.cc:654] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.773753  6070 tablet_bootstrap.cc:492] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:37.751323  6074 tablet_bootstrap.cc:654] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.783387  6070 tablet_bootstrap.cc:654] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.785885  6071 tablet_bootstrap.cc:654] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.798115  6070 tablet_bootstrap.cc:492] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:37.799333  6069 tablet_bootstrap.cc:492] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:37.800034  6069 ts_tablet_manager.cc:1403] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.041s	user 0.019s	sys 0.010s
I20260709 12:24:37.803977  6131 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.803525  6069 raft_consensus.cc:359] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.804541  6069 raft_consensus.cc:385] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.804986  6069 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.804517  6131 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.805783  6069 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.806960  6131 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:37.808243  6258 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.808630  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:37.798648  6070 ts_tablet_manager.cc:1403] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.007s
I20260709 12:24:37.808941  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:37.808883  6258 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.809963  5949 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 1: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:24:37.810686  6069 ts_tablet_manager.cc:1434] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 12:24:37.811069  5763 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:37.811722  6069 tablet_bootstrap.cc:492] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:37.812196  6131 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.812435  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:37.812798  6070 raft_consensus.cc:359] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.813199  5947 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:37.815266  6070 raft_consensus.cc:385] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.815649  6070 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.817647  6221 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.824630  6221 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.826853  6221 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:37.830354  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:37.830961  6258 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:37.828953  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:37.829732  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:37.833604  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:24:37.835799  5915 leader_election.cc:343] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:37.836637  5915 leader_election.cc:343] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:37.821358  6069 tablet_bootstrap.cc:654] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.842895  6258 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:37.843405  6258 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.843632  6074 tablet_bootstrap.cc:492] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:37.843753  6258 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.844556  6074 ts_tablet_manager.cc:1403] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.105s	user 0.025s	sys 0.010s
I20260709 12:24:37.838416  6070 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:37.848500  5840 leader_election.cc:343] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:37.837101  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:37.849819  6221 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:37.854131  6071 tablet_bootstrap.cc:492] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:37.854653  6071 ts_tablet_manager.cc:1403] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.083s	user 0.008s	sys 0.012s
I20260709 12:24:37.853495  5697 catalog_manager.cc:5697] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:37.855772  6074 raft_consensus.cc:359] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.856588  6074 raft_consensus.cc:385] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.856901  6074 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.857470  6071 raft_consensus.cc:359] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.857831  6074 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.858280  6071 raft_consensus.cc:385] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.858739  6071 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.858321  6258 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.859628  6071 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:37.861400  6258 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:37.861891  6070 ts_tablet_manager.cc:1434] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.000s
I20260709 12:24:37.863195  6070 tablet_bootstrap.cc:492] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:37.865819  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:37.866295  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:37.846858  6131 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:37.867195  5949 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.869539  6071 ts_tablet_manager.cc:1434] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:24:37.870970  6023 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:37.872081  6074 ts_tablet_manager.cc:1434] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 12:24:37.873013  6074 tablet_bootstrap.cc:492] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:37.873845  6071 tablet_bootstrap.cc:492] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:37.875226  5949 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:37.876762  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:37.877725  6258 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:37.878247  6258 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:37.879146  6258 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:37.879743  6074 tablet_bootstrap.cc:654] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.881062  6071 tablet_bootstrap.cc:654] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.889015  6023 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:37.894268  6069 tablet_bootstrap.cc:492] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:37.894924  6069 ts_tablet_manager.cc:1403] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.083s	user 0.006s	sys 0.010s
I20260709 12:24:37.898661  6069 raft_consensus.cc:359] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.902500  6069 raft_consensus.cc:385] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.903247  6069 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.904735  6069 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.912716  6069 ts_tablet_manager.cc:1434] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:24:37.916797  6069 tablet_bootstrap.cc:492] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:37.930390  5693 catalog_manager.cc:5697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:37.945833  6070 tablet_bootstrap.cc:654] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.945899  6074 tablet_bootstrap.cc:492] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:37.950119  6074 ts_tablet_manager.cc:1403] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.077s	user 0.034s	sys 0.012s
I20260709 12:24:37.948349  6258 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.953922  6258 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.955847  6258 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:37.956810  6074 raft_consensus.cc:359] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.956996  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:37.957592  6074 raft_consensus.cc:385] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.955956  6071 tablet_bootstrap.cc:492] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:37.957944  6074 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.958534  6071 ts_tablet_manager.cc:1403] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.088s	user 0.013s	sys 0.024s
I20260709 12:24:37.959631  6074 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.960762  6069 tablet_bootstrap.cc:654] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.963303  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:37.960017  5840 leader_election.cc:343] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:37.967082  5840 leader_election.cc:343] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:37.968849  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:37.969614  6155 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:37.966360  6071 raft_consensus.cc:359] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.970289  6071 raft_consensus.cc:385] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.970721  6071 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.971168  6070 tablet_bootstrap.cc:492] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:37.971760  6070 ts_tablet_manager.cc:1403] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.109s	user 0.006s	sys 0.016s
I20260709 12:24:37.971653  6071 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:37.974707  6070 raft_consensus.cc:359] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.975386  6070 raft_consensus.cc:385] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.975653  6070 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.976008  6074 ts_tablet_manager.cc:1434] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.022s	user 0.000s	sys 0.006s
I20260709 12:24:37.976316  6070 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.977320  6074 tablet_bootstrap.cc:492] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:37.983059  6070 ts_tablet_manager.cc:1434] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.006s
I20260709 12:24:37.983983  6070 tablet_bootstrap.cc:492] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:37.989852  6071 ts_tablet_manager.cc:1434] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 12:24:37.990499  6069 tablet_bootstrap.cc:492] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:37.991060  6069 ts_tablet_manager.cc:1403] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.077s	user 0.015s	sys 0.000s
I20260709 12:24:37.991889  6074 tablet_bootstrap.cc:654] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:37.994472  6069 raft_consensus.cc:359] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.995347  6069 raft_consensus.cc:385] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:37.995663  6069 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:37.996589  6069 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:37.999639  6222 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.000226  6222 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.001012  6070 tablet_bootstrap.cc:654] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:38.002671  6222 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:38.004937  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:38.006135  5949 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:38.007275  6069 ts_tablet_manager.cc:1434] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:24:38.007620  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:38.009125  6222 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.009753  6222 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.010426  5763 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:38.011157  6155 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.011421  6131 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:38.011905  6131 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.011721  6155 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:38.012288  6131 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.013283  6222 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:38.014309  6071 tablet_bootstrap.cc:492] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
W20260709 12:24:38.015811  5765 leader_election.cc:343] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:38.017068  6069 tablet_bootstrap.cc:492] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:38.017678  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:38.018391  6023 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:38.019412  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:38.019254  6222 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:38.020190  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:38.020196  5949 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:38.022977  6131 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.025318  5763 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:38.025527  6131 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:38.027397  6131 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:38.027621  6071 tablet_bootstrap.cc:654] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:38.027820  6131 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.028851  6166 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:38.030301  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:38.029552  6069 tablet_bootstrap.cc:654] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:38.030699  6260 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.031054  5949 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.031327  6260 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:38.033783  6260 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:38.033934  6131 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.036127  6074 tablet_bootstrap.cc:492] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:38.036269  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:38.036780  6074 ts_tablet_manager.cc:1403] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.060s	user 0.012s	sys 0.015s
W20260709 12:24:38.038118  5840 leader_election.cc:343] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:38.039778  6070 tablet_bootstrap.cc:492] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:38.040376  6070 ts_tablet_manager.cc:1403] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.004s
I20260709 12:24:38.040313  6074 raft_consensus.cc:359] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.041302  6074 raft_consensus.cc:385] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:38.043468  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:38.045307  6155 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:38.045082  6131 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:38.046519  5840 leader_election.cc:343] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:38.046854  6258 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.047699  6131 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:38.047487  6258 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:38.048708  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:38.049325  5948 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:38.051815  5765 leader_election.cc:343] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:38.052666  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:38.052940  6069 tablet_bootstrap.cc:492] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:38.053530  6069 ts_tablet_manager.cc:1403] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.045s	user 0.006s	sys 0.004s
I20260709 12:24:38.045615  6071 tablet_bootstrap.cc:492] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:38.055491  6071 ts_tablet_manager.cc:1403] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.004s
I20260709 12:24:38.056072  6070 raft_consensus.cc:359] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:38.056674  6070 raft_consensus.cc:385] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:38.042814  6074 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:38.058271  6074 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.056833  6070 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:38.061223  6131 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.053452  5946 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:38.065806  6258 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:38.066125  5838 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:38.063155  6131 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:38.067407  6260 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:38.067822  6260 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.068166  6260 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.068734  6131 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:38.070693  5949 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:38.068199  6069 raft_consensus.cc:359] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:38.071658  6069 raft_consensus.cc:385] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:38.071977  6069 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:38.072497  6074 ts_tablet_manager.cc:1434] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 12:24:38.072793  6069 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:38.073696  6074 tablet_bootstrap.cc:492] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:38.050504  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:38.077740  6069 ts_tablet_manager.cc:1434] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 12:24:38.076383  6070 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:38.077523  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:38.079372  6024 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.078274  6023 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:38.079423  6069 tablet_bootstrap.cc:492] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:38.081959  6071 raft_consensus.cc:359] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:38.082659  6071 raft_consensus.cc:385] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:38.082906  6071 raft_consensus.cc:740] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:38.083580  6071 consensus_queue.cc:260] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:38.081126  6070 ts_tablet_manager.cc:1434] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.002s
I20260709 12:24:38.085350  6070 tablet_bootstrap.cc:492] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:38.091094  6071 ts_tablet_manager.cc:1434] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.002s
I20260709 12:24:38.087095  5763 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:38.092547  6071 tablet_bootstrap.cc:492] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:38.092867  6131 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:38.093495  6131 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:38.094425  6131 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.075994  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:38.094955  6070 tablet_bootstrap.cc:654] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:38.087993  6069 tablet_bootstrap.cc:654] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:38.102327  6166 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.107439  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:38.107903  6166 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:38.088410  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:38.110605  5872 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:38.108249  6023 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:38.113360  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:24:38.114593  5840 leader_election.cc:343] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:38.118459  6155 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:38.133706  6155 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.134349  6155 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.134956  6074 tablet_bootstrap.cc:654] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:38.135340  6260 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:38.126672  6258 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:38.131389  6166 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:38.138088  6071 tablet_bootstrap.cc:654] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:38.124372  5763 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:38.115788  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:38.137573  6260 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:38.161978  6131 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:38.171136  6131 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.171574  6131 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.171979  6121 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.172617  6121 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.175657  6121 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:38.177714  6296 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:38.178853  5797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:38.163970  5948 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:38.180811  6024 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
W20260709 12:24:38.182080  5990 leader_election.cc:343] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:38.181336  5763 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:24:38.163393  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 43569.6us
I20260709 12:24:38.168268  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:38.185081  6223 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:38.185617  6223 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:38.185899  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:38.186558  6223 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.181979  6217 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:38.188827  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 74259.5us
W20260709 12:24:38.190900  5838 leader_election.cc:343] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:38.191452  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:38.196401  5947 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.192885  6217 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:38.214207  6217 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:38.215099  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:38.216317  5872 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:38.216486  5797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:38.218595  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:24:38.220103  5915 leader_election.cc:343] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:38.221787  6221 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:38.222975  6221 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.225291  6221 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.204852  6131 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:24:38.230577  5765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 102782us
I20260709 12:24:38.231277  6131 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:38.231539  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:38.236910  6222 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:38.236006  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:38.240739  6023 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.259655  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:38.261569  6074 tablet_bootstrap.cc:492] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:38.262475  6074 ts_tablet_manager.cc:1403] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.189s	user 0.018s	sys 0.011s
I20260709 12:24:38.262153  5799 raft_consensus.cc:2393] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 1: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:24:38.256794  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 98407.2us
I20260709 12:24:38.269281  6155 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:38.271335  6071 tablet_bootstrap.cc:492] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:38.271950  6071 ts_tablet_manager.cc:1403] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.180s	user 0.011s	sys 0.002s
I20260709 12:24:38.272065  6155 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:38.282668  5874 raft_consensus.cc:2393] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 1: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:24:38.282437  6261 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:38.293901  6071 raft_consensus.cc:359] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.294767  6071 raft_consensus.cc:385] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:38.295046  6071 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:38.295735  6071 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:38.303099  5840 leader_election.cc:343] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:38.303678  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:38.303943  6071 ts_tablet_manager.cc:1434] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:24:38.304445  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:38.305198  6071 tablet_bootstrap.cc:492] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:38.307519  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:38.308264  5946 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:38.309160  6074 raft_consensus.cc:359] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.313936  6074 raft_consensus.cc:385] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:38.314051  6070 tablet_bootstrap.cc:492] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:38.315409  6070 ts_tablet_manager.cc:1403] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.230s	user 0.017s	sys 0.007s
I20260709 12:24:38.315078  6074 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:38.320050  5987 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:38.316259  6261 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:38.321863  6302 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:38.322348  6302 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.323267  6302 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.317425  6074 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.326120  6070 raft_consensus.cc:359] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:38.326958  6070 raft_consensus.cc:385] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:38.327267  6070 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:38.328249  6074 ts_tablet_manager.cc:1434] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.000s
I20260709 12:24:38.331496  6261 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:38.334950  6130 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.335598  6130 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:38.338634  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:38.340162  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:38.341406  5874 raft_consensus.cc:2393] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 1: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:24:38.338773  6130 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:38.327907  6070 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:38.343190  6166 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:38.342288  6261 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:24:38.348551  5947 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:38.349344  6074 tablet_bootstrap.cc:492] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:38.353241  5763 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:38.354773  6221 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:38.357465  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:38.358183  5798 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.360018  6121 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.360536  6121 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.362866  6121 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:38.374974  6070 ts_tablet_manager.cc:1434] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.000s
I20260709 12:24:38.379405  5797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:38.380986  6070 tablet_bootstrap.cc:492] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:38.386039  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:38.387349  5948 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:24:38.388417  6070 tablet_bootstrap.cc:654] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:38.391749  6221 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:38.392772  6071 tablet_bootstrap.cc:654] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:38.393540  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:38.394349  5872 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.396852  5798 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:38.397753  6222 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:38.401410  6074 tablet_bootstrap.cc:654] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:38.402237  6069 tablet_bootstrap.cc:492] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:38.418154  6069 ts_tablet_manager.cc:1403] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.339s	user 0.013s	sys 0.028s
I20260709 12:24:38.402338  5838 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:38.403489  6302 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.396838  6023 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:38.407634  5796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:38.420989  5796 raft_consensus.cc:2393] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 1: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:24:38.408756  6155 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:38.409870  5696 catalog_manager.cc:5697] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:38.424997  5696 catalog_manager.cc:5697] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:38.428431  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:24:38.435937  5915 leader_election.cc:343] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:38.436347  5765 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:38.437899  6130 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:38.438503  6130 raft_consensus.cc:697] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:38.439658  6130 consensus_queue.cc:237] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:38.439702  6069 raft_consensus.cc:359] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:38.427207  6155 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:38.467669  6155 raft_consensus.cc:697] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:38.468918  6155 consensus_queue.cc:237] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:38.443430  6069 raft_consensus.cc:385] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:38.451565  6261 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:38.473163  6069 raft_consensus.cc:740] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:38.451611  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:38.476231  6069 consensus_queue.cc:260] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:38.431749  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:38.497017  6069 ts_tablet_manager.cc:1434] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.078s	user 0.009s	sys 0.000s
I20260709 12:24:38.508934  6069 tablet_bootstrap.cc:492] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:38.481726  6166 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:38.520438  6303 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:38.523048  5745 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:24:38.523651  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 95828.6us
I20260709 12:24:38.526961  6301 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.527804  6301 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.530365  5872 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:38.494352  6221 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.536259  6221 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:38.591163  6069 tablet_bootstrap.cc:654] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:38.597664  6301 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:38.600514  6302 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:38.600900  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:38.601863  6022 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:38.575302  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:38.603214  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:38.604279  6217 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:38.604831  6217 raft_consensus.cc:697] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:38.606256  6221 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:38.606916  6221 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:38.607282  6221 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.607697  6221 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:38.605854  6217 consensus_queue.cc:237] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:38.623818  6070 tablet_bootstrap.cc:492] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:38.624446  6070 ts_tablet_manager.cc:1403] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.248s	user 0.026s	sys 0.030s
I20260709 12:24:38.629782  6071 tablet_bootstrap.cc:492] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:38.630700  6071 ts_tablet_manager.cc:1403] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.326s	user 0.034s	sys 0.026s
I20260709 12:24:38.633253  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:38.634189  5799 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:38.646688  6074 tablet_bootstrap.cc:492] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:38.647395  6074 ts_tablet_manager.cc:1403] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.310s	user 0.018s	sys 0.001s
I20260709 12:24:38.649607  6303 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:38.657505  6223 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:38.660321  6221 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.660841  6070 raft_consensus.cc:359] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:38.661953  6070 raft_consensus.cc:385] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:38.662410  6070 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:38.652819  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:38.650032  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:38.654803  6074 raft_consensus.cc:359] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:38.667217  6074 raft_consensus.cc:385] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:38.667706  6074 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:38.666301  6070 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:38.668458  6074 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:38.672535  6071 raft_consensus.cc:359] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:38.673345  6071 raft_consensus.cc:385] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:38.673681  6071 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:38.674419  6071 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:38.677914  6070 ts_tablet_manager.cc:1434] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.002s
W20260709 12:24:38.680349  5915 leader_election.cc:343] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:38.684114  6071 ts_tablet_manager.cc:1434] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.002s
I20260709 12:24:38.685523  6071 tablet_bootstrap.cc:492] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:38.684064  6070 tablet_bootstrap.cc:492] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:38.688066  6074 ts_tablet_manager.cc:1434] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.001s
I20260709 12:24:38.673385  5872 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:38.697283  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:38.697657  5913 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:38.698057  5948 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.727599  5693 catalog_manager.cc:5697] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:38.736110  6223 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.736905  6223 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:38.741082  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:38.741990  5872 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:38.744038  5763 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:38.745196  6222 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:38.745700  6222 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.746116  6222 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.780041  5697 catalog_manager.cc:5697] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:38.787155  6074 tablet_bootstrap.cc:492] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:38.755571  6221 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:38.759052  6223 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:24:38.804862  6325 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.121s	user 0.000s	sys 0.002s
I20260709 12:24:38.806811  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:38.807559  6020 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:38.810078  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:38.809816  6222 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:38.812338  6222 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:38.816231  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:38.818859  6070 tablet_bootstrap.cc:654] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:38.788911  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 91378.4us
I20260709 12:24:38.802881  6221 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:24:38.793110  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 109574us
I20260709 12:24:38.827965  6221 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.828828  6221 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.828843  6069 tablet_bootstrap.cc:492] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:38.831426  6069 ts_tablet_manager.cc:1403] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.333s	user 0.026s	sys 0.018s
I20260709 12:24:38.835536  6131 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:38.839761  6069 raft_consensus.cc:359] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:38.840749  6069 raft_consensus.cc:385] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:38.841076  6069 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:38.821008  5872 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.856065  6071 tablet_bootstrap.cc:654] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:38.842367  6069 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:38.880507  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:38.881393  5798 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:38.904838  6324 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.905767  6324 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:38.907886  6301 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.908535  6324 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:38.908756  6301 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.907976  6298 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:24:38.911393  6301 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:38.911424  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:38.912858  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:38.913256  5874 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:38.914724  5948 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:38.915406  5798 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:24:38.915526  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:38.918437  6069 ts_tablet_manager.cc:1434] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.087s	user 0.007s	sys 0.001s
I20260709 12:24:38.919977  6069 tablet_bootstrap.cc:492] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:38.922857  6074 tablet_bootstrap.cc:654] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:38.929056  5693 catalog_manager.cc:5697] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 1 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:38.920238  6324 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:24:38.920436  5765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 94940.3us
I20260709 12:24:38.943599  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:38.943883  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:38.944788  6334 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.937708  6324 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.946784  5796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:38.947019  5987 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:38.929950  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:38.964479  6327 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:38.965234  6327 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:38.966166  6327 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.967046  6334 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:38.967954  6121 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:38.968580  6121 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:38.956269  6324 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.971956  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:38.972713  5871 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
W20260709 12:24:38.975045  5915 leader_election.cc:343] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:38.975461  6131 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:38.975977  5915 leader_election.cc:343] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:38.976439  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:38.932622  6020 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:24:38.977430  6333 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:38.969580  6334 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:38.966120  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:38.975642  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:38.982277  5913 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:38.969444  6121 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:38.980690  5948 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:38.987586  6332 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:38.998134  6332 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.011734  6332 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:39.025355  6070 tablet_bootstrap.cc:492] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:39.030177  6070 ts_tablet_manager.cc:1403] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.351s	user 0.014s	sys 0.011s
I20260709 12:24:39.032627  6071 tablet_bootstrap.cc:492] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:39.011229  6324 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:39.034236  6021 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 1: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:24:39.033294  6071 ts_tablet_manager.cc:1403] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.348s	user 0.015s	sys 0.009s
I20260709 12:24:39.035463  6324 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:39.007110  6131 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:24:39.033267  6070 raft_consensus.cc:359] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:39.042136  6070 raft_consensus.cc:385] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:24:39.042447  5765 leader_election.cc:343] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:39.043540  6071 raft_consensus.cc:359] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:39.042508  6070 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:39.044387  6071 raft_consensus.cc:385] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:39.044790  6071 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:39.045194  6070 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:39.046317  6261 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:39.048206  6070 ts_tablet_manager.cc:1434] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:24:39.051376  6070 tablet_bootstrap.cc:492] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:39.048879  6221 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:39.052138  6071 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:39.057329  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:39.058133  5873 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:39.060225  6221 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:39.059985  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:39.060858  5870 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:39.062013  6070 tablet_bootstrap.cc:654] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:39.064119  6332 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:24:39.066170  5990 leader_election.cc:343] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:39.069983  6069 tablet_bootstrap.cc:654] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:39.021427  6222 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.050s
I20260709 12:24:39.055022  5872 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:39.074656  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:39.085000  5948 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:24:39.086529  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:39.078586  5873 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:39.095752  6071 ts_tablet_manager.cc:1434] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.061s	user 0.005s	sys 0.003s
I20260709 12:24:39.096877  6071 tablet_bootstrap.cc:492] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:39.090402  6332 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:39.099234  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:39.099673  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:39.103178  6223 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.106972  5874 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 1: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:24:39.109079  6223 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:39.110755  6324 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:39.111868  6324 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:24:39.093600  5765 leader_election.cc:343] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:39.092240  5915 leader_election.cc:343] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:39.090374  5870 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:39.115291  6131 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.113014  6324 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:39.126677  6074 tablet_bootstrap.cc:492] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:39.127233  6074 ts_tablet_manager.cc:1403] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.340s	user 0.011s	sys 0.004s
I20260709 12:24:39.120772  6261 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.134738  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:39.120523  6131 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:39.139874  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:39.141645  6131 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:39.137768  6261 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:39.142522  6071 tablet_bootstrap.cc:654] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:39.129106  6223 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:39.145486  6332 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:39.125168  6327 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:24:39.155479  6074 raft_consensus.cc:359] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:39.156484  6074 raft_consensus.cc:385] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:39.171188  5763 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:39.156757  6074 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:39.184651  6261 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:39.186072  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:39.192716  6223 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:24:39.193486  5763 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 88299.9us
I20260709 12:24:39.195086  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:39.204470  6074 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:39.201336  5693 catalog_manager.cc:5697] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 0 to 2, leader changed from <none> to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 2 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:39.211668  6223 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:39.224232  6074 ts_tablet_manager.cc:1434] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.097s	user 0.005s	sys 0.003s
I20260709 12:24:39.224068  6223 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:39.209832  5949 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:39.225883  6121 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:39.216778  5840 leader_election.cc:343] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:39.217845  6131 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:39.226470  6121 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:39.228848  6121 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:39.232797  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:39.233686  5872 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:24:39.230780  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:39.238036  6074 tablet_bootstrap.cc:492] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:39.245831  5838 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:39.260900  6069 tablet_bootstrap.cc:492] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:39.261493  6069 ts_tablet_manager.cc:1403] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.342s	user 0.010s	sys 0.015s
I20260709 12:24:39.258318  6296 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:39.258745  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:39.280695  6296 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.294891  6296 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:39.283774  5913 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:39.303758  5796 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:39.306854  6332 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:39.307516  6332 raft_consensus.cc:697] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:39.308219  6074 tablet_bootstrap.cc:654] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:39.309947  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 52058.5us
I20260709 12:24:39.315822  6070 tablet_bootstrap.cc:492] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:39.316524  6070 ts_tablet_manager.cc:1403] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.265s	user 0.009s	sys 0.005s
I20260709 12:24:39.316350  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:39.296554  5949 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:39.321606  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:39.323014  6070 raft_consensus.cc:359] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:39.331044  6070 raft_consensus.cc:385] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:39.331434  6070 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:39.332158  6070 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:39.332113  6332 consensus_queue.cc:237] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:39.340838  6070 ts_tablet_manager.cc:1434] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.006s
I20260709 12:24:39.341974  6070 tablet_bootstrap.cc:492] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:39.351946  6071 tablet_bootstrap.cc:492] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:39.352569  6071 ts_tablet_manager.cc:1403] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.256s	user 0.019s	sys 0.009s
I20260709 12:24:39.361083  6069 raft_consensus.cc:359] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:39.386452  6069 raft_consensus.cc:385] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:39.387022  6069 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:39.388878  6069 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:39.356354  6296 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:39.413045  6069 ts_tablet_manager.cc:1434] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.151s	user 0.014s	sys 0.000s
I20260709 12:24:39.415782  6069 tablet_bootstrap.cc:492] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:39.418510  5797 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:24:39.419154  6071 raft_consensus.cc:359] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:39.420051  6071 raft_consensus.cc:385] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:39.420367  6071 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:39.420872  5697 catalog_manager.cc:5697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:39.405068  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:39.424425  6327 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:39.421197  6071 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:39.456393  6071 ts_tablet_manager.cc:1434] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.104s	user 0.008s	sys 0.001s
I20260709 12:24:39.462781  6296 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:39.455279  6360 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.478139  6360 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:39.478490  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:39.479323  5947 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:39.481003  6360 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:39.487251  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:24:39.490854  5765 leader_election.cc:343] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:39.455753  5763 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:24:39.493594  5763 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 48598.3us
I20260709 12:24:39.494804  5763 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:39.488291  5693 catalog_manager.cc:5697] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 1 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:39.498061  5693 catalog_manager.cc:5697] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 1 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:39.638959  5794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:39.533376  5840 leader_election.cc:343] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:39.642771  5795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:24:39.643250  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 109747us
W20260709 12:24:39.671376  5840 leader_election.cc:343] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:39.673051  5840 leader_election.cc:343] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:39.673947  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:39.562292  6069 tablet_bootstrap.cc:654] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:39.562263  6070 tablet_bootstrap.cc:654] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:39.500375  6071 tablet_bootstrap.cc:492] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:39.544428  6074 tablet_bootstrap.cc:492] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:39.711171  6074 ts_tablet_manager.cc:1403] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.486s	user 0.043s	sys 0.044s
I20260709 12:24:39.555536  6121 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:24:39.518954  6327 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.716630  6360 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:39.716924  6327 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:39.503847  6359 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:39.720336  6359 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.720994  6359 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:39.736663  6131 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:39.744386  6327 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:24:39.749058  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 76052.4us
I20260709 12:24:39.760460  6332 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:39.782743  6074 raft_consensus.cc:359] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:39.796620  6074 raft_consensus.cc:385] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:39.797430  6074 raft_consensus.cc:740] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:39.805728  6130 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.806962  6130 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:39.811602  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:39.821768  6071 tablet_bootstrap.cc:654] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:39.826524  6070 tablet_bootstrap.cc:492] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:39.828282  6130 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:39.800747  6074 consensus_queue.cc:260] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:39.829531  6368 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:39.833534  6074 ts_tablet_manager.cc:1434] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.122s	user 0.007s	sys 0.003s
I20260709 12:24:39.829710  6121 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:24:39.831027  6070 ts_tablet_manager.cc:1403] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.489s	user 0.014s	sys 0.019s
I20260709 12:24:39.835837  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:39.837000  5874 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:39.837296  6074 tablet_bootstrap.cc:492] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:39.845505  5763 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:39.846858  6131 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:39.847393  6131 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.847905  6131 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:39.868723  5947 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:39.871484  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:39.872239  6131 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:24:39.831275  5913 leader_election.cc:343] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:39.876852  6069 tablet_bootstrap.cc:492] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:39.882972  6069 ts_tablet_manager.cc:1403] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.467s	user 0.012s	sys 0.009s
W20260709 12:24:39.877880  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 114694us
I20260709 12:24:39.847523  6359 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:39.888015  6131 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:39.891100  6359 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:39.888604  5990 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:39.889742  6019 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:39.890125  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:39.903209  5872 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:39.892248  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 61174.6us
I20260709 12:24:39.903870  6070 raft_consensus.cc:359] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:39.936233  6121 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:39.889385  6069 raft_consensus.cc:359] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:39.947979  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:39.946830  6023 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:24:39.957122  6069 raft_consensus.cc:385] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:39.957469  5838 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:39.954012  6070 raft_consensus.cc:385] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:39.958983  6372 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:39.959173  6070 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:39.957911  6069 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:39.960251  6372 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:39.961726  6372 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:39.966872  6070 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:39.967063  6069 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:39.972525  6070 ts_tablet_manager.cc:1434] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.137s	user 0.010s	sys 0.002s
I20260709 12:24:39.950217  6074 tablet_bootstrap.cc:654] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:39.974581  6023 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:39.986217  6069 ts_tablet_manager.cc:1434] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.103s	user 0.008s	sys 0.004s
I20260709 12:24:39.991974  6131 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.992650  6131 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:39.994856  6131 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:24:39.989679  5915 leader_election.cc:343] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:39.998008  6071 tablet_bootstrap.cc:492] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:39.998433  5915 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:40.006624  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:40.007484  5949 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:39.999249  6071 ts_tablet_manager.cc:1403] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.538s	user 0.024s	sys 0.014s
W20260709 12:24:40.005614  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 117318us
W20260709 12:24:40.013960  5990 consensus_peers.cc:597] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:40.016543  6301 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:40.017472  6301 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:24:40.018491  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 61294.5us
I20260709 12:24:40.006202  5872 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:40.032862  6324 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:24:40.042776  5765 leader_election.cc:343] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:40.057103  6070 tablet_bootstrap.cc:492] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:40.062786  6071 raft_consensus.cc:359] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:40.063815  6071 raft_consensus.cc:385] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.064357  6071 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.065322  6071 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:40.067322  5763 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:40.071252  6359 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:40.072044  6359 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:40.073107  6359 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:40.077001  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:40.078915  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:40.082736  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:40.086856  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:40.116143  5693 catalog_manager.cc:5697] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 1 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:40.119040  5765 leader_election.cc:343] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:40.121352  5949 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:40.134280  6069 tablet_bootstrap.cc:492] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:40.150463  6070 tablet_bootstrap.cc:654] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.152982  6071 ts_tablet_manager.cc:1434] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.145s	user 0.008s	sys 0.000s
W20260709 12:24:40.127409  5765 leader_election.cc:343] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:40.158748  6335 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:40.163801  6301 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:40.177392  6071 tablet_bootstrap.cc:492] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:40.178890  6301 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:40.181893  6324 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.182592  6324 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:24:40.185286  5745 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:40.187913  5763 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:40.189944  5697 catalog_manager.cc:5697] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:40.194216  6069 tablet_bootstrap.cc:654] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.197889  5763 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:40.198995  6223 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:40.206765  6223 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:40.247045  6324 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:40.251260  6074 tablet_bootstrap.cc:492] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:40.252081  6074 ts_tablet_manager.cc:1403] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.415s	user 0.018s	sys 0.007s
W20260709 12:24:40.274312  5765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 47824.3us
I20260709 12:24:40.300806  6223 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:40.334457  6070 tablet_bootstrap.cc:492] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:40.335101  6070 ts_tablet_manager.cc:1403] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.310s	user 0.036s	sys 0.020s
I20260709 12:24:40.344630  6071 tablet_bootstrap.cc:654] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.366802  6069 tablet_bootstrap.cc:492] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:40.367408  6069 ts_tablet_manager.cc:1403] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.234s	user 0.023s	sys 0.017s
I20260709 12:24:40.373896  6223 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:40.366216  6372 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.384939  6372 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:40.391772  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:40.392614  6070 raft_consensus.cc:359] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:24:40.393548  5840 leader_election.cc:343] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:40.394073  6074 raft_consensus.cc:359] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:40.395392  6372 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:40.393530  6070 raft_consensus.cc:385] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.397279  6074 raft_consensus.cc:385] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.398883  6070 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.398964  6074 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.402233  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:24:40.402786  6388 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:24:40.402437  6369 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:24:40.403205  5948 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:40.406567  6074 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:40.406972  5838 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:40.409011  6074 ts_tablet_manager.cc:1434] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.157s	user 0.009s	sys 0.000s
I20260709 12:24:40.409862  6155 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:40.410391  6074 tablet_bootstrap.cc:492] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:40.410590  6155 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.411278  6155 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:40.376644  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 99895.1us
I20260709 12:24:40.416908  6301 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.417564  6301 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:40.430867  6070 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:40.431598  5795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:40.435871  5915 leader_election.cc:343] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:40.438822  6070 ts_tablet_manager.cc:1434] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.103s	user 0.009s	sys 0.000s
I20260709 12:24:40.440284  6070 tablet_bootstrap.cc:492] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:40.462508  5799 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:40.441150  6074 tablet_bootstrap.cc:654] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.441018  6069 raft_consensus.cc:359] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:40.441751  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:40.466616  6069 raft_consensus.cc:385] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.426050  6301 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:40.443315  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:40.467197  6069 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.469120  6069 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:40.447041  6155 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:40.449617  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:40.477937  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:40.477530  5697 catalog_manager.cc:5697] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:40.478775  5946 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:40.488286  6396 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.495359  6071 tablet_bootstrap.cc:492] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:40.496134  6071 ts_tablet_manager.cc:1403] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.319s	user 0.012s	sys 0.003s
I20260709 12:24:40.491892  6372 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.499547  6372 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:40.506204  6155 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:40.507802  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:24:40.510386  5840 leader_election.cc:343] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:40.513479  6069 ts_tablet_manager.cc:1434] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.146s	user 0.008s	sys 0.002s
I20260709 12:24:40.514968  6069 tablet_bootstrap.cc:492] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:40.515451  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:40.516978  6395 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.531075  6394 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.531713  6394 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:24:40.532255  5913 leader_election.cc:343] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:40.532889  5913 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:40.534123  6394 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:40.534600  6333 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:40.541029  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:40.542312  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:40.544714  5763 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:40.489140  6396 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:40.547335  6131 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:40.550047  6071 raft_consensus.cc:359] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:40.551090  6071 raft_consensus.cc:385] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.552407  6071 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.558967  6019 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:40.564373  6396 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:40.568234  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:40.569255  5872 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:24:40.569625  6070 tablet_bootstrap.cc:654] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.571125  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:40.571612  6372 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:40.574508  5946 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:40.602813  5795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:40.610556  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:40.611635  6074 tablet_bootstrap.cc:492] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:40.612260  6074 ts_tablet_manager.cc:1403] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.202s	user 0.008s	sys 0.010s
I20260709 12:24:40.589136  6071 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:40.614082  6070 tablet_bootstrap.cc:492] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:40.614521  6071 ts_tablet_manager.cc:1434] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.118s	user 0.007s	sys 0.006s
I20260709 12:24:40.615489  6074 raft_consensus.cc:359] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:40.614717  6070 ts_tablet_manager.cc:1403] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.175s	user 0.012s	sys 0.001s
I20260709 12:24:40.616339  6074 raft_consensus.cc:385] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.616652  6074 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.623730  6070 raft_consensus.cc:359] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:40.615634  5794 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:40.600152  5838 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:24:40.640658  5838 leader_election.cc:343] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:40.641505  6372 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:40.642189  6372 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:40.643244  6372 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:40.648115  6070 raft_consensus.cc:385] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.615839  6069 tablet_bootstrap.cc:654] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.637769  6223 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:40.625607  6071 tablet_bootstrap.cc:492] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:40.656347  6070 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.656338  6155 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:40.658377  6070 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:40.660151  6223 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:40.670104  6070 ts_tablet_manager.cc:1434] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.002s
W20260709 12:24:40.663103  5840 leader_election.cc:343] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:40.670727  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:40.671492  6070 tablet_bootstrap.cc:492] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:40.672034  6155 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:40.674705  6069 tablet_bootstrap.cc:492] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:40.675273  6069 ts_tablet_manager.cc:1403] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.161s	user 0.013s	sys 0.001s
I20260709 12:24:40.676962  6071 tablet_bootstrap.cc:654] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.678339  6069 raft_consensus.cc:359] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:40.665402  5990 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:40.688900  6396 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:40.617527  6074 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:40.701093  6074 ts_tablet_manager.cc:1434] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.088s	user 0.008s	sys 0.002s
I20260709 12:24:40.685979  6070 tablet_bootstrap.cc:654] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.663830  5697 catalog_manager.cc:5697] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:40.696437  5693 catalog_manager.cc:5697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:40.709205  6071 tablet_bootstrap.cc:492] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:40.709968  6071 ts_tablet_manager.cc:1403] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.085s	user 0.014s	sys 0.015s
I20260709 12:24:40.713832  6071 raft_consensus.cc:359] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:40.714620  6071 raft_consensus.cc:385] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.714915  6071 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.715790  6069 raft_consensus.cc:385] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.716233  6069 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.717681  6071 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:40.718180  6069 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:40.724634  6074 tablet_bootstrap.cc:492] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:40.725296  6069 ts_tablet_manager.cc:1434] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.050s	user 0.010s	sys 0.000s
I20260709 12:24:40.743453  6071 ts_tablet_manager.cc:1434] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:24:40.744956  6071 tablet_bootstrap.cc:492] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:40.754948  6069 tablet_bootstrap.cc:492] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:40.756281  6155 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.756850  6155 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:40.759613  6070 tablet_bootstrap.cc:492] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:40.760355  6070 ts_tablet_manager.cc:1403] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.089s	user 0.009s	sys 0.008s
I20260709 12:24:40.761025  6155 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:40.764403  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:40.765112  6071 tablet_bootstrap.cc:654] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.765872  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:40.766669  5949 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:40.768044  6070 raft_consensus.cc:359] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:40.768953  6070 raft_consensus.cc:385] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.769272  6070 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.770045  6070 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:40.771816  6070 ts_tablet_manager.cc:1434] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:24:40.772919  6070 tablet_bootstrap.cc:492] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:40.768286  5838 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:40.775545  6372 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:40.776015  6372 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.776343  6372 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:40.780293  5840 leader_election.cc:343] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:40.780344  6074 tablet_bootstrap.cc:654] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.783334  6070 tablet_bootstrap.cc:654] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.788386  6069 tablet_bootstrap.cc:654] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.809141  6069 tablet_bootstrap.cc:492] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:40.809736  6069 ts_tablet_manager.cc:1403] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.055s	user 0.016s	sys 0.000s
I20260709 12:24:40.815212  6069 raft_consensus.cc:359] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:40.818135  6071 tablet_bootstrap.cc:492] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:40.820336  6070 tablet_bootstrap.cc:492] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:40.821230  6070 ts_tablet_manager.cc:1403] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.049s	user 0.006s	sys 0.006s
I20260709 12:24:40.821216  6071 ts_tablet_manager.cc:1403] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.077s	user 0.014s	sys 0.012s
I20260709 12:24:40.824702  6070 raft_consensus.cc:359] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:40.820722  6069 raft_consensus.cc:385] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.825699  6069 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.825253  6071 raft_consensus.cc:359] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:40.825394  6070 raft_consensus.cc:385] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.826490  6071 raft_consensus.cc:385] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.827009  6070 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.827212  6071 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.828527  6071 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:40.829100  6069 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:40.827798  6070 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:40.832397  6070 ts_tablet_manager.cc:1434] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:24:40.834700  6074 tablet_bootstrap.cc:492] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:40.838160  6071 ts_tablet_manager.cc:1434] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:24:40.839102  6074 ts_tablet_manager.cc:1403] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.116s	user 0.010s	sys 0.004s
I20260709 12:24:40.839398  6069 ts_tablet_manager.cc:1434] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 12:24:40.840101  6071 tablet_bootstrap.cc:492] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:40.840632  6069 tablet_bootstrap.cc:492] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:40.842476  6070 tablet_bootstrap.cc:492] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:40.849288  6071 tablet_bootstrap.cc:654] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.849444  6074 raft_consensus.cc:359] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:40.850306  6074 raft_consensus.cc:385] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.850584  6074 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.851419  6074 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:40.859876  6074 ts_tablet_manager.cc:1434] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 12:24:40.861111  6074 tablet_bootstrap.cc:492] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:40.861213  6070 tablet_bootstrap.cc:654] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.866163  6069 tablet_bootstrap.cc:654] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.882761  6372 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:40.892418  6069 tablet_bootstrap.cc:492] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:40.892243  5794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:40.894459  6069 ts_tablet_manager.cc:1403] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.000s
W20260709 12:24:40.894762  5840 leader_election.cc:343] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:40.897275  6071 tablet_bootstrap.cc:492] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:40.897938  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:40.898629  5949 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:40.901880  6069 raft_consensus.cc:359] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:40.902778  6069 raft_consensus.cc:385] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.903064  6069 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.904155  6069 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:40.886426  6070 tablet_bootstrap.cc:492] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:40.906364  6070 ts_tablet_manager.cc:1403] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.002s
I20260709 12:24:40.906584  6069 ts_tablet_manager.cc:1434] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:24:40.897990  6071 ts_tablet_manager.cc:1403] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.058s	user 0.030s	sys 0.016s
I20260709 12:24:40.910753  6394 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:40.913889  6074 tablet_bootstrap.cc:654] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.916206  6069 tablet_bootstrap.cc:492] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:40.910708  6372 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:40.921083  6070 raft_consensus.cc:359] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:40.924774  6070 raft_consensus.cc:385] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.925189  6070 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.922662  5949 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:40.923854  6071 raft_consensus.cc:359] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:40.926568  6071 raft_consensus.cc:385] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.926995  6071 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.927928  6021 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:40.928570  6070 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:40.928138  6071 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:40.934676  6071 ts_tablet_manager.cc:1434] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:24:40.935986  6074 tablet_bootstrap.cc:492] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:40.936673  6074 ts_tablet_manager.cc:1403] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.076s	user 0.009s	sys 0.004s
I20260709 12:24:40.937319  6071 tablet_bootstrap.cc:492] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:40.930769  5838 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:40.939145  6372 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:40.939710  6372 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:40.940096  6074 raft_consensus.cc:359] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:40.935883  6070 ts_tablet_manager.cc:1434] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 12:24:40.940688  6372 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:40.941025  6074 raft_consensus.cc:385] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:40.958086  6074 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:40.958911  6074 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:40.961660  6074 ts_tablet_manager.cc:1434] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 12:24:40.965023  6074 tablet_bootstrap.cc:492] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:40.942459  6070 tablet_bootstrap.cc:492] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:40.944516  6069 tablet_bootstrap.cc:654] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.973060  6074 tablet_bootstrap.cc:654] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.950417  6071 tablet_bootstrap.cc:654] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.982744  6070 tablet_bootstrap.cc:654] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.993935  6223 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:40.996402  6395 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.997156  6395 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:40.999732  6395 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:41.001890  5794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:41.004945  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:41.010299  6074 tablet_bootstrap.cc:492] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:41.010979  6074 ts_tablet_manager.cc:1403] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.001s
I20260709 12:24:41.013147  6394 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.014061  6394 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.016691  6394 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:40.996490  6069 tablet_bootstrap.cc:492] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:41.021267  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:41.022215  5946 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:41.024022  6069 ts_tablet_manager.cc:1403] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.108s	user 0.016s	sys 0.006s
I20260709 12:24:41.025453  5763 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:41.013101  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:41.029676  6223 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:41.030206  6332 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.030833  6332 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.033212  6332 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:41.034766  6071 tablet_bootstrap.cc:492] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:41.035211  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:41.030261  6223 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.036168  6223 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.035424  5794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:41.037770  5915 leader_election.cc:343] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.035422  6071 ts_tablet_manager.cc:1403] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.098s	user 0.013s	sys 0.008s
W20260709 12:24:41.038703  5915 leader_election.cc:343] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.039211  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:41.040382  6333 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:41.039083  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:41.041319  6019 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:41.028932  6069 raft_consensus.cc:359] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:41.085400  6074 raft_consensus.cc:359] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:41.084646  6396 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.082340  6071 raft_consensus.cc:359] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:41.099462  5765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 47013.3us
I20260709 12:24:41.111372  6333 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.111972  6333 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.085816  6069 raft_consensus.cc:385] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.101743  6071 raft_consensus.cc:385] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.113814  6069 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.114010  6071 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.114581  6074 raft_consensus.cc:385] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.114921  6074 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.121342  6333 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:41.121376  6069 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:41.117527  6071 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.126278  5694 catalog_manager.cc:5697] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:41.099922  6369 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.134002  6369 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:41.134912  6069 ts_tablet_manager.cc:1434] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.110s	user 0.004s	sys 0.004s
I20260709 12:24:41.135486  6071 ts_tablet_manager.cc:1434] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.097s	user 0.011s	sys 0.007s
I20260709 12:24:41.136130  6069 tablet_bootstrap.cc:492] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:41.143750  6155 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.144754  6155 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:41.155448  6131 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.156302  6131 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:41.160421  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:41.161168  5949 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:41.161212  6131 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:41.147212  6369 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:41.163676  6074 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:24:41.164422  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 53238.8us
I20260709 12:24:41.161141  6155 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:41.167820  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:41.168671  5872 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:41.170274  5763 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:41.172025  6130 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:41.173698  6071 tablet_bootstrap.cc:492] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:41.175315  5838 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:41.177202  6369 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:41.177814  6131 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.178195  6369 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.178748  6369 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.178591  6131 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:41.173452  6130 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.186944  6130 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.188936  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:41.190399  5949 raft_consensus.cc:2393] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 1: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:24:41.190984  6131 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:41.182087  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:41.174242  6223 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.197180  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:41.210412  5794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:41.226459  6021 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:41.217410  6074 ts_tablet_manager.cc:1434] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.206s	user 0.006s	sys 0.007s
I20260709 12:24:41.225535  6155 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:41.228850  5915 leader_election.cc:343] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:41.229765  5915 leader_election.cc:343] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.226670  6070 tablet_bootstrap.cc:492] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:41.230144  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:41.230610  6070 ts_tablet_manager.cc:1403] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.288s	user 0.029s	sys 0.018s
I20260709 12:24:41.231576  6333 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:41.215155  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:24:41.227181  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 99973.4us
I20260709 12:24:41.234606  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:41.228227  6369 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:41.228714  6074 tablet_bootstrap.cc:492] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:41.234090  6070 raft_consensus.cc:359] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.236312  6070 raft_consensus.cc:385] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.236272  5763 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:41.239475  6369 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:41.240962  6359 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.236821  6070 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.242558  6130 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:41.243038  6070 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.244912  6130 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:41.250347  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:41.251027  5872 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.258679  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:41.259642  5948 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.263175  6396 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.264037  6396 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:41.270069  6070 ts_tablet_manager.cc:1434] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.003s
I20260709 12:24:41.271636  5872 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:41.273123  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:41.273900  6023 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.274932  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:41.275617  5799 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:41.278159  5794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:41.278611  6019 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.281363  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:41.282136  5944 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.284574  6023 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:41.290503  5948 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:41.296517  6396 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:41.297739  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:41.298766  5838 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:41.299768  6369 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:41.300331  6369 raft_consensus.cc:697] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:41.300621  6223 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:24:41.301646  5745 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:41.301322  6369 consensus_queue.cc:237] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:41.303586  6074 tablet_bootstrap.cc:654] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:41.305922  6071 tablet_bootstrap.cc:654] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:41.270107  5840 leader_election.cc:343] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.308065  5763 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:41.308707  5944 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:41.309026  5795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:41.309732  5795 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.312129  5763 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:41.314745  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:41.315666  6021 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.320140  6131 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:41.321012  6131 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:24:41.321599  5840 leader_election.cc:343] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.322184  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:41.326179  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:41.326781  6069 tablet_bootstrap.cc:654] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:41.328507  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:41.330049  6429 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:41.322238  6131 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.338390  6070 tablet_bootstrap.cc:492] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:41.324420  6394 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:41.325076  6359 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:41.350136  5795 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:41.351502  6394 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:41.352670  6394 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:41.401235  6074 tablet_bootstrap.cc:492] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:41.402033  6074 ts_tablet_manager.cc:1403] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.174s	user 0.015s	sys 0.009s
I20260709 12:24:41.407850  6074 raft_consensus.cc:359] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:41.409245  6429 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:41.409788  6074 raft_consensus.cc:385] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.410619  6074 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.411098  6395 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:41.412286  6074 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:41.418797  6021 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
W20260709 12:24:41.424140  5765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 100642us
I20260709 12:24:41.428917  6069 tablet_bootstrap.cc:492] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:41.428857  6074 ts_tablet_manager.cc:1434] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.001s
I20260709 12:24:41.429677  6069 ts_tablet_manager.cc:1403] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.294s	user 0.011s	sys 0.014s
I20260709 12:24:41.434104  6131 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.434651  6131 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.441872  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:41.442323  6074 tablet_bootstrap.cc:492] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:41.444136  6131 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:41.442768  5949 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:41.453899  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:41.459182  6070 tablet_bootstrap.cc:654] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:41.459169  6069 raft_consensus.cc:359] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:41.462330  6069 raft_consensus.cc:385] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.462610  6069 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
W20260709 12:24:41.463018  5765 leader_election.cc:343] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.465735  6071 tablet_bootstrap.cc:492] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:41.466377  6071 ts_tablet_manager.cc:1403] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.303s	user 0.018s	sys 0.006s
I20260709 12:24:41.480041  5763 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:41.483831  6071 raft_consensus.cc:359] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:41.484992  6074 tablet_bootstrap.cc:654] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:41.485944  6130 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:41.492578  6130 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.493055  6130 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.489034  5697 catalog_manager.cc:5697] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:41.487682  6369 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:41.489076  6071 raft_consensus.cc:385] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.499230  6071 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.500244  6071 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:41.507604  6069 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:41.532963  6071 ts_tablet_manager.cc:1434] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.000s
I20260709 12:24:41.534087  6069 ts_tablet_manager.cc:1434] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.104s	user 0.008s	sys 0.000s
I20260709 12:24:41.566452  6130 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.570771  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:41.571549  5949 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.580545  6332 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.581221  6332 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.583678  6332 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:41.577133  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:41.587936  5794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:41.591073  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:41.594913  5949 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
W20260709 12:24:41.600265  5765 leader_election.cc:343] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.596320  6333 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.601184  6333 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:24:41.606879  5915 leader_election.cc:343] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.607108  5697 catalog_manager.cc:5697] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:41.607693  5915 leader_election.cc:343] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.610235  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:41.616145  6427 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:41.623150  5763 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:41.626231  6394 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:41.624704  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:41.627463  6394 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:41.571766  6130 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:41.609380  5697 catalog_manager.cc:5697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:41.632189  5913 leader_election.cc:343] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.629679  6394 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.637200  6333 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:41.638381  6069 tablet_bootstrap.cc:492] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:41.644294  6074 tablet_bootstrap.cc:492] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:41.645200  6074 ts_tablet_manager.cc:1403] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.203s	user 0.008s	sys 0.016s
I20260709 12:24:41.651371  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:24:41.669963  5915 leader_election.cc:343] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.670830  6071 tablet_bootstrap.cc:492] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:41.673662  6070 tablet_bootstrap.cc:492] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
W20260709 12:24:41.673772  6440 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.114s	user 0.004s	sys 0.000s
I20260709 12:24:41.681699  6070 ts_tablet_manager.cc:1403] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.349s	user 0.024s	sys 0.051s
I20260709 12:24:41.681288  5915 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:41.694072  6070 raft_consensus.cc:359] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.694916  6070 raft_consensus.cc:385] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.695212  6070 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.695873  6070 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.698155  6070 ts_tablet_manager.cc:1434] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 12:24:41.699587  6070 tablet_bootstrap.cc:492] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:41.700862  6071 tablet_bootstrap.cc:654] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:41.687884  6333 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:41.705286  6074 raft_consensus.cc:359] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.706080  6074 raft_consensus.cc:385] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.706357  6074 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.707203  6074 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.709789  6074 ts_tablet_manager.cc:1434] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.002s
I20260709 12:24:41.725437  6074 tablet_bootstrap.cc:492] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:41.740005  6394 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.740654  6394 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.743652  6394 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:41.746328  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:41.747176  5949 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:41.747658  6070 tablet_bootstrap.cc:654] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:41.748675  5763 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:41.749518  6429 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:24:41.753701  6359 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:41.754179  6359 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.754547  6359 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.755162  5696 catalog_manager.cc:5697] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:41.757215  6069 tablet_bootstrap.cc:654] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:41.763631  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:41.764484  6024 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:41.777604  6359 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.782094  6359 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:41.783116  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:41.783951  5949 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.789858  6074 tablet_bootstrap.cc:654] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:41.791321  6443 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:24:41.797335  6071 tablet_bootstrap.cc:492] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:41.797964  6071 ts_tablet_manager.cc:1403] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.204s	user 0.024s	sys 0.011s
I20260709 12:24:41.799048  6070 tablet_bootstrap.cc:492] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:41.799826  6070 ts_tablet_manager.cc:1403] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.100s	user 0.019s	sys 0.010s
I20260709 12:24:41.800736  5949 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:41.801982  5763 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:41.801558  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:41.803972  6019 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.804603  6071 raft_consensus.cc:359] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:41.805439  6071 raft_consensus.cc:385] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.805753  6071 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.806511  6071 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:41.808136  6359 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:41.808748  6359 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:41.809653  6359 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:41.814116  6429 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:41.814853  6071 ts_tablet_manager.cc:1434] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:24:41.817240  6070 raft_consensus.cc:359] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:41.818454  6070 raft_consensus.cc:385] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.818800  6070 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.820248  6070 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:41.824242  6070 ts_tablet_manager.cc:1434] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 12:24:41.834206  6070 tablet_bootstrap.cc:492] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:41.835255  6429 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.835866  6431 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:41.835984  6429 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:41.837560  6431 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:41.838474  6429 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:41.840871  6071 tablet_bootstrap.cc:492] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:41.841972  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:24:41.843634  5840 leader_election.cc:343] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.847790  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:41.851132  5840 leader_election.cc:343] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.851517  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:41.858799  6070 tablet_bootstrap.cc:654] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:41.858340  6155 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:41.880224  5693 catalog_manager.cc:5697] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:41.887220  6019 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:41.900035  6394 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:41.900446  6121 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.905273  6451 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:24:41.908111  6069 tablet_bootstrap.cc:492] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:41.912516  6131 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:41.928768  6069 ts_tablet_manager.cc:1403] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.291s	user 0.005s	sys 0.011s
I20260709 12:24:41.931564  6121 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:41.939381  6121 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:41.940644  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:41.941416  5872 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:41.942407  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:41.943691  6155 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.943591  5798 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:41.943699  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:41.944518  6155 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:41.947559  6121 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:41.948005  6121 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.948348  6121 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.953557  6069 raft_consensus.cc:359] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:41.972219  6069 raft_consensus.cc:385] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.972688  6069 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.958386  6121 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:41.978502  5795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:41.967959  6155 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:24:41.980741  5840 leader_election.cc:343] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.982889  6121 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:41.983410  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:41.984119  5872 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.985368  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:24:41.995532  5840 leader_election.cc:343] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.995626  5795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:41.996197  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:41.996349  5795 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.000658  6155 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:42.004388  6071 tablet_bootstrap.cc:654] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:41.982028  6069 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.012744  6069 ts_tablet_manager.cc:1434] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.084s	user 0.006s	sys 0.003s
I20260709 12:24:42.012959  5872 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:42.017302  6070 tablet_bootstrap.cc:492] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:42.017946  6070 ts_tablet_manager.cc:1403] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.184s	user 0.048s	sys 0.054s
I20260709 12:24:42.022461  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:42.023456  6069 tablet_bootstrap.cc:492] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:42.030922  5795 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:42.034042  6395 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:42.056327  6074 tablet_bootstrap.cc:492] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:42.057057  6074 ts_tablet_manager.cc:1403] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.332s	user 0.019s	sys 0.011s
I20260709 12:24:42.068042  6069 tablet_bootstrap.cc:654] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.068873  6074 raft_consensus.cc:359] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:42.070782  6074 raft_consensus.cc:385] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.070219  6070 raft_consensus.cc:359] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:42.071121  6074 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.071480  6070 raft_consensus.cc:385] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.072199  6070 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.067116  6395 raft_consensus.cc:697] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:42.072566  6074 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:42.075608  6131 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:42.075489  6070 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:42.080164  6395 consensus_queue.cc:237] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:42.081058  6333 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.082540  6333 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:42.083989  6074 ts_tablet_manager.cc:1434] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.002s
I20260709 12:24:42.084834  6333 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:42.085939  6074 tablet_bootstrap.cc:492] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:42.086896  5794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:42.089370  5915 leader_election.cc:343] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:42.090617  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:42.092118  6071 tablet_bootstrap.cc:492] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
W20260709 12:24:42.092252  5913 leader_election.cc:343] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:42.092998  6071 ts_tablet_manager.cc:1403] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.277s	user 0.020s	sys 0.001s
I20260709 12:24:42.092921  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:42.096022  6074 tablet_bootstrap.cc:654] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.096414  6071 raft_consensus.cc:359] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.097282  6071 raft_consensus.cc:385] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.080916  6070 ts_tablet_manager.cc:1434] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.063s	user 0.007s	sys 0.001s
I20260709 12:24:42.097985  6333 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:42.099301  6070 tablet_bootstrap.cc:492] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:42.099864  6069 tablet_bootstrap.cc:492] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:42.100459  6069 ts_tablet_manager.cc:1403] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.077s	user 0.007s	sys 0.007s
I20260709 12:24:42.105019  6069 raft_consensus.cc:359] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:42.105824  6069 raft_consensus.cc:385] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.106164  6069 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.109875  6070 tablet_bootstrap.cc:654] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.110824  6069 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:42.097556  6071 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.124842  6069 ts_tablet_manager.cc:1434] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.002s
I20260709 12:24:42.127296  6074 tablet_bootstrap.cc:492] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:42.127861  6074 ts_tablet_manager.cc:1403] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.005s
I20260709 12:24:42.129092  6071 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.130653  6074 raft_consensus.cc:359] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:42.131467  6074 raft_consensus.cc:385] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.131747  6074 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.132064  6071 ts_tablet_manager.cc:1434] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.002s
I20260709 12:24:42.132439  6074 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:42.134198  6074 ts_tablet_manager.cc:1434] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:24:42.135402  6069 tablet_bootstrap.cc:492] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:42.136605  6074 tablet_bootstrap.cc:492] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
W20260709 12:24:42.136770  5765 consensus_peers.cc:597] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af -> Peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Couldn't send request to peer 6add331326ca47038308a447bf798fa7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:42.148661  6071 tablet_bootstrap.cc:492] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:42.152901  6069 tablet_bootstrap.cc:654] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.171029  6069 tablet_bootstrap.cc:492] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:42.171599  6069 ts_tablet_manager.cc:1403] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.014s
I20260709 12:24:42.175786  6070 tablet_bootstrap.cc:492] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:42.176486  6070 ts_tablet_manager.cc:1403] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.077s	user 0.008s	sys 0.006s
I20260709 12:24:42.176571  6069 raft_consensus.cc:359] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.177493  6069 raft_consensus.cc:385] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.177811  6069 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.178673  6069 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.180675  6071 tablet_bootstrap.cc:654] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.184422  6070 raft_consensus.cc:359] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:42.185257  6070 raft_consensus.cc:385] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.188542  6069 ts_tablet_manager.cc:1434] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 12:24:42.190157  6069 tablet_bootstrap.cc:492] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:42.185545  6070 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.197468  6070 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:42.201197  6070 ts_tablet_manager.cc:1434] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:24:42.201731  6121 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.202339  6121 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:42.202767  6069 tablet_bootstrap.cc:654] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.202821  5697 catalog_manager.cc:5697] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 1 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:42.205564  5794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:42.206677  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:42.207365  5872 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:42.207371  5794 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:42.208750  5990 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:24:42.209667  6460 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:42.210022  6460 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.210355  6460 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.211513  6121 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:42.224001  6074 tablet_bootstrap.cc:654] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.222945  6460 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:42.234519  6460 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:42.228710  6070 tablet_bootstrap.cc:492] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:42.240057  5795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:42.241068  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:42.241246  5795 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.241830  5872 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.243999  6071 tablet_bootstrap.cc:492] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:42.244577  6071 ts_tablet_manager.cc:1403] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.096s	user 0.012s	sys 0.004s
I20260709 12:24:42.247699  6071 raft_consensus.cc:359] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.248427  6071 raft_consensus.cc:385] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.248718  6071 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.251065  6071 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.253376  6071 ts_tablet_manager.cc:1434] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:24:42.254168  6070 tablet_bootstrap.cc:654] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.256628  6069 tablet_bootstrap.cc:492] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:42.257213  6069 ts_tablet_manager.cc:1403] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.067s	user 0.027s	sys 0.023s
I20260709 12:24:42.260288  6332 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.260969  6071 tablet_bootstrap.cc:492] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:42.260849  6332 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.265739  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:42.266505  5869 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:42.268139  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:42.268487  6069 raft_consensus.cc:359] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:42.269196  6069 raft_consensus.cc:385] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.269500  6069 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.270750  6332 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:42.273878  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:24:42.276998  5915 leader_election.cc:343] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:42.278628  6372 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.279131  5795 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:42.279261  6372 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.282198  6074 tablet_bootstrap.cc:492] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:42.282768  6074 ts_tablet_manager.cc:1403] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.146s	user 0.018s	sys 0.001s
I20260709 12:24:42.282948  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:42.283586  5949 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:42.284905  5838 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:42.285805  6429 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:42.286398  6429 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.286796  6429 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.290498  5990 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:24:42.269023  6301 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:42.292001  6301 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.292251  6460 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:42.270718  6069 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:42.292469  6301 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.292935  6460 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:42.272634  6071 tablet_bootstrap.cc:654] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.294025  6460 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:42.299327  6069 ts_tablet_manager.cc:1434] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.004s
I20260709 12:24:42.311272  6372 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:42.309659  6069 tablet_bootstrap.cc:492] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:42.291031  6074 raft_consensus.cc:359] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.334285  6074 raft_consensus.cc:385] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.334745  6074 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.335958  6074 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.340195  6429 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.337025  5872 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:42.334844  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:42.346405  6069 tablet_bootstrap.cc:654] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.346509  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:42.346740  6301 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.347275  5949 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:42.353614  5840 leader_election.cc:343] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:42.360781  6301 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:42.350968  6074 ts_tablet_manager.cc:1434] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.068s	user 0.009s	sys 0.001s
I20260709 12:24:42.361545  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:42.363897  5869 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.364540  6074 tablet_bootstrap.cc:492] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:42.348641  6429 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:42.364176  6431 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:42.362310  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:42.367790  6021 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.368805  6071 tablet_bootstrap.cc:492] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:42.369436  6071 ts_tablet_manager.cc:1403] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.109s	user 0.017s	sys 0.000s
I20260709 12:24:42.372880  6071 raft_consensus.cc:359] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:42.373138  5949 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:42.373926  6070 tablet_bootstrap.cc:492] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:42.374547  6070 ts_tablet_manager.cc:1403] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.146s	user 0.014s	sys 0.026s
I20260709 12:24:42.374949  5838 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:42.376036  6429 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:42.376577  6429 raft_consensus.cc:697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:42.377461  6429 consensus_queue.cc:237] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.381848  6431 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:42.382601  6074 tablet_bootstrap.cc:654] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.396566  6071 raft_consensus.cc:385] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.397032  6071 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.397266  6431 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.397848  6071 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:42.398115  6431 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:42.400625  6071 ts_tablet_manager.cc:1434] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.000s
I20260709 12:24:42.401847  6070 raft_consensus.cc:359] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:42.402632  6070 raft_consensus.cc:385] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.402899  6070 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.403296  6071 tablet_bootstrap.cc:492] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:42.403551  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:42.404199  5946 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:42.407049  5763 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:42.400914  6431 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:42.408440  6431 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:42.408870  6431 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.409281  6431 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.410164  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:42.415715  5872 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:42.412302  6070 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:42.418483  6070 ts_tablet_manager.cc:1434] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 12:24:42.418488  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:42.419687  6070 tablet_bootstrap.cc:492] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
W20260709 12:24:42.424060  5840 leader_election.cc:343] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:42.437000  6071 tablet_bootstrap.cc:654] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.412200  5869 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:42.424768  6021 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:42.451822  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:42.455716  6301 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:42.456403  6301 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:42.457517  6301 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.458901  6070 tablet_bootstrap.cc:654] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.463894  5693 catalog_manager.cc:5697] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 1 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:42.486104  5745 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:42.495371  6431 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:42.497988  6431 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:42.500698  6074 tablet_bootstrap.cc:492] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:42.503000  6074 ts_tablet_manager.cc:1403] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.139s	user 0.012s	sys 0.002s
I20260709 12:24:42.506156  6074 raft_consensus.cc:359] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.506953  6074 raft_consensus.cc:385] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.507313  6074 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.508442  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:42.508220  6074 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.510589  5869 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.510685  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:42.536435  6333 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.537132  6333 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:42.518843  5946 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.541867  5794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:42.543569  5915 leader_election.cc:343] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:42.551453  6069 tablet_bootstrap.cc:492] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:42.552274  6069 ts_tablet_manager.cc:1403] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.243s	user 0.024s	sys 0.008s
I20260709 12:24:42.523759  5696 catalog_manager.cc:5697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:42.575734  6460 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.576581  6460 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:42.581984  6074 ts_tablet_manager.cc:1434] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.079s	user 0.007s	sys 0.001s
I20260709 12:24:42.582422  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:42.583092  5949 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:42.583113  6074 tablet_bootstrap.cc:492] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:42.587073  6460 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:42.545753  6333 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:42.589289  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:42.572867  6461 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.057s
I20260709 12:24:42.598333  6460 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:42.673243  6070 tablet_bootstrap.cc:492] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:42.673969  6070 ts_tablet_manager.cc:1403] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.255s	user 0.027s	sys 0.044s
I20260709 12:24:42.677001  6070 raft_consensus.cc:359] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:42.677840  6070 raft_consensus.cc:385] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.678115  6070 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.679055  6070 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:42.680822  5869 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:42.642735  5794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:42.684665  5794 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 0.
I20260709 12:24:42.685673  6477 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.686473  6477 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:42.687609  6071 tablet_bootstrap.cc:492] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:42.688175  6071 ts_tablet_manager.cc:1403] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.285s	user 0.016s	sys 0.023s
I20260709 12:24:42.690963  6429 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.691059  5763 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:42.691635  6429 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.692553  6359 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:42.693099  6359 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:42.694011  6359 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:42.655102  6460 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.686378  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:42.711521  6395 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:24:42.715500  6069 raft_consensus.cc:359] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:42.724874  6069 raft_consensus.cc:385] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.725319  6069 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.706130  6460 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:42.731806  5915 leader_election.cc:343] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:42.732493  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:42.734740  6485 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.735131  6301 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:42.735260  6485 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.741446  6429 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:42.744444  6485 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:42.727819  6069 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:42.745863  5946 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:42.754398  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:42.760265  6069 ts_tablet_manager.cc:1434] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.208s	user 0.010s	sys 0.001s
I20260709 12:24:42.765475  6070 ts_tablet_manager.cc:1434] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.091s	user 0.007s	sys 0.001s
I20260709 12:24:42.770915  6070 tablet_bootstrap.cc:492] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:42.729820  6477 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:42.760514  6461 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:42.810777  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 79032.9us
W20260709 12:24:42.813532  5840 leader_election.cc:343] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:42.814996  6074 tablet_bootstrap.cc:654] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.815539  6069 tablet_bootstrap.cc:492] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
W20260709 12:24:42.820767  5765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 84675.2us
I20260709 12:24:42.823896  6131 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.824456  6131 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:42.820437  6071 raft_consensus.cc:359] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:42.827956  6071 raft_consensus.cc:385] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:42.816013  6461 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:42.832984  6461 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:42.833822  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:42.834632  5946 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 0.
I20260709 12:24:42.839149  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:42.829017  6071 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:42.844444  6071 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:42.845046  6018 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.848546  6131 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:24:42.845781  5915 leader_election.cc:343] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:42.854275  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:42.855360  5949 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:24:42.856679  6071 ts_tablet_manager.cc:1434] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.168s	user 0.004s	sys 0.007s
I20260709 12:24:42.858255  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:42.868305  6069 tablet_bootstrap.cc:654] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:42.860268  5693 catalog_manager.cc:5697] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 1 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:42.877719  5840 leader_election.cc:343] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:42.843304  6460 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:42.900252  6071 tablet_bootstrap.cc:492] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:42.890100  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:42.901679  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:42.902447  5870 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:42.902896  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:42.881585  6395 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 12:24:42.899022  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:42.940093  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:42.966557  6074 tablet_bootstrap.cc:492] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:42.967336  6074 ts_tablet_manager.cc:1403] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.384s	user 0.018s	sys 0.015s
I20260709 12:24:42.982873  6461 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:42.983067  6332 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:42.984138  6460 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:42.990252  5913 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:42.992076  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:42.992934  5871 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:42.985302  6121 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:42.998226  6332 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:42.998800  6332 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.999157  6332 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:43.001188  6070 tablet_bootstrap.cc:654] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:43.007714  5765 leader_election.cc:343] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:43.014034  6301 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.015736  6301 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:43.017737  6477 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:43.018800  6301 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:43.032596  6071 tablet_bootstrap.cc:654] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:43.036588  6074 raft_consensus.cc:359] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:43.037598  6074 raft_consensus.cc:385] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:43.037964  6074 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:43.031262  5695 catalog_manager.cc:5697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:43.044688  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 73553us
I20260709 12:24:43.050268  5795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:43.060537  5795 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:43.066433  5796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:43.069042  6074 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:43.073673  5915 leader_election.cc:343] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:43.049849  5838 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:43.111482  6074 ts_tablet_manager.cc:1434] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.144s	user 0.005s	sys 0.004s
I20260709 12:24:43.127676  6372 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:43.154130  6332 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:43.158200  6332 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:43.154515  5795 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:43.162267  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:43.163074  5873 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:43.172436  5763 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:43.174136  6130 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:43.174140  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:43.175026  6130 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.175477  6130 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:43.144217  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 94442.9us
W20260709 12:24:43.177785  5838 leader_election.cc:343] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:43.178464  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:43.179529  6429 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:43.180424  6429 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:43.160143  6372 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.181247  6372 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:43.199082  6372 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:43.211797  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:43.194418  6130 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:24:43.200191  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 67815.1us
W20260709 12:24:43.205248  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 47813.4us
I20260709 12:24:43.220676  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:43.234059  5990 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:24:43.238636  6396 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:43.239395  6396 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:43.240639  6396 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:43.275545  6071 tablet_bootstrap.cc:492] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:43.276353  6071 ts_tablet_manager.cc:1403] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.381s	user 0.030s	sys 0.014s
I20260709 12:24:43.258061  6333 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:43.258229  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:43.255702  6130 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:24:43.329797  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 111780us
I20260709 12:24:43.294970  6333 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.301312  5944 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:43.340924  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 129477us
I20260709 12:24:43.301352  5869 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 1: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:24:43.342455  6069 tablet_bootstrap.cc:492] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:43.343097  6069 ts_tablet_manager.cc:1403] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.534s	user 0.010s	sys 0.009s
W20260709 12:24:43.340106  5915 leader_election.cc:343] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:43.354772  6074 tablet_bootstrap.cc:492] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:43.340709  6333 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:43.375811  6070 tablet_bootstrap.cc:492] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:43.376701  6070 ts_tablet_manager.cc:1403] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.606s	user 0.047s	sys 0.067s
I20260709 12:24:43.403185  6022 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:43.406342  6372 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:43.421929  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:43.422667  5871 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:43.413558  6069 raft_consensus.cc:359] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:43.423763  6069 raft_consensus.cc:385] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:43.424036  6069 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:43.428274  6074 tablet_bootstrap.cc:654] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:43.424712  6069 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:43.442972  6070 raft_consensus.cc:359] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:43.445113  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:43.443917  6071 raft_consensus.cc:359] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:43.479990  6071 raft_consensus.cc:385] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:43.449985  6069 ts_tablet_manager.cc:1434] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.107s	user 0.007s	sys 0.000s
I20260709 12:24:43.483229  6071 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:43.484561  6071 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:43.444176  6070 raft_consensus.cc:385] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:43.486011  6070 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:43.475430  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:43.457870  6372 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.465523  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:43.500804  5949 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 1: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:24:43.490624  6333 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:43.515123  6071 ts_tablet_manager.cc:1434] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.238s	user 0.007s	sys 0.006s
I20260709 12:24:43.520207  6070 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:43.535887  6069 tablet_bootstrap.cc:492] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
W20260709 12:24:43.536300  5840 leader_election.cc:343] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:43.499330  6130 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
W20260709 12:24:43.558702  5765 leader_election.cc:343] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:43.503060  6372 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:43.571334  6333 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:43.571682  5944 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
W20260709 12:24:43.572474  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 230506us
I20260709 12:24:43.576767  6071 tablet_bootstrap.cc:492] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
I20260709 12:24:43.584630  6070 ts_tablet_manager.cc:1434] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.205s	user 0.005s	sys 0.005s
I20260709 12:24:43.586109  5796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:43.587232  5871 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
W20260709 12:24:43.589869  5915 leader_election.cc:343] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:43.604728  6070 tablet_bootstrap.cc:492] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:43.605115  6509 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.127s
I20260709 12:24:43.615182  6500 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.615881  6500 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:43.654476  5913 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:43.666519  6510 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:43.613709  6372 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:43.689041  6500 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:24:43.628844  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 167573us
I20260709 12:24:43.694249  5838 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:43.695369  6489 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:43.703516  6074 tablet_bootstrap.cc:492] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:43.704156  6074 ts_tablet_manager.cc:1403] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.358s	user 0.016s	sys 0.012s
I20260709 12:24:43.706640  6070 tablet_bootstrap.cc:654] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:43.668015  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:43.720038  6069 tablet_bootstrap.cc:654] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:43.731321  6071 tablet_bootstrap.cc:654] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:43.738688  5840 leader_election.cc:343] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:43.758284  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:43.760607  5763 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:43.761560  6131 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:43.762115  6131 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:43.762940  6131 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:43.748914  6074 raft_consensus.cc:359] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:43.765492  6074 raft_consensus.cc:385] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:43.765852  6074 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:43.768453  6503 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.769197  6503 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:43.772398  6130 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:24:43.778287  5745 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:43.779438  5694 catalog_manager.cc:5697] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 1 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:43.783998  5838 leader_election.cc:343] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:43.784741  5838 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:43.797168  6489 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:43.799181  5798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:43.800017  5798 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:43.801385  5915 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:43.785801  6074 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:43.806846  6485 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:43.814522  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:43.815428  5872 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:43.808272  6485 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.808420  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:43.832648  6074 ts_tablet_manager.cc:1434] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.128s	user 0.011s	sys 0.000s
I20260709 12:24:43.820473  6503 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:43.847349  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:43.847821  6070 tablet_bootstrap.cc:492] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:43.848673  6070 ts_tablet_manager.cc:1403] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.249s	user 0.025s	sys 0.019s
I20260709 12:24:43.855233  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:43.856180  6074 tablet_bootstrap.cc:492] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:43.828191  6485 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:43.856374  5874 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:43.831045  5871 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:43.892513  5763 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:43.903889  6520 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:43.904512  6520 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.904994  6520 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:43.935640  6070 raft_consensus.cc:359] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:43.936661  6070 raft_consensus.cc:385] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:43.937014  6070 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:43.955693  6520 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:43.958598  6520 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:43.969048  5871 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:43.970887  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:43.972332  6503 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:43.972889  6503 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:43.973886  6503 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:44.001010  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:43.992635  6070 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:44.015657  6074 tablet_bootstrap.cc:654] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.027330  6070 ts_tablet_manager.cc:1434] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.178s	user 0.009s	sys 0.003s
I20260709 12:24:44.028753  6070 tablet_bootstrap.cc:492] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:44.000115  6461 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 12:24:44.010950  6500 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.033826  6485 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:44.041445  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:44.074693  6069 tablet_bootstrap.cc:492] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:44.077112  6069 ts_tablet_manager.cc:1403] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.551s	user 0.044s	sys 0.034s
I20260709 12:24:44.036725  6500 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:44.085117  6500 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:44.089324  6071 tablet_bootstrap.cc:492] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:44.089967  6071 ts_tablet_manager.cc:1403] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.546s	user 0.013s	sys 0.014s
I20260709 12:24:44.058400  6461 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.094835  6461 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:44.097071  6461 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:44.080546  6070 tablet_bootstrap.cc:654] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.102603  5945 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 1: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:24:44.102959  6485 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:44.106225  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:44.106280  5794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:44.109804  5871 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.110016  5794 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 1: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:24:44.112951  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:44.114449  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:44.118528  5874 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:24:44.119110  6071 raft_consensus.cc:359] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:44.121135  6071 raft_consensus.cc:385] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.121518  6071 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.122668  6071 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:44.097477  5695 catalog_manager.cc:5697] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:44.126230  6332 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:44.139220  6071 ts_tablet_manager.cc:1434] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.000s
I20260709 12:24:44.149403  6071 tablet_bootstrap.cc:492] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Bootstrap starting.
W20260709 12:24:44.160715  5763 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 147552us
I20260709 12:24:44.177071  6069 raft_consensus.cc:359] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:44.178530  6069 raft_consensus.cc:385] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.179574  6069 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.178946  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:44.183560  6069 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:44.192797  6489 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.199862  6489 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:44.192979  6485 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.202690  6485 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:44.200107  5796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:44.213340  6069 ts_tablet_manager.cc:1434] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.136s	user 0.007s	sys 0.002s
W20260709 12:24:44.209432  5765 leader_election.cc:343] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.206777  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:44.205351  6485 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:44.219064  5874 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 0.
I20260709 12:24:44.221352  6070 tablet_bootstrap.cc:492] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:44.222020  6070 ts_tablet_manager.cc:1403] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.194s	user 0.011s	sys 0.010s
I20260709 12:24:44.225730  5913 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:44.231787  5990 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:44.236302  6069 tablet_bootstrap.cc:492] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Bootstrap starting.
I20260709 12:24:44.238727  6121 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.239950  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:44.244313  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:44.244691  5946 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:44.237274  6332 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:24:44.246296  5840 leader_election.cc:343] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.246762  6332 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.247140  6332 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.220781  6486 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:24:44.276553  5763 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:44.278314  6130 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:44.278997  6130 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.279433  6130 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.282593  6070 raft_consensus.cc:359] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:44.283854  6070 raft_consensus.cc:385] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.284231  6070 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.289780  6070 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:44.304077  6130 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:44.321200  6130 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:44.339375  6070 ts_tablet_manager.cc:1434] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.117s	user 0.008s	sys 0.001s
I20260709 12:24:44.342267  6074 tablet_bootstrap.cc:492] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:44.342623  6121 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:44.343099  6074 ts_tablet_manager.cc:1403] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.493s	user 0.022s	sys 0.004s
W20260709 12:24:44.346863  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 92154.7us
I20260709 12:24:44.353113  6070 tablet_bootstrap.cc:492] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:44.340787  6489 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:44.375676  6074 raft_consensus.cc:359] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:44.382064  6074 raft_consensus.cc:385] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.353209  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:44.386849  6074 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.389014  6074 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:44.395648  6074 ts_tablet_manager.cc:1434] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.000s
I20260709 12:24:44.396813  6074 tablet_bootstrap.cc:492] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
W20260709 12:24:44.399565  5765 leader_election.cc:343] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.387279  6071 tablet_bootstrap.cc:654] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.387559  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:44.382781  5871 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:44.406819  5945 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.407721  6515 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:24:44.412760  6069 tablet_bootstrap.cc:654] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.415817  6130 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:44.418264  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:44.422246  6505 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.422813  6505 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:44.424921  6505 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:44.419986  5763 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:44.429106  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:44.430320  5871 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:24:44.432755  6070 tablet_bootstrap.cc:654] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.436345  5796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:44.444412  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:24:44.446221  5915 leader_election.cc:343] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.432823  6074 tablet_bootstrap.cc:654] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.456174  6130 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:44.456805  5694 catalog_manager.cc:5697] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 1 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:44.407773  6332 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:44.459774  6130 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:44.460391  6332 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:44.460934  6130 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:24:44.462839  5840 leader_election.cc:343] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.463389  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:44.445611  5946 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:24:44.464861  6489 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:44.466898  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:24:44.468595  5915 leader_election.cc:343] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.469142  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:44.484319  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:44.485497  5871 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.478534  6505 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.506111  6332 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:44.510896  5796 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.645855  6069 tablet_bootstrap.cc:492] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: No bootstrap required, opened a new log
I20260709 12:24:44.646730  6069 ts_tablet_manager.cc:1403] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Time spent bootstrapping tablet: real 0.420s	user 0.011s	sys 0.009s
I20260709 12:24:44.662624  6069 raft_consensus.cc:359] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:44.664057  6069 raft_consensus.cc:385] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.664431  6069 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.664814  5945 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:44.671911  5871 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:44.673928  6071 tablet_bootstrap.cc:492] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: No bootstrap required, opened a new log
I20260709 12:24:44.674568  6071 ts_tablet_manager.cc:1403] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent bootstrapping tablet: real 0.533s	user 0.074s	sys 0.094s
I20260709 12:24:44.674515  5913 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:44.675976  6332 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:44.676512  6332 raft_consensus.cc:697] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:44.677953  6071 raft_consensus.cc:359] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:44.678751  6071 raft_consensus.cc:385] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.679059  6071 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.679730  6071 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:44.665236  6069 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:44.681704  6069 ts_tablet_manager.cc:1434] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.004s
I20260709 12:24:44.677386  6332 consensus_queue.cc:237] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:44.690604  6070 tablet_bootstrap.cc:492] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:44.691250  6070 ts_tablet_manager.cc:1403] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.338s	user 0.020s	sys 0.020s
I20260709 12:24:44.704326  6514 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:44.708251  6070 raft_consensus.cc:359] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:44.709288  6070 raft_consensus.cc:385] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.709674  6070 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.710448  6070 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:44.715595  6071 ts_tablet_manager.cc:1434] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 12:24:44.721601  6070 ts_tablet_manager.cc:1434] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 12:24:44.723583  6070 tablet_bootstrap.cc:492] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
W20260709 12:24:44.734972  5763 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 150274us
I20260709 12:24:44.758093  5763 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:44.759054  6131 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:44.759600  6131 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:44.760881  6131 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:44.782196  6497 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.782851  6497 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:44.785269  6497 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:44.781077  5695 catalog_manager.cc:5697] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:44.788547  6074 tablet_bootstrap.cc:492] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:44.800258  6074 ts_tablet_manager.cc:1403] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.404s	user 0.009s	sys 0.015s
I20260709 12:24:44.785373  5693 catalog_manager.cc:5697] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 1 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:44.786444  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:24:44.814787  5840 leader_election.cc:343] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.788201  6505 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:44.821885  5794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:44.834132  6074 raft_consensus.cc:359] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:44.834962  6074 raft_consensus.cc:385] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.835287  6074 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.835760  6070 tablet_bootstrap.cc:654] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:44.836079  6074 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:44.850585  6074 ts_tablet_manager.cc:1434] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.050s	user 0.003s	sys 0.004s
I20260709 12:24:44.853389  6074 tablet_bootstrap.cc:492] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:44.861337  6074 tablet_bootstrap.cc:654] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:44.863953  5840 leader_election.cc:343] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.864581  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:44.873440  6372 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:44.879174  6500 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.879879  6500 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:44.883201  6500 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:44.884380  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:44.887965  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:44.888772  5868 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
W20260709 12:24:44.889967  5765 leader_election.cc:343] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.893079  5763 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:44.894934  6515 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:44.895432  6515 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.897560  6515 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.912658  6121 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.913303  6121 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:44.915803  6121 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:44.921087  6552 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.922834  6552 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:44.930541  6552 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:44.930889  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:44.932633  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:44.933424  5796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:24:44.940230  5915 leader_election.cc:343] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:44.941068  5915 leader_election.cc:343] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:44.941510  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:44.943110  6503 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:44.947324  6515 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:44.951931  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:44.954676  5871 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:24:44.957031  6074 tablet_bootstrap.cc:492] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:44.957880  6074 ts_tablet_manager.cc:1403] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.105s	user 0.016s	sys 0.001s
I20260709 12:24:44.957881  6515 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:24:44.960119  5745 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:44.961649  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:44.963074  5871 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.963132  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:44.957091  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:44.965370  6070 tablet_bootstrap.cc:492] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:44.966049  6070 ts_tablet_manager.cc:1403] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.243s	user 0.012s	sys 0.005s
I20260709 12:24:44.965929  6542 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.961409  6074 raft_consensus.cc:359] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:44.979794  6074 raft_consensus.cc:385] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:44.980355  6074 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:44.981849  5871 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:44.981244  6074 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:44.983309  5763 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:44.983682  6015 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:44.987582  5694 catalog_manager.cc:5697] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:44.993314  6515 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:44.993674  6015 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:44.998027  6461 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.998564  6461 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:45.004910  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:45.006158  5874 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:45.007942  6074 ts_tablet_manager.cc:1434] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.002s
I20260709 12:24:44.994494  6070 raft_consensus.cc:359] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:45.009445  6070 raft_consensus.cc:385] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.009822  6070 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.008846  5796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:45.010857  6070 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:45.008836  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:45.013084  6121 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:45.013437  6121 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.013861  6121 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:45.017133  6515 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:45.018152  6515 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:45.021528  6074 tablet_bootstrap.cc:492] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:45.023981  6542 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:45.013468  6070 ts_tablet_manager.cc:1434] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.032s	user 0.004s	sys 0.000s
I20260709 12:24:45.026046  6070 tablet_bootstrap.cc:492] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:45.026365  6555 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:45.038626  6074 tablet_bootstrap.cc:654] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.004885  6461 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:45.042377  6070 tablet_bootstrap.cc:654] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.049464  6555 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:45.055629  6121 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:45.065523  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:45.068318  6074 tablet_bootstrap.cc:492] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:45.075555  5868 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:45.065500  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:45.076015  6074 ts_tablet_manager.cc:1403] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.008s
I20260709 12:24:45.079352  6074 raft_consensus.cc:359] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:45.080163  6074 raft_consensus.cc:385] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.080444  6074 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.084103  6070 tablet_bootstrap.cc:492] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: No bootstrap required, opened a new log
I20260709 12:24:45.084721  6070 ts_tablet_manager.cc:1403] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.059s	user 0.007s	sys 0.007s
I20260709 12:24:45.087422  6070 raft_consensus.cc:359] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:45.088161  6070 raft_consensus.cc:385] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.088442  6070 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.089178  6070 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:45.081178  6074 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:45.092026  6074 ts_tablet_manager.cc:1434] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:24:45.093240  6074 tablet_bootstrap.cc:492] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:45.064790  6121 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:45.099618  6074 tablet_bootstrap.cc:654] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.104578  5868 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:24:45.107553  6070 ts_tablet_manager.cc:1434] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:24:45.110772  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:45.112195  6121 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:45.113010  6121 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:45.113911  6121 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:45.121770  6359 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.123642  6359 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:45.124882  6520 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.128513  6359 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:45.125447  6520 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:45.131440  6520 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:45.132347  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:45.132578  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:24:45.134099  5765 leader_election.cc:343] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:45.136934  5765 leader_election.cc:343] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:45.149528  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:45.157612  5868 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:45.159283  5763 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:45.164260  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:45.165112  5943 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 0.
I20260709 12:24:45.170104  6520 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:45.170617  6520 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.170971  6520 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.175738  5763 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:45.175261  5695 catalog_manager.cc:5697] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:45.184756  6074 tablet_bootstrap.cc:492] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:45.187114  6074 ts_tablet_manager.cc:1403] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.094s	user 0.024s	sys 0.028s
I20260709 12:24:45.188565  6515 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:45.190075  6515 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.191040  6515 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.221223  6074 raft_consensus.cc:359] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:45.222997  6074 raft_consensus.cc:385] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.223321  6074 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.225123  6074 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:45.227772  6074 ts_tablet_manager.cc:1434] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 12:24:45.228503  5694 catalog_manager.cc:5697] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:45.242743  6074 tablet_bootstrap.cc:492] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Bootstrap starting.
I20260709 12:24:45.243350  6515 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:45.274497  6552 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:45.274950  6359 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:45.281935  6121 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.282681  6121 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:45.287976  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:45.288703  5943 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.249121  6520 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:45.290930  6520 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:45.296133  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:45.297036  6015 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.304155  6016 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.307610  6515 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:45.309444  6121 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:45.309321  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:45.311659  5871 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.309525  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:45.318774  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:45.321218  5943 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:45.329581  6552 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:45.323017  6372 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.331221  6074 tablet_bootstrap.cc:654] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:45.337024  6372 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:45.339591  6372 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:45.346575  5871 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:45.351667  5763 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:45.358716  6555 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:45.359266  6555 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:45.360152  6555 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:45.381880  6074 tablet_bootstrap.cc:492] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: No bootstrap required, opened a new log
I20260709 12:24:45.382416  6074 ts_tablet_manager.cc:1403] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Time spent bootstrapping tablet: real 0.140s	user 0.017s	sys 0.003s
I20260709 12:24:45.385643  6074 raft_consensus.cc:359] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:45.386335  6074 raft_consensus.cc:385] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:45.386628  6074 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Initialized, Role: FOLLOWER
I20260709 12:24:45.387326  6074 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:45.326665  6015 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:24:45.400418  6074 ts_tablet_manager.cc:1434] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:24:45.342170  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:45.343181  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:45.343338  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:45.402148  6019 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:45.402271  5947 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
W20260709 12:24:45.326331  5765 leader_election.cc:343] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:45.403373  5765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 77098.1us
I20260709 12:24:45.406070  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:45.405490  5763 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:24:45.407006  6539 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:45.407457  6539 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.407776  6539 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:45.409971  6359 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:45.423238  6131 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:45.423758  6131 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:45.424316  5990 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:45.424516  6131 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:45.425776  6461 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:45.478981  6539 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:45.482275  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:45.482438  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:45.483174  5947 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:45.483847  6018 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:45.494292  6539 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:45.499006  5868 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:24:45.501849  6018 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:24:45.503712  6461 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:45.504688  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:45.505661  6539 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:45.506160  6539 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:45.506415  5694 catalog_manager.cc:5697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:45.506994  6539 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:45.508147  5694 catalog_manager.cc:5697] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 1 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:45.501899  5947 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:24:45.527037  5695 catalog_manager.cc:5697] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 0 to 2, leader changed from <none> to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:45.546617  5799 raft_consensus.cc:3055] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:45.547035  5799 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:24:45.547832  5799 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:45.548839  5799 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:45.557247  5799 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:24:45.557606  5799 pending_rounds.cc:85] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 0
I20260709 12:24:45.557966  5799 pending_rounds.cc:107] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:24:45.558411  5799 raft_consensus.cc:2889] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:45.562228  6542 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:45.809505  6515 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:45.811879  6542 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.812466  6542 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:45.816195  5799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:45.816843  6542 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:45.816195  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:45.818271  5947 raft_consensus.cc:2393] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 1: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:24:45.819296  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:45.822167  6542 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:45.833392  6131 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:45.834151  6542 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:45.849735  6555 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:45.866182  6515 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:46.001019  5946 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:24:46.012097  6539 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:46.027536  6018 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:24:46.031392  6372 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:47.009197  2067 tablet_server.cc:179] TabletServer@127.2.4.193:0 shutting down...
W20260709 12:24:47.022790  5990 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 452 similar messages]
W20260709 12:24:47.031924  5990 consensus_peers.cc:597] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:47.043442  5915 consensus_peers.cc:597] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:47.055927  5915 consensus_peers.cc:597] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:47.086306  5990 consensus_peers.cc:597] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:47.093235  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:47.094467  2067 tablet_replica.cc:333] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.095448  2067 raft_consensus.cc:2243] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.096537  2067 raft_consensus.cc:2272] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.099812  2067 tablet_replica.cc:333] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.100700  2067 raft_consensus.cc:2243] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.101758  2067 raft_consensus.cc:2272] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.104307  2067 tablet_replica.cc:333] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
W20260709 12:24:47.104365  5990 consensus_peers.cc:597] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:47.105224  2067 raft_consensus.cc:2243] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.106415  2067 raft_consensus.cc:2272] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:47.108660  5990 consensus_peers.cc:597] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:47.109248  2067 tablet_replica.cc:333] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.110301  2067 raft_consensus.cc:2243] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.110845  2067 raft_consensus.cc:2272] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.113396  2067 tablet_replica.cc:333] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.114324  2067 raft_consensus.cc:2243] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.115319  2067 raft_consensus.cc:2272] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.117880  2067 tablet_replica.cc:333] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.118745  2067 raft_consensus.cc:2243] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.119313  2067 raft_consensus.cc:2272] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.121800  2067 tablet_replica.cc:333] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.122668  2067 raft_consensus.cc:2243] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:24:47.123616  5915 consensus_peers.cc:597] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:47.123768  2067 raft_consensus.cc:2272] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.126531  2067 tablet_replica.cc:333] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.127493  2067 raft_consensus.cc:2243] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.128046  2067 raft_consensus.cc:2272] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.130606  2067 tablet_replica.cc:333] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.131449  2067 raft_consensus.cc:2243] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.132498  2067 raft_consensus.cc:2272] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.135291  2067 tablet_replica.cc:333] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.136183  2067 raft_consensus.cc:2243] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.137192  2067 raft_consensus.cc:2272] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.139786  2067 tablet_replica.cc:333] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.140681  2067 raft_consensus.cc:2243] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.141263  2067 raft_consensus.cc:2272] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.144667  2067 tablet_replica.cc:333] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.145593  2067 raft_consensus.cc:2243] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.146701  2067 raft_consensus.cc:2272] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:47.149010  5915 consensus_peers.cc:597] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:47.152123  2067 tablet_replica.cc:333] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.153158  2067 raft_consensus.cc:2243] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.153782  2067 raft_consensus.cc:2272] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.156179  2067 tablet_replica.cc:333] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.157019  2067 raft_consensus.cc:2243] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.157608  2067 raft_consensus.cc:2272] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.160470  2067 tablet_replica.cc:333] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.161193  2067 raft_consensus.cc:2243] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.162448  2067 raft_consensus.cc:2272] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.164551  2067 tablet_replica.cc:333] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.165426  2067 raft_consensus.cc:2243] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.166126  2067 raft_consensus.cc:2272] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.168792  2067 tablet_replica.cc:333] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.169744  2067 raft_consensus.cc:2243] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.170383  2067 raft_consensus.cc:2272] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.173324  2067 tablet_replica.cc:333] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.174038  2067 raft_consensus.cc:2243] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.174912  2067 raft_consensus.cc:2272] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.176858  2067 tablet_replica.cc:333] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.177508  2067 raft_consensus.cc:2243] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.178432  2067 raft_consensus.cc:2272] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.180655  2067 tablet_replica.cc:333] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.181354  2067 raft_consensus.cc:2243] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.181895  2067 raft_consensus.cc:2272] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.183881  2067 tablet_replica.cc:333] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.184581  2067 raft_consensus.cc:2243] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.185778  2067 raft_consensus.cc:2272] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.188594  2067 tablet_replica.cc:333] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.189603  2067 raft_consensus.cc:2243] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.190833  2067 raft_consensus.cc:2272] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.193482  2067 tablet_replica.cc:333] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
W20260709 12:24:47.194168  5915 consensus_peers.cc:597] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:47.194228  2067 raft_consensus.cc:2243] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.195199  2067 raft_consensus.cc:2272] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.197463  2067 tablet_replica.cc:333] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.198196  2067 raft_consensus.cc:2243] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.199220  2067 raft_consensus.cc:2272] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.201910  2067 tablet_replica.cc:333] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.202766  2067 raft_consensus.cc:2243] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.203312  2067 raft_consensus.cc:2272] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.206007  2067 tablet_replica.cc:333] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.206924  2067 raft_consensus.cc:2243] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.207492  2067 raft_consensus.cc:2272] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.211534  2067 tablet_replica.cc:333] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.212397  2067 raft_consensus.cc:2243] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.212955  2067 raft_consensus.cc:2272] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.215430  2067 tablet_replica.cc:333] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.216084  2067 raft_consensus.cc:2243] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.216573  2067 raft_consensus.cc:2272] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.218636  2067 tablet_replica.cc:333] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.219270  2067 raft_consensus.cc:2243] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.220194  2067 raft_consensus.cc:2272] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.222422  2067 tablet_replica.cc:333] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.223076  2067 raft_consensus.cc:2243] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.224120  2067 raft_consensus.cc:2272] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.226565  2067 tablet_replica.cc:333] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.227465  2067 raft_consensus.cc:2243] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.228533  2067 raft_consensus.cc:2272] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.231335  2067 tablet_replica.cc:333] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.232132  2067 raft_consensus.cc:2243] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.232685  2067 raft_consensus.cc:2272] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:47.232816  5840 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:47.234582  5840 consensus_peers.cc:597] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:47.235224  2067 tablet_replica.cc:333] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.236006  2067 raft_consensus.cc:2243] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.236410  2067 raft_consensus.cc:2272] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.238942  2067 tablet_replica.cc:333] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.239681  2067 raft_consensus.cc:2243] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.240203  2067 raft_consensus.cc:2272] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.242462  2067 tablet_replica.cc:333] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.243237  2067 raft_consensus.cc:2243] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.244160  2067 raft_consensus.cc:2272] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.246804  2067 tablet_replica.cc:333] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.247407  2067 raft_consensus.cc:2243] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.247910  2067 raft_consensus.cc:2272] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.250543  2067 tablet_replica.cc:333] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.251156  2067 raft_consensus.cc:2243] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.252120  2067 raft_consensus.cc:2272] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.254886  2067 tablet_replica.cc:333] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.255597  2067 raft_consensus.cc:2243] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.256587  2067 raft_consensus.cc:2272] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.259087  2067 tablet_replica.cc:333] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.259799  2067 raft_consensus.cc:2243] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.260259  2067 raft_consensus.cc:2272] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.262148  2067 tablet_replica.cc:333] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.262729  2067 raft_consensus.cc:2243] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.263172  2067 raft_consensus.cc:2272] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.265587  2067 tablet_replica.cc:333] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.266259  2067 raft_consensus.cc:2243] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.266809  2067 raft_consensus.cc:2272] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.269411  2067 tablet_replica.cc:333] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.269990  2067 raft_consensus.cc:2243] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.270440  2067 raft_consensus.cc:2272] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.272706  2067 tablet_replica.cc:333] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.273411  2067 raft_consensus.cc:2243] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.274394  2067 raft_consensus.cc:2272] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.276819  2067 tablet_replica.cc:333] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.277369  2067 raft_consensus.cc:2243] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:47.277949  2067 raft_consensus.cc:2272] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:47.281095  2067 tablet_replica.cc:333] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: stopping tablet replica
I20260709 12:24:47.281903  2067 raft_consensus.cc:2243] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:47.283295  2067 raft_consensus.cc:2272] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:47.284416  5840 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:47.292112  5990 consensus_peers.cc:597] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:47.295755  5915 consensus_peers.cc:597] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:47.297781  5990 consensus_peers.cc:597] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:47.365522  2067 tablet_server.cc:196] TabletServer@127.2.4.193:0 shutdown complete.
W20260709 12:24:47.388999  5840 consensus_peers.cc:597] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:47.389129  5990 consensus_peers.cc:597] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:47.403555  5840 consensus_peers.cc:597] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:47.449088  5915 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:47.454367  5840 consensus_peers.cc:597] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:47.456614  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:47.466418  6598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:47.476784  5990 consensus_peers.cc:597] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:47.481654  6599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:47.485074  2067 server_base.cc:1061] running on GCE node
W20260709 12:24:47.487026  6601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:47.491707  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:47.492166  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:47.492432  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783599887492407 us; error 0 us; skew 500 ppm
I20260709 12:24:47.493268  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:47.497242  2067 webserver.cc:533] Webserver started at http://127.2.4.193:35095/ using document root <none> and password file <none>
I20260709 12:24:47.498064  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:47.498286  2067 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:24:47.503782  2067 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:24:47.513485  6607 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:47.514792  2067 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.002s
I20260709 12:24:47.515291  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-0-root
uuid: "1fc57407d56241b38a94d543e073f8af"
format_stamp: "Formatted at 2026-07-09 12:24:31 on dist-test-slave-4qvp"
I20260709 12:24:47.516004  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:47.539140  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:47.540535  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:47.542397  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:47.547676  6614 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 12:24:47.574721  5840 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:47.600723  5840 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:47.904390  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:24:47.904778  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.358s	user 0.008s	sys 0.000s
I20260709 12:24:47.905248  2067 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:24:47.911725  6614 tablet_bootstrap.cc:492] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:47.933054  6614 tablet_bootstrap.cc:492] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:47.934607  6614 tablet_bootstrap.cc:492] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:47.935438  6614 ts_tablet_manager.cc:1403] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.024s	user 0.021s	sys 0.000s
I20260709 12:24:47.938813  6614 raft_consensus.cc:359] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:47.939791  6614 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:47.940654  6614 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:47.943956  6614 ts_tablet_manager.cc:1434] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:24:47.945011  6614 tablet_bootstrap.cc:492] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:47.983690  6614 tablet_bootstrap.cc:492] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=1 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:47.984797  6614 tablet_bootstrap.cc:492] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:47.985585  6614 ts_tablet_manager.cc:1403] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.041s	user 0.026s	sys 0.004s
I20260709 12:24:47.988682  6614 raft_consensus.cc:359] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:47.989809  6614 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:47.990612  6614 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:47.992830  6614 ts_tablet_manager.cc:1434] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.004s
I20260709 12:24:47.993760  6614 tablet_bootstrap.cc:492] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:48.046844  6614 tablet_bootstrap.cc:492] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:48.048106  6614 tablet_bootstrap.cc:492] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:48.049093  6614 ts_tablet_manager.cc:1403] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.056s	user 0.029s	sys 0.013s
I20260709 12:24:48.052040  6614 raft_consensus.cc:359] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.052755  6614 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.053468  6614 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.055758  6614 ts_tablet_manager.cc:1434] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:24:48.057463  6614 tablet_bootstrap.cc:492] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:48.081441  6614 tablet_bootstrap.cc:492] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:48.082484  6614 tablet_bootstrap.cc:492] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:48.083163  6614 ts_tablet_manager.cc:1403] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.026s	user 0.019s	sys 0.000s
I20260709 12:24:48.085747  6614 raft_consensus.cc:359] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.086475  6614 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.087208  6614 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.089159  6614 ts_tablet_manager.cc:1434] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:24:48.090178  6614 tablet_bootstrap.cc:492] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:48.116362  6614 tablet_bootstrap.cc:492] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:48.116457  6622 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.117143  6622 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.119491  6622 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.122181  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.117677  6614 tablet_bootstrap.cc:492] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:48.127663  6614 ts_tablet_manager.cc:1403] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.038s	user 0.023s	sys 0.004s
W20260709 12:24:48.128065  5840 leader_election.cc:336] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.128690  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:48.129663  6622 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.130645  6614 raft_consensus.cc:359] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.131436  6614 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.132171  6614 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.134104  6614 ts_tablet_manager.cc:1434] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:24:48.134960  6614 tablet_bootstrap.cc:492] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:48.157289  6614 tablet_bootstrap.cc:492] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:48.158198  6614 tablet_bootstrap.cc:492] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:48.158819  6614 ts_tablet_manager.cc:1403] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.007s
I20260709 12:24:48.161302  6614 raft_consensus.cc:359] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.162109  6614 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.164777  6614 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.167790  6614 ts_tablet_manager.cc:1434] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:24:48.169117  6614 tablet_bootstrap.cc:492] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:48.187652  6602 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.188346  6602 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.190465  6622 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.190896  6602 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:48.191263  6622 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.193603  6614 tablet_bootstrap.cc:492] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:48.193868  6539 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.194916  6614 tablet_bootstrap.cc:492] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:48.195859  6614 ts_tablet_manager.cc:1403] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.027s	user 0.020s	sys 0.004s
W20260709 12:24:48.197604  5990 leader_election.cc:336] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.198652  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.203994  6614 raft_consensus.cc:359] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.204877  6614 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.194481  6539 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.205866  6614 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.207139  6539 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.209020  6614 ts_tablet_manager.cc:1434] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:24:48.210094  6614 tablet_bootstrap.cc:492] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:48.213889  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.237072  6622 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.241158  6614 tablet_bootstrap.cc:492] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:48.241680  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:48.242004  6615 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.243012  6615 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.246624  6615 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.246860  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.248700  6614 tablet_bootstrap.cc:492] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:48.249536  6614 ts_tablet_manager.cc:1403] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.012s
I20260709 12:24:48.252712  6614 raft_consensus.cc:359] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.253476  6614 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.254452  6614 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.258663  6614 ts_tablet_manager.cc:1434] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:24:48.259650  6614 tablet_bootstrap.cc:492] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
W20260709 12:24:48.264560  5915 leader_election.cc:336] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.264758  6602 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.265321  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:48.265468  6602 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.266247  6615 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.267771  6602 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:48.269093  6602 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.269742  6602 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.272948  6622 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:24:48.273676  5990 leader_election.cc:336] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.273586  6622 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.276063  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:48.276197  6622 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:24:48.277885  5840 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:48.283684  5990 leader_election.cc:336] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.283824  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:48.284807  6622 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.284929  6591 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.278239  6602 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.286664  6622 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.287233  6622 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.289397  6622 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.290356  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.293318  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:48.294045  2067 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:24:48.294471  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.389s	user 0.343s	sys 0.005s
I20260709 12:24:48.297219  6614 tablet_bootstrap.cc:492] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:48.298408  6614 tablet_bootstrap.cc:492] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:48.299372  6614 ts_tablet_manager.cc:1403] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.008s
I20260709 12:24:48.300405  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:48.301177  5946 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:48.302307  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:24:48.303519  5840 leader_election.cc:336] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.303716  6614 raft_consensus.cc:359] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.304272  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:48.304718  6614 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.304018  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.305676  6018 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:48.306700  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:48.307317  6623 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.308028  6602 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.308576  6602 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.309005  6602 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.308256  6591 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.316833  6591 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.317443  6591 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.305708  6614 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.319062  6461 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.319813  6461 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.319945  6602 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.321269  6614 ts_tablet_manager.cc:1434] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 12:24:48.322307  6614 tablet_bootstrap.cc:492] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:48.322362  6461 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:24:48.329555  5990 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:48.334465  5990 leader_election.cc:336] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.340023  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:48.340452  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:48.339664  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:24:48.341933  5840 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.342110  5871 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:48.342722  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:48.344340  6591 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:48.348141  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:48.352629  5947 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.354719  6602 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:24:48.353996  5990 leader_election.cc:336] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.354365  6614 tablet_bootstrap.cc:492] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:48.357292  6614 tablet_bootstrap.cc:492] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:48.358409  6614 ts_tablet_manager.cc:1403] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.036s	user 0.022s	sys 0.004s
I20260709 12:24:48.361784  6623 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.361060  6539 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.362579  6591 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.362799  6539 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:24:48.363229  5840 leader_election.cc:336] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.363509  6539 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.363405  6591 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.364930  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:48.360607  6602 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.366777  6461 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.367424  6595 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.368778  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:48.370002  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:48.367480  6461 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.366070  6602 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.373320  6602 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.373647  6595 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.374076  6591 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.374188  6595 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.374866  6595 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.378784  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:48.379675  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:24:48.380513  5990 leader_election.cc:336] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.373327  6461 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.381213  6461 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.381843  6461 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:24:48.385329  5990 leader_election.cc:336] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.386075  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:48.389741  6615 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.390755  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:48.392504  6602 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:48.389876  5990 leader_election.cc:336] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.393249  6602 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.391947  6615 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.362810  6614 raft_consensus.cc:359] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.400801  6614 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.369311  5947 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:24:48.392906  6461 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:48.403000  6614 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.388067  6539 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.407282  6614 ts_tablet_manager.cc:1434] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
W20260709 12:24:48.408006  5915 leader_election.cc:336] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.408139  6595 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.412887  6595 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:48.410048  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:48.409868  6614 tablet_bootstrap.cc:492] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
W20260709 12:24:48.413247  5840 leader_election.cc:336] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.413506  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:48.413748  6539 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.416023  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:48.422628  6616 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.423300  6616 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.417383  6634 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.424504  6634 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.425287  6622 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.425576  6616 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.425976  6622 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.427663  6634 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:48.428184  6622 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:48.419167  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:48.429242  5871 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:48.419559  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:48.430788  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:48.431118  6602 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:48.420722  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:48.415796  6461 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.433185  6637 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.433947  6617 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.434792  6617 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:24:48.435564  5840 leader_election.cc:336] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.435427  6617 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.436101  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:48.428956  6622 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.437111  6622 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.414137  6615 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:48.430804  5874 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.439508  6622 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:48.440192  6623 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.440671  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:48.441279  5947 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:48.420166  6591 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.442655  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:48.442520  6591 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.443612  6623 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.444015  6623 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.444393  6623 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.445550  6591 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:24:48.446923  5840 leader_election.cc:336] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.449532  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:48.450465  6602 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:48.451925  6602 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.454253  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:48.456117  6595 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.456794  6595 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.457389  6595 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.466331  6586 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.467265  6586 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.471179  6586 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:24:48.476989  5915 leader_election.cc:336] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.478667  6616 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.446585  6617 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.481200  6617 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:24:48.481437  5840 leader_election.cc:336] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.482146  5871 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:48.481777  6617 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.483247  6623 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.483656  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:48.484742  6591 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.485260  6591 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.485770  6591 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.489743  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:48.492261  5871 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:48.480475  6616 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.498921  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:48.504443  6622 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.505321  6622 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:48.507072  6616 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.509716  6616 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.538511  6616 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.511444  6461 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.539659  6461 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:48.540400  5915 leader_election.cc:336] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.541687  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:24:48.514170  5840 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.542732  6018 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:48.543645  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:48.544643  5947 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:48.508308  6622 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:48.434469  6637 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:48.531174  6615 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.531754  6617 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:48.534191  5693 catalog_manager.cc:5697] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:48.549921  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:48.550844  5947 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.543998  6539 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.552943  6539 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.510532  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:48.542050  6461 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.512933  6589 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.545998  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:24:48.554520  5840 leader_election.cc:336] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.557786  5874 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:24:48.555482  6589 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.556558  6616 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.560236  6539 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.561003  6589 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.561712  6461 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.562256  6461 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.548305  6615 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.567602  6637 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:48.569149  6615 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.563717  6602 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.570601  6461 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.565402  6634 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:24:48.572557  5840 leader_election.cc:336] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.573526  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.566421  6589 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.603029  6589 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.576426  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.606161  6019 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:48.575408  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.607678  6015 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:48.584301  5947 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:24:48.563269  6617 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.564076  6623 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:24:48.615741  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 52755.2us
I20260709 12:24:48.609464  6634 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.616757  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:48.617702  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:48.609915  6602 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.620659  6634 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.628489  6597 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:48.614061  6617 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.629580  6597 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:48.627527  6595 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.646871  6602 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.647248  6617 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.647037  6595 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:24:48.650564  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 64391.9us
W20260709 12:24:48.651414  5915 leader_election.cc:336] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:48.652377  5915 leader_election.cc:336] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.666846  6640 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.667727  6640 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.670756  6643 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:48.671545  6643 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:48.668583  6589 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.679788  6614 tablet_bootstrap.cc:492] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:48.672694  6643 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.684383  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:48.690006  6589 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.690464  6589 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.691751  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:48.693125  6589 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.700886  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.702060  6015 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:48.703496  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:48.704447  6018 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.709311  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:48.687886  6640 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:48.718526  6614 tablet_bootstrap.cc:492] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:48.719609  6614 ts_tablet_manager.cc:1403] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.310s	user 0.064s	sys 0.081s
I20260709 12:24:48.691313  6623 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.717155  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.721920  6623 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.722446  6019 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:48.724625  6623 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.726761  6642 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.727265  6642 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.727727  6642 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.708686  6597 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.741948  6649 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.742429  6649 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.742739  6617 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.742714  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.743286  6617 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.744571  6015 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:48.742808  6649 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.741165  6634 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:48.753232  5840 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.753105  6614 raft_consensus.cc:359] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:48.755915  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 139251us
I20260709 12:24:48.759234  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:48.755955  6614 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:48.750517  6016 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 2: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:24:48.764946  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:48.766657  5947 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:48.769956  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:48.771067  5874 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.771876  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:48.772960  5946 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:48.761664  6614 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.753072  6617 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.784775  6614 ts_tablet_manager.cc:1434] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.001s
I20260709 12:24:48.759969  5868 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:48.786556  6614 tablet_bootstrap.cc:492] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:48.790676  5874 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:24:48.754873  6650 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:24:48.794751  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 85507.5us
I20260709 12:24:48.749825  6616 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:24:48.798840  5915 leader_election.cc:336] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.805027  6634 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.797178  6591 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.806325  6616 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:48.806623  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:48.807396  5869 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:48.807726  6591 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:48.798539  6643 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.808073  6589 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.808595  6643 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.810850  6643 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.827548  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:48.830655  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:48.831525  5869 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.834199  6642 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.836045  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:48.836629  6642 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.836777  5947 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:48.836977  6654 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.837798  6634 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.840979  6640 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.797890  6650 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.842182  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.842962  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:48.874387  5946 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.846517  6589 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.847335  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.834250  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:48.852514  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.870466  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:48.870604  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:24:48.894598  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 60449.1us
I20260709 12:24:48.925379  5874 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.925598  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:48.926803  5987 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:48.928035  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:48.908238  6650 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:48.900454  6020 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:48.908959  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:48.859540  6654 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.912511  5869 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:24:48.937849  6623 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.901809  6019 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:24:48.938445  6623 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.877863  6622 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.938956  6623 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.913934  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.939210  6602 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:48.939134  6622 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.939919  6602 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:24:48.932935  5990 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.940526  6654 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:48.941345  6024 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.941813  6652 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.942207  6652 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.942601  6652 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.941025  6602 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:48.883306  6640 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:48.949350  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:24:48.934182  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 63881.3us
I20260709 12:24:48.952013  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:24:48.886046  5915 leader_election.cc:336] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:48.954000  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68001.8us
W20260709 12:24:48.954730  5915 leader_election.cc:336] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.874550  6634 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:48.941355  5990 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:48.958994  5990 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.935114  5874 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:24:48.938071  6643 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.939983  6622 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.965480  6643 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:48.965884  6650 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:48.966137  6643 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.966624  6650 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:48.967715  6650 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:48.914106  5946 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
W20260709 12:24:48.955359  5915 leader_election.cc:336] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:48.959792  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:48.949899  6461 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.983310  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:24:48.989125  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 61155us
I20260709 12:24:48.990804  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:48.977151  6654 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.969318  6017 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:48.993582  5695 catalog_manager.cc:5697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:48.995057  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:49.002152  6595 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:49.002753  6595 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:49.002415  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:49.003218  6595 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.003870  6647 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.988408  6616 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:49.005394  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 82154.7us
I20260709 12:24:48.997114  6615 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.979252  6649 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:49.019804  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:49.020998  6020 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.010514  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:48.998783  6018 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:24:49.009280  6647 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:49.028554  5946 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:24:49.038611  5990 leader_election.cc:336] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:49.039726  5990 leader_election.cc:336] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.039805  6649 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:49.034576  6024 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:24:49.034610  6647 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.045202  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:49.047152  6616 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:49.047742  6616 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:49.047952  6655 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:49.048403  6655 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:49.028015  6615 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:49.048769  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:49.053941  6615 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.054724  6024 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:24:49.055069  6595 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:49.045432  6461 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:49.056778  6652 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:49.057343  6461 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:49.058859  5990 leader_election.cc:336] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:49.059201  5915 leader_election.cc:336] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:49.059725  5915 leader_election.cc:336] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:49.059847  5990 leader_election.cc:336] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:49.060437  5915 leader_election.cc:336] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:49.060572  5990 leader_election.cc:336] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:49.061177  5990 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:49.063014  5990 leader_election.cc:336] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.058427  6015 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 2: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:24:49.069240  6614 tablet_bootstrap.cc:492] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:49.048835  6655 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.070569  6614 tablet_bootstrap.cc:492] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:49.071779  6614 ts_tablet_manager.cc:1403] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.285s	user 0.044s	sys 0.024s
I20260709 12:24:49.076915  6614 raft_consensus.cc:359] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:49.077832  6614 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:49.078610  6622 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:24:49.063736  5990 leader_election.cc:336] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.081058  6622 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:49.084514  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:49.083194  6614 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:49.090631  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:49.097574  6640 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:49.098776  6634 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:49.099104  6623 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:49.100396  6614 ts_tablet_manager.cc:1434] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.028s	user 0.010s	sys 0.000s
I20260709 12:24:49.101315  6623 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:49.101471  6614 tablet_bootstrap.cc:492] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:49.104133  6595 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:49.048668  6616 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:49.105213  6589 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:49.106200  6589 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:49.106678  6589 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.108080  6652 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:24:49.109324  5990 leader_election.cc:336] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.109953  6617 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:49.120042  6617 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:49.120496  6617 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.113114  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:24:49.124068  5990 leader_election.cc:336] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.124790  6019 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:49.125058  5990 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.127261  6020 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:24:49.130424  6615 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:49.132874  6615 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:49.136194  5695 catalog_manager.cc:5697] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:49.138195  5695 catalog_manager.cc:5697] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:49.139736  6461 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:49.112512  6655 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:49.143340  6655 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:49.116477  6647 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:49.104902  6643 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:49.149847  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 176091us
I20260709 12:24:49.109449  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:49.153842  6616 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:49.154034  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44734.5us
I20260709 12:24:49.155267  6619 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.155747  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:49.155920  6619 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:49.157900  6649 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:49.158437  6649 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:49.148924  6589 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:49.159745  5915 leader_election.cc:336] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.159386  6649 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:49.162320  5915 leader_election.cc:336] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.165643  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:49.166440  5868 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:49.169193  6019 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:24:49.172739  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:49.173717  6019 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 2: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:24:49.174154  6643 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:49.162883  6016 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 2: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:24:49.175483  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:49.175709  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:49.178644  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:24:49.181420  5915 leader_election.cc:336] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.183383  6461 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:49.185495  6656 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:49.196867  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:49.198390  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:49.186513  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:49.200009  6024 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.200654  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:49.201583  6019 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 2: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:24:49.186374  6656 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:49.203825  6619 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:24:49.207659  5990 consensus_peers.cc:597] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:49.208261  5990 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:49.208725  5990 leader_election.cc:336] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:49.209362  5990 leader_election.cc:336] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.214799  6461 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:49.211440  6615 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:49.215639  5990 leader_election.cc:336] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.215909  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:49.216625  6020 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.219411  6617 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:49.210268  6619 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.219848  6589 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:49.204645  6539 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:49.226501  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:49.228010  6647 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:49.229542  6616 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.224251  6619 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:49.254223  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:49.255330  6619 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:24:49.257736  5915 leader_election.cc:336] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.258306  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:49.221028  6652 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:49.229561  6617 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:49.204419  6656 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:49.235899  5990 leader_election.cc:336] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.240434  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:49.281068  6642 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:24:49.283875  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 48010.3us
W20260709 12:24:49.286063  5990 leader_election.cc:336] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:49.286742  5990 leader_election.cc:336] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.285341  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:24:49.244184  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 45933.2us
I20260709 12:24:49.244472  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:49.291215  5948 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 2: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:24:49.249749  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:49.251545  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:49.241077  6539 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:49.252465  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:49.258821  5695 catalog_manager.cc:5697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 2 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:49.293817  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 44028us
I20260709 12:24:49.255252  5946 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 2: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:24:49.252071  6024 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:24:49.290277  6671 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:49.304198  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:49.301013  5943 raft_consensus.cc:2393] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 2: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:24:49.317907  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:49.318751  5874 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.323493  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:49.323896  6673 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:49.324471  6673 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:24:49.324882  6673 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.298084  6539 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.324767  5869 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 2: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:24:49.327199  5915 leader_election.cc:336] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:49.334548  5915 leader_election.cc:336] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.330302  5947 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:24:49.301720  6015 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.291421  6652 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:49.303743  6016 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.302112  6647 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:49.343397  6615 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.348417  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:49.352022  5874 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:24:49.321509  6671 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:49.360960  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:49.365837  6461 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.366660  5987 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:49.368485  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:49.374313  5945 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:49.395267  6020 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:24:49.395897  6652 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:49.398416  5840 leader_election.cc:336] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:49.399085  5840 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.399485  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:49.407547  6660 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:24:49.403367  5840 leader_election.cc:336] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:49.411438  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:49.414561  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:49.417742  5949 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:24:49.419934  6648 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:49.420153  6586 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:24:49.422022  5990 consensus_peers.cc:597] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:49.425110  6660 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:24:49.386267  6647 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:49.430864  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:49.435789  6614 tablet_bootstrap.cc:492] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:49.439369  6622 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:49.440503  6675 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.460198  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:49.443326  6016 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:24:49.461892  6461 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:24:49.462826  5840 consensus_peers.cc:597] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:49.461719  6586 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:49.466177  6666 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:49.467360  6643 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.447849  6671 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:49.468590  6622 raft_consensus.cc:697] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:49.469157  6539 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:49.470984  6619 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.470989  6015 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:24:49.475510  5987 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:49.493389  6614 tablet_bootstrap.cc:492] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:49.459124  6648 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:49.512996  6614 ts_tablet_manager.cc:1403] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.412s	user 0.022s	sys 0.011s
I20260709 12:24:49.497382  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:49.509747  5869 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:49.535230  6614 raft_consensus.cc:359] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:49.539618  6614 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:49.546052  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:49.517242  6591 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.533113  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:49.547624  6614 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:49.544462  6646 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:49.518361  6619 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:49.554610  6652 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:49.535533  6586 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:49.566464  6589 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:49.474790  6675 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:49.551615  6647 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:24:49.596560  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 121448us
I20260709 12:24:49.604034  6673 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:49.600360  6539 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:49.604707  5695 catalog_manager.cc:5697] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:49.578758  6646 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:49.616319  6589 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:24:49.627995  6673 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:49.627310  6589 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:49.595728  6666 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:49.637401  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:24:49.638433  5874 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.592289  6652 raft_consensus.cc:697] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:49.690574  5874 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:24:49.583994  6648 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:49.585806  6622 consensus_queue.cc:237] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:49.765061  6614 ts_tablet_manager.cc:1434] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.250s	user 0.015s	sys 0.016s
I20260709 12:24:49.766258  6614 tablet_bootstrap.cc:492] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
W20260709 12:24:49.769795  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 128819us
I20260709 12:24:49.763767  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:24:49.772908  6653 log.cc:927] Time spent T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:24:49.646909  6698 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.142s
I20260709 12:24:49.815269  6022 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.824584  6619 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:49.830120  6619 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.830816  6619 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:49.765564  6646 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:49.833073  6619 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:49.770890  6666 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:49.834043  6675 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:49.845289  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:49.811324  6652 consensus_queue.cc:237] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:49.861536  6696 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:49.864394  6675 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.881508  6675 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:49.883873  6675 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:49.884505  6696 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:49.885408  6696 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:49.941978  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:49.948305  6619 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.948899  6675 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.950639  6675 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:49.954833  6702 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.956058  6702 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:49.959820  6675 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:49.960143  6016 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:49.959770  6619 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:49.962440  6675 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.966338  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:24:49.995136  5915 consensus_peers.cc:597] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:49.999830  5915 consensus_peers.cc:597] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:50.000813  5915 consensus_peers.cc:597] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:50.006690  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:50.010069  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:50.013321  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:50.022913  6020 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:50.033795  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:50.042572  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:50.036996  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:50.027110  6675 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:50.067353  6725 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.068691  6725 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:50.048960  6022 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
W20260709 12:24:50.093685  5915 consensus_peers.cc:597] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:50.094563  5915 consensus_peers.cc:597] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:50.096203  5838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.000s	sys 0.012s
W20260709 12:24:50.096776  5838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.000s	sys 0.012s
W20260709 12:24:50.104632  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 578087us
I20260709 12:24:50.110381  6015 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:50.111454  6609 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:50.132647  5870 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:50.136196  6642 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:50.138118  6669 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:50.140749  5947 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:24:50.151464  5915 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:50.152496  5915 consensus_peers.cc:597] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:50.153507  5915 consensus_peers.cc:597] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:50.160679  6650 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:24:50.169525  6702 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:50.170511  6702 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:50.175181  6619 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:50.177704  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:50.178102  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:50.158143  6725 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:50.181958  6675 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:50.196846  6642 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:50.201982  6648 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:24:50.190585  5915 consensus_peers.cc:597] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:50.214843  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 56839us
I20260709 12:24:50.249948  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:50.252080  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:50.254202  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:50.254690  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:50.252456  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:50.252324  5693 catalog_manager.cc:5697] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 2 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:50.261564  5915 consensus_peers.cc:597] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:50.286321  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 80281.3us
W20260709 12:24:50.287012  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 175908us
I20260709 12:24:50.289320  6609 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:50.296741  5870 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:50.296775  6675 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:50.263087  5693 catalog_manager.cc:5697] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 2 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:50.318754  5915 consensus_peers.cc:597] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:50.341813  5693 catalog_manager.cc:5697] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 2 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:50.371166  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 82078.9us
I20260709 12:24:50.380394  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:50.383602  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:50.385741  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:50.388767  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:50.390808  5868 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:50.377517  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:50.402274  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:24:50.403056  5915 consensus_peers.cc:597] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:50.388849  6614 tablet_bootstrap.cc:492] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:50.416472  6732 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:50.429562  5948 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:24:50.438078  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 60777.5us
I20260709 12:24:50.449512  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:50.451220  6732 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:50.452806  6634 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:50.453017  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:50.453359  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:24:50.478577  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 56560us
W20260709 12:24:50.488055  5990 consensus_peers.cc:597] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:50.498370  5840 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:50.529616  6614 tablet_bootstrap.cc:492] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
W20260709 12:24:50.536082  5990 leader_election.cc:336] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:50.534617  6732 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:50.559234  5695 catalog_manager.cc:5697] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:50.561445  6736 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 12:24:50.568809  5990 consensus_peers.cc:597] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:50.561514  6015 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:24:50.580156  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 126919us
I20260709 12:24:50.581293  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:50.583011  6020 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:50.583276  6732 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:50.584263  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:50.585693  6614 ts_tablet_manager.cc:1403] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.820s	user 0.054s	sys 0.046s
I20260709 12:24:50.589499  6732 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:50.594251  6614 raft_consensus.cc:359] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:50.595098  6614 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:50.596364  6614 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:50.598361  6591 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:50.601778  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:50.613070  6020 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:50.613497  6732 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:50.614878  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:50.619002  6732 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:50.620788  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:50.626554  6732 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:50.629012  6614 ts_tablet_manager.cc:1434] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.043s	user 0.002s	sys 0.007s
I20260709 12:24:50.630455  6614 tablet_bootstrap.cc:492] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:50.622897  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:24:50.637720  5915 consensus_peers.cc:597] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:50.634186  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:24:50.642760  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 77405.8us
I20260709 12:24:50.651990  6732 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:50.662246  6715 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:24:50.666378  6737 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:50.667104  6617 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:24:50.667136  6737 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:50.672161  6737 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:50.681243  6693 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:24:50.682142  6693 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:50.688303  6643 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:50.690382  6693 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:50.728259  6732 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.729130  5870 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:50.729000  6732 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:50.731488  6732 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:24:50.750178  5990 consensus_peers.cc:597] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:50.750931  5990 consensus_peers.cc:597] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:50.751371  5990 consensus_peers.cc:597] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:50.751889  5990 consensus_peers.cc:597] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:50.755338  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:50.755949  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:50.758553  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:50.759914  6732 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:50.760334  6022 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:24:50.761744  5990 consensus_peers.cc:597] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:50.762342  5990 consensus_peers.cc:597] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:50.763943  6655 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:24:50.783960  5840 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:50.784619  5840 consensus_peers.cc:597] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:50.785164  5840 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:50.783547  6667 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:24:50.787732  5840 consensus_peers.cc:597] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:50.786002  5990 consensus_peers.cc:597] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:50.796828  5840 consensus_peers.cc:597] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:50.800684  5990 consensus_peers.cc:597] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:50.801496  5990 consensus_peers.cc:597] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:50.802067  5990 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:50.839679  6017 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:50.842803  6647 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:24:50.847090  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:50.849782  6732 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.850484  6732 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:50.875633  5693 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40818 (request call id 119) took 1248 ms. Trace:
I20260709 12:24:50.886160  6732 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:50.890048  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:50.880893  5693 rpcz_store.cc:276] 0709 12:24:49.625147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.625340 (+   193us) service_pool.cc:224] Handling call
0709 12:24:50.873342 (+1248002us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:50.548456 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:50.569675 (+ 21219us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:50.569739 (+    64us) write_op.cc:435] Acquired schema lock
0709 12:24:50.569772 (+    33us) tablet.cc:662] Decoding operations
0709 12:24:50.570573 (+   801us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:50.570680 (+   107us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:50.570712 (+    32us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:24:50.571232 (+   520us) tablet.cc:701] Row locks acquired
0709 12:24:50.571282 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:24:50.571514 (+   232us) write_op.cc:270] Start()
0709 12:24:50.571700 (+   186us) write_op.cc:276] Timestamp: P: 1783599890571435 usec, L: 0
0709 12:24:50.571724 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:24:50.572759 (+  1035us) log.cc:844] Serialized 1294 byte log entry
0709 12:24:50.647065 (+ 74306us) op_driver.cc:464] REPLICATION: finished
0709 12:24:50.795576 (+148511us) write_op.cc:301] APPLY: starting
0709 12:24:50.795693 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:50.795932 (+   239us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:50.796109 (+   177us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:50.796770 (+   661us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:50.797005 (+   235us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:24:50.797077 (+    72us) write_op.cc:312] APPLY: finished
0709 12:24:50.797371 (+   294us) log.cc:844] Serialized 34 byte log entry
0709 12:24:50.797616 (+   245us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:50.803419 (+  5803us) write_op.cc:454] Released schema lock
0709 12:24:50.805224 (+  1805us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:50.845106 (+ 39882us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:50.845140 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":148159,"apply.run_cpu_time_us":2821,"apply.run_wall_time_us":34414,"num_ops":3,"prepare.queue_time_us":170922,"prepare.run_cpu_time_us":4686,"prepare.run_wall_time_us":44550,"replication_time_us":75127,"thread_start_us":212492,"threads_started":2}]]}
I20260709 12:24:50.924429  5873 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:50.927582  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:50.930927  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:50.932273  6732 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:50.958845  5840 consensus_peers.cc:597] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:50.965317  5840 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:50.978430  6643 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.979193  6643 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:50.979773  6614 tablet_bootstrap.cc:492] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:50.981046  6614 tablet_bootstrap.cc:492] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:50.982774  6614 ts_tablet_manager.cc:1403] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.353s	user 0.028s	sys 0.028s
W20260709 12:24:50.984652  5840 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:50.981549  6643 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:51.008898  5694 catalog_manager.cc:5697] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:51.011552  5694 catalog_manager.cc:5697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:51.013818  6771 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:51.017483  6614 raft_consensus.cc:359] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:51.026234  6615 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.026543  6614 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.027016  6615 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:51.028370  6614 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:51.030998  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:51.031728  6614 ts_tablet_manager.cc:1434] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.040s	user 0.004s	sys 0.003s
I20260709 12:24:51.032876  6614 tablet_bootstrap.cc:492] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:51.028404  6652 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
W20260709 12:24:51.034447  5990 leader_election.cc:336] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:24:51.021834  6765 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:24:51.042268  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:51.043154  6020 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:24:51.046244  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:24:51.029254  6615 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:24:51.057173  5840 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:51.031762  5942 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:24:51.054034  6655 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:51.054821  5694 catalog_manager.cc:5697] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:51.095238  6771 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.095937  6771 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:51.064630  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:24:51.137409  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 72874.3us
W20260709 12:24:51.139681  5915 leader_election.cc:336] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:51.117092  5694 catalog_manager.cc:5697] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:51.142849  6617 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:51.159155  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:24:51.160347  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 43810.7us
I20260709 12:24:51.162496  6647 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:51.163021  6647 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.163353  6647 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.153680  6617 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.170506  6617 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.179468  6771 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:51.148437  5694 catalog_manager.cc:5697] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:51.309098  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 64691.8us
I20260709 12:24:51.296928  6647 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:51.310511  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:24:51.312511  6781 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:24:51.315776  6617 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:24:51.317449  6780 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.076s	user 0.003s	sys 0.000s
W20260709 12:24:51.303774  5840 consensus_peers.cc:597] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:51.337944  6022 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:51.339210  6647 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:24:51.327997  5840 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:51.344493  6617 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:51.346005  6017 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:24:51.366601  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:51.367723  6017 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 3: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:24:51.389343  5990 consensus_peers.cc:597] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:51.391695  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:24:51.383133  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:51.393729  5944 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 3: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:24:51.397089  5990 consensus_peers.cc:597] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:51.397801  6769 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:24:51.407908  6769 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:51.410979  6769 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.411345  6769 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.418937  6614 tablet_bootstrap.cc:492] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:51.420020  6614 tablet_bootstrap.cc:492] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:51.420791  6614 ts_tablet_manager.cc:1403] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.388s	user 0.028s	sys 0.017s
W20260709 12:24:51.423038  5915 leader_election.cc:336] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:51.423627  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:24:51.424710  5840 consensus_peers.cc:597] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:51.425339  5840 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:51.425171  6614 raft_consensus.cc:359] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:51.426431  6614 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.428119  6788 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:51.433960  6614 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:51.438139  6614 ts_tablet_manager.cc:1434] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
W20260709 12:24:51.438648  5990 leader_election.cc:336] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:51.439429  5990 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:51.440377  6654 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:51.439052  5695 catalog_manager.cc:5697] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 2 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:51.439210  6614 tablet_bootstrap.cc:492] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:51.445575  5695 catalog_manager.cc:5697] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 2 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:51.466012  6769 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:24:51.489580  5840 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:51.491971  5693 catalog_manager.cc:5697] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:51.497802  6769 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:24:51.500262  5693 catalog_manager.cc:5697] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:51.501904  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:24:51.502250  5693 catalog_manager.cc:5697] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 1 to 2, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:51.502758  6015 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.536602  6015 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
W20260709 12:24:51.574891  5990 consensus_peers.cc:597] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:51.576740  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:24:51.579945  6769 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:51.580502  6769 raft_consensus.cc:697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:24:51.581466  6769 consensus_queue.cc:237] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:24:51.588616  5840 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:24:51.602698  6614 tablet_bootstrap.cc:492] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:51.603976  6614 tablet_bootstrap.cc:492] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:51.604703  6614 ts_tablet_manager.cc:1403] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.166s	user 0.026s	sys 0.026s
I20260709 12:24:51.607872  6614 raft_consensus.cc:359] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:51.608666  6614 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.618420  6614 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:51.621136  6614 ts_tablet_manager.cc:1434] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 12:24:51.622139  6614 tablet_bootstrap.cc:492] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:51.658932  5693 catalog_manager.cc:5697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 1 to 3, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 3 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:51.661832  6614 tablet_bootstrap.cc:492] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:51.663483  6614 tablet_bootstrap.cc:492] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:51.664485  6614 ts_tablet_manager.cc:1403] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.011s
I20260709 12:24:51.668006  6614 raft_consensus.cc:359] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:51.668746  6614 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.669870  6614 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:51.672444  6614 ts_tablet_manager.cc:1434] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:24:51.673754  6614 tablet_bootstrap.cc:492] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:51.701972  6614 tablet_bootstrap.cc:492] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:51.703368  6614 tablet_bootstrap.cc:492] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:51.704370  6614 ts_tablet_manager.cc:1403] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.031s	user 0.019s	sys 0.007s
I20260709 12:24:51.708199  6614 raft_consensus.cc:359] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:51.709061  6614 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.710451  6614 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:51.713908  6614 ts_tablet_manager.cc:1434] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.000s
I20260709 12:24:51.715224  6614 tablet_bootstrap.cc:492] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:51.760993  6614 tablet_bootstrap.cc:492] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:51.762424  6614 tablet_bootstrap.cc:492] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:51.763437  6614 ts_tablet_manager.cc:1403] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.049s	user 0.022s	sys 0.007s
I20260709 12:24:51.772912  6614 raft_consensus.cc:359] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:51.779830  6614 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.780762  6614 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:51.784011  6614 ts_tablet_manager.cc:1434] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 12:24:51.785264  6614 tablet_bootstrap.cc:492] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:51.811340  6614 tablet_bootstrap.cc:492] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:51.812743  6614 tablet_bootstrap.cc:492] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:51.813704  6614 ts_tablet_manager.cc:1403] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.029s	user 0.022s	sys 0.003s
I20260709 12:24:51.818202  6829 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.193:38701 every 8 connection(s)
I20260709 12:24:51.817828  6614 raft_consensus.cc:359] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:51.819312  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.193:38701
I20260709 12:24:51.820675  6614 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:51.821733  6614 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:51.911594  6614 ts_tablet_manager.cc:1434] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.097s	user 0.007s	sys 0.029s
I20260709 12:24:51.912842  6614 tablet_bootstrap.cc:492] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:52.012634  6785 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:52.020161  5915 consensus_peers.cc:597] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:24:52.021106  6766 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:52.054199  6022 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:52.058651  6769 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:52.087035  6762 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:52.090338  6614 tablet_bootstrap.cc:492] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:52.091825  6614 tablet_bootstrap.cc:492] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:52.082367  6831 heartbeater.cc:344] Connected to a master server at 127.2.4.254:41049
I20260709 12:24:52.092658  6614 ts_tablet_manager.cc:1403] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.180s	user 0.023s	sys 0.001s
I20260709 12:24:52.093032  6831 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:52.094314  6831 heartbeater.cc:507] Master 127.2.4.254:41049 requested a full tablet report, sending...
I20260709 12:24:52.097147  6614 raft_consensus.cc:359] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:52.098165  6614 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.099270  6614 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:52.101542  6793 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:52.130316  6773 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:52.101564  6614 ts_tablet_manager.cc:1434] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:24:52.146189  6614 tablet_bootstrap.cc:492] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:52.156958  6732 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.175756  6775 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:52.186867  6732 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:52.189285  6732 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:24:52.191119  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:24:52.196895  6732 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.218258  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:52.235692  6768 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:52.219813  6732 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:52.241817  6732 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:24:52.256021  5990 consensus_peers.cc:597] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:52.260670  6778 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:52.296144  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:52.330264  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:52.332475  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:24:52.336925  6732 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:52.339324  6793 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:52.406072  5840 consensus_peers.cc:597] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:52.376629  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:24:52.424948  6845 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.139s	user 0.001s	sys 0.001s
I20260709 12:24:52.405778  6732 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:24:52.435269  6766 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:52.419287  6854 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.427177  6732 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:52.435626  6855 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:52.484113  6614 tablet_bootstrap.cc:492] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:52.449589  6854 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:52.485977  6762 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:52.487509  6854 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:24:52.444188  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 86425.8us
I20260709 12:24:52.490998  6614 tablet_bootstrap.cc:492] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:52.492280  6614 ts_tablet_manager.cc:1403] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.350s	user 0.024s	sys 0.020s
I20260709 12:24:52.500630  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:52.532974  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:52.546455  6848 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:24:52.547181  6848 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:24:52.550078  5915 consensus_peers.cc:597] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:52.559096  6614 raft_consensus.cc:359] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:52.560010  6614 raft_consensus.cc:740] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.562901  6614 consensus_queue.cc:260] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:24:52.570458  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:52.561574  6732 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:52.574690  6773 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:52.583033  6614 ts_tablet_manager.cc:1434] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.090s	user 0.003s	sys 0.007s
I20260709 12:24:52.585930  6614 tablet_bootstrap.cc:492] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:52.580725  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:52.581187  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:52.611549  6848 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:24:52.612412  6848 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:52.613936  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:52.615954  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:24:52.618047  6732 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:52.654737  5915 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:24:52.657838  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:52.681087  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:24:52.683710  6867 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:52.727957  5915 consensus_peers.cc:597] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:52.728989  5915 consensus_peers.cc:597] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:52.755563  5693 ts_manager.cc:194] Re-registered known tserver with Master: 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:52.757234  6614 tablet_bootstrap.cc:492] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:52.758309  6614 tablet_bootstrap.cc:492] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:52.758944  6614 ts_tablet_manager.cc:1403] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.173s	user 0.019s	sys 0.000s
I20260709 12:24:52.761178  6614 raft_consensus.cc:359] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:52.761773  6614 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.762457  6614 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:52.764254  6614 ts_tablet_manager.cc:1434] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:24:52.765021  6614 tablet_bootstrap.cc:492] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:52.779749  6614 tablet_bootstrap.cc:492] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:52.780606  6614 tablet_bootstrap.cc:492] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:52.781225  6614 ts_tablet_manager.cc:1403] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 12:24:52.783607  6614 raft_consensus.cc:359] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:52.784219  6614 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.785036  6614 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:52.787271  6614 ts_tablet_manager.cc:1434] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:24:52.788199  6614 tablet_bootstrap.cc:492] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:52.791975  5693 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50588
W20260709 12:24:52.792615  5990 consensus_peers.cc:597] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:24:52.797435  6831 heartbeater.cc:499] Master 127.2.4.254:41049 was elected leader, sending a full tablet report...
I20260709 12:24:52.813410  6614 tablet_bootstrap.cc:492] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:52.814539  6614 tablet_bootstrap.cc:492] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:52.815212  6614 ts_tablet_manager.cc:1403] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.027s	user 0.020s	sys 0.000s
I20260709 12:24:52.821115  6614 raft_consensus.cc:359] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:52.821851  6614 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.823587  6614 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:52.826078  6614 ts_tablet_manager.cc:1434] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:24:52.826889  6614 tablet_bootstrap.cc:492] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:52.843163  6614 tablet_bootstrap.cc:492] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:52.844234  6614 tablet_bootstrap.cc:492] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:52.844838  6614 ts_tablet_manager.cc:1403] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.002s
I20260709 12:24:52.847420  6614 raft_consensus.cc:359] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:52.848141  6614 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.848862  6614 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:52.850745  6614 ts_tablet_manager.cc:1434] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:24:52.851420  6614 tablet_bootstrap.cc:492] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:52.866212  6614 tablet_bootstrap.cc:492] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:52.867254  6614 tablet_bootstrap.cc:492] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:52.868105  6614 ts_tablet_manager.cc:1403] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.005s
I20260709 12:24:52.870949  6614 raft_consensus.cc:359] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:52.871716  6614 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:52.872419  6614 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:52.874830  6614 ts_tablet_manager.cc:1434] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:24:52.875696  6773 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:52.875728  6614 tablet_bootstrap.cc:492] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
W20260709 12:24:53.116690  5840 consensus_peers.cc:597] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:53.117595  5840 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:53.166427  5990 consensus_peers.cc:597] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:24:53.171291  5990 consensus_peers.cc:597] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:53.178877  6614 tablet_bootstrap.cc:492] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:24:53.180052  5990 consensus_peers.cc:597] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:53.189463  6614 tablet_bootstrap.cc:492] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:53.190220  6614 ts_tablet_manager.cc:1403] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.315s	user 0.029s	sys 0.031s
I20260709 12:24:53.194357  6614 raft_consensus.cc:359] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:53.195006  6614 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.195719  6614 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:53.199942  6614 ts_tablet_manager.cc:1434] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:24:53.201992  6614 tablet_bootstrap.cc:492] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:53.242043  6614 tablet_bootstrap.cc:492] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.243044  6614 tablet_bootstrap.cc:492] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:53.243772  6614 ts_tablet_manager.cc:1403] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.042s	user 0.020s	sys 0.007s
I20260709 12:24:53.246685  6614 raft_consensus.cc:359] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:53.247494  6614 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.248245  6614 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:53.250634  6614 ts_tablet_manager.cc:1434] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:24:53.252893  6614 tablet_bootstrap.cc:492] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:53.273986  6614 tablet_bootstrap.cc:492] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.275198  6614 tablet_bootstrap.cc:492] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
W20260709 12:24:53.273396  5990 consensus_peers.cc:597] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:53.275959  6614 ts_tablet_manager.cc:1403] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.001s
I20260709 12:24:53.278991  6614 raft_consensus.cc:359] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:53.279700  6614 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.280378  6614 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:24:53.282823  5990 consensus_peers.cc:597] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:24:53.283977  6614 ts_tablet_manager.cc:1434] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:24:53.285259  6614 tablet_bootstrap.cc:492] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
W20260709 12:24:53.299618  5840 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:24:53.319382  6614 tablet_bootstrap.cc:492] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.320576  6614 tablet_bootstrap.cc:492] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:53.321348  6614 ts_tablet_manager.cc:1403] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.036s	user 0.020s	sys 0.003s
I20260709 12:24:53.324343  6614 raft_consensus.cc:359] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:53.326526  6614 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.327487  6614 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:53.330302  6614 ts_tablet_manager.cc:1434] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:24:53.332265  6614 tablet_bootstrap.cc:492] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:53.354813  6614 tablet_bootstrap.cc:492] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.356141  6614 tablet_bootstrap.cc:492] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:53.357020  6614 ts_tablet_manager.cc:1403] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.005s
I20260709 12:24:53.360078  6614 raft_consensus.cc:359] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:53.360842  6614 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.361785  6614 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:53.363876  6614 ts_tablet_manager.cc:1434] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:24:53.366048  6614 tablet_bootstrap.cc:492] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:53.383999  6614 tablet_bootstrap.cc:492] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.385288  6614 tablet_bootstrap.cc:492] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:53.386091  6614 ts_tablet_manager.cc:1403] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.008s
I20260709 12:24:53.388926  6614 raft_consensus.cc:359] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:53.389693  6614 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.390452  6614 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:53.392228  6614 ts_tablet_manager.cc:1434] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:24:53.393275  6614 tablet_bootstrap.cc:492] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:53.424741  6614 tablet_bootstrap.cc:492] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.425876  6614 tablet_bootstrap.cc:492] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:53.426501  6614 ts_tablet_manager.cc:1403] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.018s
I20260709 12:24:53.428872  6614 raft_consensus.cc:359] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:53.429433  6614 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.430064  6614 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:53.431865  6614 ts_tablet_manager.cc:1434] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:24:53.432929  6614 tablet_bootstrap.cc:492] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:53.459468  6614 tablet_bootstrap.cc:492] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.460624  6614 tablet_bootstrap.cc:492] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:53.461423  6614 ts_tablet_manager.cc:1403] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.005s
I20260709 12:24:53.464426  6614 raft_consensus.cc:359] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:53.465260  6614 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.466308  6614 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:53.468448  6614 ts_tablet_manager.cc:1434] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:24:53.469362  6614 tablet_bootstrap.cc:492] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:53.507532  6762 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:53.516044  6614 tablet_bootstrap.cc:492] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.517925  6614 tablet_bootstrap.cc:492] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:53.518702  6614 ts_tablet_manager.cc:1403] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.050s	user 0.015s	sys 0.015s
I20260709 12:24:53.519697  6773 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:53.521948  6614 raft_consensus.cc:359] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:53.524502  6614 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.525329  6614 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:53.528865  6614 ts_tablet_manager.cc:1434] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:24:53.530840  6614 tablet_bootstrap.cc:492] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:53.549142  6793 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:53.563248  6614 tablet_bootstrap.cc:492] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.571776  6614 tablet_bootstrap.cc:492] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:53.572657  6614 ts_tablet_manager.cc:1403] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.043s	user 0.012s	sys 0.007s
I20260709 12:24:53.575682  6614 raft_consensus.cc:359] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:53.576402  6614 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.577144  6614 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:53.587657  6614 ts_tablet_manager.cc:1434] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:24:53.588944  6614 tablet_bootstrap.cc:492] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
W20260709 12:24:53.642001  6895 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:24:53.687319  6654 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.688081  6766 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:53.688691  6654 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:53.701083  6654 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:53.711046  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:24:53.711906  5942 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:24:53.719144  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:53.722400  6878 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:53.722982  6878 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.723606  6878 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:53.726065  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:24:53.727131  6762 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:24:53.742753  6878 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:53.747392  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:24:53.748122  5942 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:53.752372  6614 tablet_bootstrap.cc:492] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.772847  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:24:53.751359  6762 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:53.779665  6614 tablet_bootstrap.cc:492] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:53.777878  6773 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 4
I20260709 12:24:53.780557  6614 ts_tablet_manager.cc:1403] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.192s	user 0.035s	sys 0.027s
I20260709 12:24:53.783515  6614 raft_consensus.cc:359] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:53.784312  6614 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.785174  6614 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:53.778486  5942 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:24:53.754190  6878 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:24:53.787547  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:24:53.788589  6614 ts_tablet_manager.cc:1434] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:24:53.788549  6878 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:53.789227  6878 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:24:53.789757  6614 tablet_bootstrap.cc:492] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:53.790496  6878 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:53.862562  6773 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:24:53.864207  5693 catalog_manager.cc:5697] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 4, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 4 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:53.883797  6614 tablet_bootstrap.cc:492] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.888417  6614 tablet_bootstrap.cc:492] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:53.890240  6614 ts_tablet_manager.cc:1403] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.101s	user 0.024s	sys 0.026s
I20260709 12:24:53.899444  6614 raft_consensus.cc:359] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:53.900552  6614 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.901505  6614 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:53.904223  6614 ts_tablet_manager.cc:1434] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:24:53.907742  6614 tablet_bootstrap.cc:492] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
W20260709 12:24:53.927307  5990 consensus_peers.cc:597] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:24:53.930851  6614 tablet_bootstrap.cc:492] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.932096  6614 tablet_bootstrap.cc:492] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:53.932898  6614 ts_tablet_manager.cc:1403] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.006s
I20260709 12:24:53.935856  6614 raft_consensus.cc:359] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:53.936604  6614 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.937305  6614 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:24:53.939594  6614 ts_tablet_manager.cc:1434] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:24:53.940455  6614 tablet_bootstrap.cc:492] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:53.953862  6766 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:53.966766  6614 tablet_bootstrap.cc:492] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.968050  6614 tablet_bootstrap.cc:492] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:53.968850  6614 ts_tablet_manager.cc:1403] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.029s	user 0.023s	sys 0.000s
I20260709 12:24:53.971632  6614 raft_consensus.cc:359] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:53.972366  6614 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:53.973165  6614 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:24:53.975526  6614 ts_tablet_manager.cc:1434] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260709 12:24:53.976670  6614 tablet_bootstrap.cc:492] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:53.993947  6614 tablet_bootstrap.cc:492] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.995111  6614 tablet_bootstrap.cc:492] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:53.995921  6614 ts_tablet_manager.cc:1403] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 12:24:53.999130  6614 raft_consensus.cc:359] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:53.999823  6614 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.000541  6614 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:54.002633  6614 ts_tablet_manager.cc:1434] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:24:54.003515  6614 tablet_bootstrap.cc:492] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:54.032068  6614 tablet_bootstrap.cc:492] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:54.032886  6614 tablet_bootstrap.cc:492] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:54.033490  6614 ts_tablet_manager.cc:1403] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.011s
I20260709 12:24:54.036098  6614 raft_consensus.cc:359] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:54.036772  6614 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.037470  6614 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:54.039742  6614 ts_tablet_manager.cc:1434] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:24:54.040601  6614 tablet_bootstrap.cc:492] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Bootstrap starting.
I20260709 12:24:54.061751  6614 tablet_bootstrap.cc:492] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:54.063546  6614 tablet_bootstrap.cc:492] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Bootstrap complete.
I20260709 12:24:54.064232  6614 ts_tablet_manager.cc:1403] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.004s
I20260709 12:24:54.066789  6614 raft_consensus.cc:359] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:54.067495  6614 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Initialized, Role: FOLLOWER
I20260709 12:24:54.068089  6614 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:24:54.069999  6614 ts_tablet_manager.cc:1434] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:24:54.173156  6762 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:54.194150  6766 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:54.263227  6766 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:24:54.264776  6591 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:54.276873  5944 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:24:54.280323  6880 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:01.965749  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:02.735864  5838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.004s	sys 0.002s
W20260709 12:25:02.736459  5838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.004s	sys 0.002s
I20260709 12:25:02.819788  6948 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:02.821122  6948 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:02.826542  6948 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:02.848650  5728 debug-util.cc:398] Leaking SignalData structure 0x7b0800394f20 after lost signal to thread 2070
W20260709 12:25:02.850157  5728 debug-util.cc:398] Leaking SignalData structure 0x7b0800388ba0 after lost signal to thread 5731
I20260709 12:25:02.870458  6948 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:02.871326  6948 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:02.873773  6948 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:02.885902  6948 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:02.886487  6948 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:02.888082  6951 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:02.889825  6951 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:02.888643  6948 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:02.898267  6951 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:02.943909  6952 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:02.950080  6925 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:02.957952  6925 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:02.960498  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.803s	user 0.003s	sys 0.009s
W20260709 12:25:02.965240  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.006s	sys 0.010s
I20260709 12:25:02.966878  6959 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:02.957767  6952 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:02.968446  6960 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:02.955350  6944 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:02.979861  6944 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:02.981988  6943 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:02.982957  6943 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:02.955953  6956 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:02.977046  6925 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:02.987948  5728 debug-util.cc:398] Leaking SignalData structure 0x7b0800157d80 after lost signal to thread 5899
I20260709 12:25:02.989210  6952 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:02.991254  6943 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:02.989492  5728 debug-util.cc:398] Leaking SignalData structure 0x7b080011b760 after lost signal to thread 5973
I20260709 12:25:02.995620  6932 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:02.996706  6932 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:02.998544  5728 debug-util.cc:398] Leaking SignalData structure 0x7b08002416c0 after lost signal to thread 6829
I20260709 12:25:03.004658  6959 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.008436  6959 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:03.009106  6959 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.020222  6925 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.019845  6960 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.020876  6925 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.019232  6943 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.027624  6959 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.026119  6956 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:02.983345  6944 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:03.037775  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 49538.1us
I20260709 12:25:03.044987  6946 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.045583  6946 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.045984  6953 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.046561  6953 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.049834  6951 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.050824  6951 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.052111  6952 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.052954  6952 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.058475  6954 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.059679  6954 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.075783  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.077159  6944 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.078454  6944 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.097218  6949 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.098248  6949 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.029503  6943 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.119261  6925 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.131882  6960 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.161245  6925 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.140311  6951 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.168889  6925 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.142145  6959 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.166934  6960 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:25:03.214022  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 76755.7us
I20260709 12:25:03.204993  6952 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.224749  6925 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.208654  6954 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.208384  6961 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.210265  6953 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.213439  6959 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.220196  6956 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.229249  6955 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.231827  6943 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.232820  6944 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.232564  6955 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.219960  6960 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.235050  6961 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.218726  6939 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.235929  6955 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:03.236694  6959 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.237116  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.230692  6965 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.237308  6961 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.238885  6965 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.236415  6939 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.230762  6966 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.246781  6956 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.285599  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.282024  6976 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.245849  6959 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.231503  6972 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.271032  6954 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.271139  6961 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.226533  6952 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.231073  6958 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.314967  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.269941  6955 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.274600  6943 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.340894  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.342109  6925 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.338778  6944 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.347434  6961 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.367630  6974 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.368317  6974 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.369374  6961 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:03.337393  6967 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.347461  6949 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.370074  6967 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.367638  6944 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.372408  6967 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.373492  6944 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.349038  6954 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.375187  6932 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.376142  6954 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.350405  6973 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.351790  6946 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.366978  6980 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.377532  6980 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.378599  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.379092  6946 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.379760  6946 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.380052  6980 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.361951  6976 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.380295  6959 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.376861  6973 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.348583  6952 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.381951  6961 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.382545  6961 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.383103  6973 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.383484  6952 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.308643  6966 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.384745  6961 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.362309  6956 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.385782  6966 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:03.385931  6946 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.380190  6974 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.386868  6956 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:03.387888  6974 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.388008  6966 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.388561  6974 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.388631  6966 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.390974  6974 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.391410  6966 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.391531  6974 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.392004  6966 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.392266  6974 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.392577  6966 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.350790  6958 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.395071  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.395488  6958 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:03.363354  6968 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.395903  5869 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:25:03.396296  6968 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.363473  6960 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.398720  6976 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.363976  6965 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:03.364240  6970 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.364609  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.399886  6970 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.400177  6766 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:25:03.400833  6966 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.366024  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.401876  6793 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:25:03.337617  6953 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.366957  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.373198  6972 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.403721  6953 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.403873  6943 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.404493  6972 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.375778  6925 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.406263  6943 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.381199  6959 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.406639  6925 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.402392  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:03.352192  6936 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.407825  6936 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.408324  6959 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.409021  6980 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:03.383673  6973 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.366233  6957 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.410230  6936 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:03.410032  6957 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.410854  6951 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.402348  6970 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.409438  6980 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.411455  6951 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.411995  6980 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.412266  6957 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.413089  6973 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.413715  6951 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.363086  6969 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.406376  6953 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.426220  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.436954  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.416486  6955 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.440717  6969 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.445569  6957 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.446204  6957 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.446728  6978 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.447384  6978 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.448187  6969 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.448412  6957 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:03.440804  6944 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.440750  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.449467  6939 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.449277  6944 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.451344  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.452931  6944 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.453040  6968 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.416569  6949 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.454267  6949 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.455570  6982 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.456401  6949 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.456249  6982 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.456884  6978 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:03.460858  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.461572  5867 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:03.462380  6932 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.465340  6959 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.458643  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.466480  6959 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.464828  6966 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.467875  6966 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.468576  6982 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.469101  6959 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.466454  6932 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.469993  6966 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.470315  6962 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.470981  6962 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.471465  6932 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.472705  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.473783  6982 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.474232  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.475178  6016 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:25:03.474373  6982 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.476169  6962 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.477177  6961 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.477813  6961 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.479974  6961 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.480479  6961 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.481056  6986 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.481215  6956 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.481940  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.482156  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.482419  6956 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.477979  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.478377  6962 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.483824  6973 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:03.458832  6955 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.481029  6961 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.484818  6956 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.484732  6962 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.486289  6961 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.488044  6962 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.488390  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.489019  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.489519  5944 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:25:03.489693  5868 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:03.490864  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.491714  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.487984  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.492317  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.492915  5870 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:25:03.493091  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.493762  5874 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:03.494206  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.494840  5913 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:03.497764  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.498620  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.504192  5867 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:03.499219  6932 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:03.503407  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.499481  6957 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.505378  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.506104  6957 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.508045  6756 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:03.508381  6957 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.499940  6966 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.509233  6966 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.511063  6982 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.511575  6966 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.499854  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.500509  6974 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.505456  6986 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.514135  6985 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.514107  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.515699  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.517411  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.505525  6932 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.518735  6932 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.520462  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.520591  6988 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.521868  6961 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.521872  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.522511  6961 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.522881  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.523341  6756 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:25:03.523547  6766 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:03.524390  6984 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.524801  6961 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.525179  6982 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.525836  6982 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.526337  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.522058  6988 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.529920  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.533517  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.525082  6984 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.542126  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.542924  6016 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:25:03.543429  6984 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.540997  6988 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.515443  6986 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.546006  6984 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.546543  6984 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.548341  6984 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:03.549209  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.552062  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.552799  6762 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:03.553206  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.553697  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.555370  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.557029  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.551000  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.559398  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.545403  6932 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.560081  5873 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:03.560544  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.560981  5942 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:03.562165  6932 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.563438  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:03.563510  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:03.564427  5873 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.565981  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.563701  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.566838  6756 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:03.567203  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.567890  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.566973  5867 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:25:03.568809  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.569672  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.570389  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.572221  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.572657  6986 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:03.573153  6986 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.573616  6986 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:03.574115  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.574839  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.575721  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.576337  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.576393  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.578171  5940 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:25:03.579242  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.579332  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.528502  6982 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:03.514981  6985 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.582707  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.583508  5873 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:03.583738  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.583194  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.588349  5941 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:03.588651  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.584458  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.584161  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.589780  5942 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:03.584604  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.583272  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.590747  5947 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:03.592435  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.586851  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.597141  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.597967  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.599007  6773 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:25:03.599216  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.601845  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.603003  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.605186  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.606519  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.607208  6985 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:03.607080  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.608105  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.609299  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.609357  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.611716  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.611958  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.613180  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:03.614146  6951 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:03.614295  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:03.614609  6951 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.615046  6951 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.616263  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:03.619946  6984 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.620591  6984 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.621335  6980 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.622115  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.623013  5943 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:03.623122  6984 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.624238  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.624480  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.624928  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.625399  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.626163  6020 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:03.626477  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.627805  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.628793  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.628715  6980 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:03.630743  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.631076  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.631091  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.631716  6022 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:03.631490  6016 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:03.631458  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.633505  6986 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.634685  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.635317  5913 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:03.635466  6756 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:03.635710  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:03.636445  5873 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.637362  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:03.637830  6980 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:03.638454  6980 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:03.640295  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.643006  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.643860  6017 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:03.639709  6980 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.644513  5873 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:03.645998  6966 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:03.646538  6966 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.646706  6984 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.647060  6966 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:03.647434  6984 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.648869  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.650490  6984 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.655094  6986 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.656687  6951 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.658923  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.659308  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.659647  6766 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:03.659914  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.660854  6756 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:03.661180  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.662595  6017 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:03.662988  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.661715  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.663843  6951 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.664481  6768 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:25:03.660166  6773 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:25:03.666143  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.668475  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:03.670039  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.676507  6925 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:03.677026  6925 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.677368  6925 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.681988  6943 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:25:03.681905  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 65715.2us
I20260709 12:25:03.682741  6943 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.683436  6943 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.682552  5693 catalog_manager.cc:5697] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 1 to 2, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 2 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:03.685858  6986 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:03.686199  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:03.695713  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 77706.2us
I20260709 12:25:03.686537  6986 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.697903  6986 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:03.670387  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:03.702562  6980 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:03.689421  6957 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:03.703142  6980 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:03.704209  6980 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.690119  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.691556  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.685808  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:03.714078  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:03.715734  6962 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:03.717995  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:03.719106  6962 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:03.719537  6962 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.719913  6962 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.722682  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.724696  6961 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:03.725219  6961 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.725663  6961 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:03.737175  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51464.7us
I20260709 12:25:03.738070  5915 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:03.739288  6979 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:03.740161  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.740633  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.741173  5915 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:03.742508  6979 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:03.743432  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.722141  6966 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:03.747534  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 57842.7us
I20260709 12:25:03.747195  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:03.748553  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 50813.2us
I20260709 12:25:03.749727  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:03.751328  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:03.751654  6984 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:03.752444  6984 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:03.752897  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.753024  6985 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:03.753813  6985 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:25:03.753140  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 51985.7us
I20260709 12:25:03.754387  6985 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:03.755569  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.756439  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.756705  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.757184  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.758150  6017 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:03.758505  6957 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:03.759152  6957 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:03.760392  6957 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.763231  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.752924  6984 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:03.764053  6773 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:03.767407  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.767112  6957 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.771519  6943 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.775343  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.775744  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.683266  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:03.746948  6925 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.780198  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:03.736070  6961 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.792004  5915 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:03.794776  5990 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:03.796515  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:03.797129  5915 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:03.798508  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.798959  6925 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:03.800455  6948 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:03.800921  6948 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.801072  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.801455  6965 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:03.801663  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.800285  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.799609  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:03.807168  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.807662  6968 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:03.808195  6968 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.808426  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:03.801522  6948 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:03.808939  6976 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:03.809454  6976 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.809903  6976 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:03.813265  6936 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:03.808784  6961 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.815294  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:03.816852  6966 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.796756  6979 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:03.794785  6932 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:03.804062  6980 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:03.822041  6980 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:03.822510  6980 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.822961  6980 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:03.819988  6943 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.823311  6989 raft_consensus.cc:3055] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:03.823871  6989 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:03.827142  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.827421  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.828608  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:03.830067  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:03.830859  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:03.832448  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:03.833343  6961 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:03.833822  6961 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:03.833925  6966 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:03.834291  6961 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.836513  6985 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.838177  6022 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 3: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:03.841073  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:03.841832  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:03.841832  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:03.849417  6932 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:03.850059  6932 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.850469  6932 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:03.851769  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:03.852713  6979 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:03.853117  6979 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.853128  6762 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:03.853601  6979 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.854665  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:03.860091  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:03.867076  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:03.866896  6965 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:03.867859  5867 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:03.808681  6968 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:03.872428  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:03.876055  6985 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.869036  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:03.877952  5873 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 1. Current term is 2. Ops: []
W20260709 12:25:03.878391  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 195225us
I20260709 12:25:03.824920  6989 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:03.799546  6962 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.881985  6970 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:03.891433  6970 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.891800  6970 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.882143  6022 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:03.890565  5987 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:03.898448  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.899211  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:03.897699  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:03.898200  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.927510  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.881657  6986 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.895853  6969 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:25:03.897454  6965 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:03.895242  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:03.926709  5946 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:03.868228  6980 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:03.934525  6022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886"
dest_uuid: "6add331326ca47038308a447bf798fa7"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:48972
I20260709 12:25:03.935554  6022 raft_consensus.cc:606] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:03.901739  6988 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:03.877401  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:03.939970  6925 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:03.940639  6925 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:03.941071  6925 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:03.941588  6969 raft_consensus.cc:3055] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:03.942296  6969 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:03.942672  6966 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:03.945286  6793 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
W20260709 12:25:03.939877  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 44743us
I20260709 12:25:03.926548  6989 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.902388  6961 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:03.956835  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.959276  6986 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:03.964886  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:03.968452  6925 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.970417  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:03.976804  6925 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:03.972327  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:03.947846  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 86655.1us
I20260709 12:25:03.980463  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:03.981420  6966 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:03.982600  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:03.991065  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:03.948213  6969 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:03.994936  5867 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:03.998159  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:04.000449  6762 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:04.004761  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:03.967144  6972 raft_consensus.cc:3055] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:04.018548  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:04.023725  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:04.025300  5873 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:04.008083  6972 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:03.982857  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.038590  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:04.005786  6957 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:25:04.040949  6969 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:04.062013  6962 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:04.063081  6972 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:04.064551  6972 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.057319  6966 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.066107  6980 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:04.068125  6968 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.074913  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:04.089926  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:04.091006  6932 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.090059  6936 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:04.093502  6936 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:04.094591  6936 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:04.095973  6936 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.097741  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:04.082782  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 100013us
I20260709 12:25:04.099509  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:04.099958  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:04.100549  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.100922  6021 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:04.102010  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:04.102252  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.098022  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:04.103968  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:04.104445  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:04.103150  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.105553  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:04.106654  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.108278  6970 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.108630  6984 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.109843  6020 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:04.110692  6961 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:04.111152  6948 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:04.111958  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:04.098891  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:04.113497  6022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "428a50885681465a98089cb853b233b9"
dest_uuid: "6add331326ca47038308a447bf798fa7"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:48972
I20260709 12:25:04.114293  6022 raft_consensus.cc:606] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:04.096931  6976 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:04.112771  6979 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.120811  6961 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:04.123826  6932 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:04.121480  6961 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:04.127117  6961 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:04.126938  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.126513  6979 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:04.129891  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.132467  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:04.134533  6611 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:04.136010  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.140367  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:04.130676  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.144037  5867 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:04.146865  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:04.138911  6966 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:25:04.151435  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 248542us
I20260709 12:25:04.137284  6986 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:04.153177  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.155519  6785 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:04.156867  5868 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:04.142280  6948 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:04.152333  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:04.152068  6966 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:04.160435  6982 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:04.160567  6966 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.165359  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:04.166042  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:04.166414  5866 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:04.165410  5870 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:04.167202  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:04.168936  5868 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:04.170464  5870 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:04.170832  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:04.171403  6984 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:04.171000  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:04.172458  5867 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:04.173885  6968 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:04.174499  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:04.175021  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:04.161283  6982 consensus_queue.cc:1059] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:25:04.178282  6970 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:04.178313  6016 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:04.178508  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:04.180796  6986 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:04.181787  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:04.181849  6986 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:04.187847  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:04.188133  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:04.186247  6986 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.191617  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:04.183876  6997 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:25:04.198575  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:04.202080  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:04.205437  6995 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:04.206032  6995 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:04.206732  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.206974  6995 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:04.208343  6995 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.199648  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.212029  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.212270  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.214807  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.214788  5949 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:04.214788  5946 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:04.220608  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.225375  6966 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.226977  6947 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:04.227583  6947 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:04.227964  6947 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:04.232607  5943 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:04.233407  5944 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:04.236728  6016 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:04.238168  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:04.239812  5946 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:04.238731  6976 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:04.244020  6961 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.228500  6966 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:04.249078  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 114628us
I20260709 12:25:04.240607  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.250761  6611 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.253480  6611 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.275180  6932 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:04.277681  6966 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.278640  6966 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:04.279007  6966 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:04.279361  6966 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.254065  6981 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:04.279946  6981 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:04.280385  6981 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:04.247679  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 120821us
I20260709 12:25:04.282486  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:04.283335  6996 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:04.252143  6986 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:04.289095  6996 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:04.284073  6982 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:04.289961  6775 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:04.290364  6982 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:04.290786  6982 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:04.291026  6986 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:04.284821  7001 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:04.291806  7001 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:04.292300  7001 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.291877  6982 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.293911  6609 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:04.289479  6996 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:04.295070  6609 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:04.295724  6984 raft_consensus.cc:3055] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:04.296005  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:04.255000  6989 consensus_queue.cc:1059] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:25:04.285306  6961 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:04.297561  6986 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:04.297762  6961 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:04.297968  7002 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:04.298207  6961 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:04.298569  7002 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:04.298770  6961 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:04.299077  7002 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.287428  6756 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:04.295310  6996 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.285372  6975 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:25:04.300992  6996 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:04.293517  6982 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.302687  6985 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:04.303241  6989 raft_consensus.cc:3055] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:04.296196  6984 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:04.303735  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.304155  6972 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:04.304189  6988 raft_consensus.cc:3055] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:04.285287  5915 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:04.306028  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:04.311338  6975 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:04.327188  6975 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:04.313796  6976 consensus_queue.cc:1059] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:04.319294  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.322914  6994 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.331233  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:04.323750  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:04.324682  6939 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.330092  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:04.332700  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:04.333577  6948 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:04.334154  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:04.335376  6949 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.336550  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.337565  6943 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.337680  6936 raft_consensus.cc:3055] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:04.338231  6936 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:04.338979  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:04.328620  6988 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:04.339144  6936 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.340391  6946 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:04.340780  6936 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.310555  6989 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:04.342372  6976 raft_consensus.cc:3055] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:04.342834  6976 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:04.342958  6989 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.343746  6976 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.344193  6965 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:04.344684  6997 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:04.345490  6976 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:04.345777  6997 raft_consensus.cc:3055] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:04.341276  6946 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.346163  6997 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:04.347179  6997 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.327726  6975 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:04.344708  6989 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:04.350666  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 97214.6us
I20260709 12:25:04.351940  6611 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.352371  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:04.353402  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:04.347122  5915 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:04.354849  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:04.354892  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:04.357121  6968 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.357457  6932 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.358338  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:04.359102  6932 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:04.358564  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:04.360481  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:04.298036  6986 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:04.362864  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:04.362951  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:04.370574  5867 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 3: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:04.364987  6994 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:04.340390  6988 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:04.366240  6968 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:04.366559  6984 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.378774  6984 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:04.348965  6997 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.366979  6975 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:04.368739  6944 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:04.386482  6944 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:04.387003  6985 raft_consensus.cc:993] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: : Instructing follower 1fc57407d56241b38a94d543e073f8af to start an election
I20260709 12:25:04.387512  6985 raft_consensus.cc:1081] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Signalling peer 1fc57407d56241b38a94d543e073f8af to start an election
I20260709 12:25:04.387429  6944 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.388923  6944 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.367144  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.384881  5915 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:04.385399  7004 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.374753  6986 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.392552  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:04.388999  6966 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.408332  7004 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:04.389227  6947 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:04.397660  6988 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.394320  6969 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.408874  7004 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:04.376051  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.404241  6943 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:04.413826  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.392917  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:04.418402  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:04.410882  6975 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:04.422235  6932 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:04.395293  7002 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.397922  6756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "428a50885681465a98089cb853b233b9"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:43762
I20260709 12:25:04.427081  6756 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:04.427515  6756 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:04.428012  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.416438  7004 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.434073  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.434847  6016 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 3: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:04.435588  6945 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.420495  6979 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.437372  6932 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:04.441556  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:04.444804  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:04.446511  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.427893  7001 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.451913  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.455327  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.452697  6785 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:04.459259  6961 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.445138  6953 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:25:04.438273  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 75312.1us
I20260709 12:25:04.445101  6969 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:04.462028  6775 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:04.463778  6969 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:04.467036  6947 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:04.456918  6972 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:04.468174  6609 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:04.464475  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:04.451399  6943 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.461000  6995 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.472932  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.467199  6969 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.479794  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:04.481297  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:04.463982  6966 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:04.483201  6966 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.483970  6966 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:04.465572  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:04.453976  6957 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.474807  6981 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:04.477680  7002 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:04.488143  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 136279us
I20260709 12:25:04.488548  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.490101  6611 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.473027  6985 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.488963  6984 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:04.493680  6611 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.491132  6961 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:04.470297  6976 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:25:04.496622  5874 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:04.497284  6925 consensus_queue.cc:1059] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:04.484326  6966 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:04.475600  6965 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 12:25:04.489609  6945 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:04.486060  6972 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.497567  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.501338  6611 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.493914  6949 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.512813  6961 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:04.513062  7001 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:04.515394  6945 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.499529  7002 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.503114  6966 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.534499  6969 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.506516  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.537748  6936 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:04.537796  7002 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.538479  6995 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:04.538957  6936 raft_consensus.cc:3055] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:04.539350  6936 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:04.539640  6995 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:04.540023  6995 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:04.539893  6611 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.540939  6985 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:04.541354  6985 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:04.542227  6985 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:04.543488  6985 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.544569  7002 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:04.545207  7002 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:04.545729  7002 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.546675  6981 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:04.527297  6965 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:04.553188  6611 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.549980  6953 consensus_queue.cc:1059] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:04.558108  6965 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:04.562157  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:04.563135  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 65657.8us
I20260709 12:25:04.559546  6611 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:04.562183  6961 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.569705  6999 raft_consensus.cc:3055] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:04.570448  6999 raft_consensus.cc:740] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:04.571774  6999 consensus_queue.cc:260] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:04.582219  6965 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:04.588426  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.591008  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.541181  6995 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.592480  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:04.595769  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.598887  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.595024  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.607374  6762 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:04.611982  6995 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.600150  7001 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:04.615320  7001 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:04.616240  7001 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:04.617859  6999 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:04.624347  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.625321  6023 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.627296  6016 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:04.618250  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:04.540253  6936 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.621541  6965 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:04.633381  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:04.638767  6611 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:04.640550  7008 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:04.641106  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:04.642797  6951 raft_consensus.cc:3055] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:04.644251  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:04.648001  6936 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:04.652901  6948 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.659353  6951 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:04.665253  6981 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:25:04.670491  6981 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:04.670917  6981 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:04.671489  6986 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.671890  6981 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:04.673444  6981 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:04.676541  7002 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.690761  7002 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:04.677975  6961 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:04.660332  7006 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:04.677918  6966 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:04.662874  7008 raft_consensus.cc:697] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:04.695075  7008 consensus_queue.cc:237] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.695598  6966 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:04.695885  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.699973  6986 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:04.697160  7004 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:04.673477  6756 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:04.708392  7004 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:04.708392  6756 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:04.713111  6023 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:04.715642  6016 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 2. Current term is 3. Ops: []
W20260709 12:25:04.633347  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 165283us
W20260709 12:25:04.724422  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 85688.9us
I20260709 12:25:04.763188  6609 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:04.767643  7004 raft_consensus.cc:2764] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:25:04.769547  6609 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:05.120913  6932 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.121714  6932 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:05.222419  6932 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:05.258039  7001 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.074s	sys 0.107s
W20260709 12:25:05.260093  7001 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.074s	sys 0.108s
W20260709 12:25:05.301188  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.327970  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:05.328976  6947 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:05.330243  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:05.337496  6947 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:05.340842  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:05.341728  5868 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:05.344856  5913 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:05.347301  7019 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:05.347779  7019 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.348634  7018 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:05.349278  7018 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:05.352736  7018 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:05.348137  7019 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:05.363689  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:05.366061  6021 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 3: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:05.364552  7019 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:05.368541  7019 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:05.370426  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:05.371116  5868 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:05.377287  5987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.892s	user 0.054s	sys 0.090s
W20260709 12:25:05.381920  5987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.897s	user 0.057s	sys 0.090s
W20260709 12:25:05.382611  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 901301us
I20260709 12:25:05.383383  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:05.384198  7018 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:05.385712  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:05.386492  7018 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:05.387907  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:05.380796  5868 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:05.389689  5913 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:05.390566  7019 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:05.391124  7019 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:05.392088  7019 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:05.396402  7018 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:05.399397  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:05.400336  7018 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:05.402350  5987 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:05.403221  7018 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:25:05.438123  6024 consensus_meta.cc:287] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.731s	user 0.011s	sys 0.009s
I20260709 12:25:05.438884  6024 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:05.448459  7018 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:05.449282  7018 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:05.451763  6015 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:05.456223  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:05.457846  7018 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:05.467770  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:05.460562  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:05.470188  5946 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:05.463953  7023 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:05.471692  7023 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:05.491062  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:05.491372  7023 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:05.498137  7018 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:05.498741  7018 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:05.499133  7018 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:05.509944  7023 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.510634  7023 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:05.512873  7023 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:05.519476  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 750006us
I20260709 12:25:05.538937  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:05.524416  7023 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:05.525419  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:05.526144  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:05.532263  7001 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:05.532919  7008 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:25:05.543553  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.548622  7008 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:05.546098  7023 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:05.552497  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:05.553030  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.554399  7023 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:05.534844  7013 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:05.557976  7013 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:05.560504  7013 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:05.525418  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:05.521373  7004 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:05.556988  7001 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:05.564071  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:05.541815  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:05.546286  5949 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:05.564765  7001 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:05.566879  5987 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:05.545524  6951 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:05.567831  6951 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:05.565188  7008 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:05.565968  7023 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:05.571411  7023 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:05.569926  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:05.572090  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:05.574224  7008 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:05.575320  7008 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:05.576699  7008 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:05.579118  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:05.580093  7013 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:05.572202  7023 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.581338  7023 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:05.581593  7001 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:05.542055  7018 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:05.584288  6019 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 2: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:05.588260  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:05.591619  7018 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:05.592371  7018 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:05.596822  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:05.597012  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:05.597728  6018 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:05.599118  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:05.599887  7005 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.956s	user 0.006s	sys 0.006s
I20260709 12:25:05.600689  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:05.592904  7018 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:05.604797  7018 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:05.605243  7025 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.606016  7025 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:05.606551  5915 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 1.127s	user 0.020s	sys 0.016s
W20260709 12:25:05.606981  5915 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 1.127s	user 0.020s	sys 0.016s
I20260709 12:25:05.611342  6756 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:05.612650  6762 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:05.615473  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:05.621038  6756 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:05.623438  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:05.623818  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:05.625658  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:05.630586  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:05.631349  6024 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:05.631623  6024 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:05.635565  6024 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:05.635860  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:05.636606  6024 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:05.636763  6017 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:05.636938  7001 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:05.614292  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:05.642532  6953 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:05.643412  6953 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:05.644076  6953 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:05.645238  6024 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:05.667639  6017 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:05.541663  7024 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.676867  7024 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:05.675643  6024 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:25:05.676087  7018 consensus_queue.cc:1175] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7a0204b931394fc48ffc503716e0d111" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 12:25:05.674190  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:05.677363  6021 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:05.683765  6016 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:05.698160  7028 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.698982  7028 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:05.701448  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:05.701505  7027 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:05.702095  6023 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:05.704383  6611 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:05.712532  6024 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:05.707453  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:05.715770  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:05.716446  6773 raft_consensus.cc:2393] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 3: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:05.695869  5943 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:05.719599  6793 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 3: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:05.675861  7025 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:05.720335  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:05.718086  6611 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:05.723568  6762 raft_consensus.cc:2393] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 3: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:05.718957  7013 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:05.724236  6611 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:05.675935  5866 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:05.735423  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:05.738550  6611 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:05.729866  7004 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:05.742033  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:05.726076  7001 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:05.743260  7026 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:05.745676  7030 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:05.752848  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:05.745872  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:05.702394  7027 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:05.745713  6756 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:05.747162  7024 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:05.747848  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:05.758255  6979 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:05.755765  7001 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:05.748178  5867 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:05.762192  7030 raft_consensus.cc:697] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:05.763674  7030 consensus_queue.cc:237] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:05.754527  5874 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:05.754603  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:05.771185  6979 raft_consensus.cc:3055] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:05.771631  6979 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:05.772657  6979 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:05.774099  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:05.774931  6785 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
W20260709 12:25:05.775002  6015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 40) took 1176 ms (client timeout 1509 ms). Trace:
W20260709 12:25:05.775502  6015 rpcz_store.cc:269] 0709 12:25:04.598264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.598489 (+   225us) service_pool.cc:224] Handling call
0709 12:25:05.774781 (+1176292us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b54001720d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:05.774904 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1861920128}
I20260709 12:25:05.775043  7001 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:05.782137  7001 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:05.782693  7001 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:25:05.768290  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 142743us
W20260709 12:25:05.785652  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 65769.2us
I20260709 12:25:05.783619  7001 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:05.790411  6953 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:05.790851  7027 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:05.784032  7023 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:05.784271  7008 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:05.809275  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:05.810091  5871 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:05.813030  7023 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:05.813686  7008 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:05.805547  5874 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:05.799283  6953 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:25:05.793378  6979 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:05.819216  6999 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:05.819765  6999 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:05.820322  7008 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:05.816056  7013 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:05.822081  7013 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:05.822567  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:05.822731  6932 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:05.823268  6932 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:05.824204  6932 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:05.824903  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:05.826107  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:05.826432  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:05.793853  7024 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.827594  7024 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:05.830041  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:05.830756  7024 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:05.830979  5946 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 3: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:05.794986  7028 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:05.794863  6018 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:05.832146  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:05.808284  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:05.831039  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:05.833269  5873 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:05.833652  5944 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:05.818358  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:05.838698  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 553) took 1241 ms. Trace:
I20260709 12:25:05.839293  6019 rpcz_store.cc:276] 0709 12:25:04.597202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.611078 (+ 13876us) service_pool.cc:224] Handling call
0709 12:25:05.541577 (+930499us) spinlock_profiling.cc:243] Waited 9.61 ms on lock 0x7b5800480098. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:05.838409 (+296832us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b5400764654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:05.838609 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64247936}
I20260709 12:25:05.834841  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:05.844825  5990 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:05.845471  5946 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:05.845211  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:05.846179  7028 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:05.846769  6018 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:05.817093  7004 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:05.792263  5695 catalog_manager.cc:5697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 3, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 3 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:05.848143  5990 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:05.849936  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:05.853322  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:05.854828  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:05.855840  7026 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:05.856421  7023 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:05.856825  5946 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:05.857309  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:05.857511  7026 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:05.856851  7023 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.858946  7023 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:05.820838  6999 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:05.861651  5990 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:05.867592  5874 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:05.869253  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:05.865002  7024 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:05.864929  7018 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:05.881717  5943 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:05.863958  7028 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:05.889264  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 100606us
I20260709 12:25:05.869791  6999 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:05.870059  7008 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:05.889736  7024 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:05.914295  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 50793us
I20260709 12:25:05.928333  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:05.909286  7024 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:05.937000  5944 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:05.933408  5946 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:05.908955  7034 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:05.906293  7008 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:05.910236  6979 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:25:05.946834  7037 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:05.947086  6932 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:05.948473  6953 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:05.948675  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:05.912094  7030 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
W20260709 12:25:05.954610  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 61955us
I20260709 12:25:05.952164  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:05.929075  7040 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:05.960518  6762 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:25:05.942327  6945 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:05.933944  6949 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 12:25:05.965265  6981 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:05.966737  6981 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.966965  7041 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.967202  6981 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:05.967640  7041 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:05.927608  7004 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:05.969726  6768 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:05.943993  7038 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.962606  6953 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:05.950094  7035 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:05.987383  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:05.962810  7040 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.963056  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:05.970757  6949 raft_consensus.cc:3055] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:05.950749  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:05.983789  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:05.984867  7026 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:05.991183  7038 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.012375  7008 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:05.993561  7035 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:06.001680  6949 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:06.013614  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.015223  6785 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:05.994030  7040 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:06.017388  7039 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:06.018555  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 78753.9us
I20260709 12:25:05.946512  7034 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.019152  6953 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:06.010097  6932 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:25:06.053581  6932 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:06.023838  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:05.997107  6756 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:06.056797  6949 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.058228  6949 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:06.059796  6609 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:06.061496  7038 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:06.025919  7033 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:06.062659  7033 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:06.063053  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.063848  6775 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:06.064355  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:06.066116  7041 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:06.067881  7035 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:06.059952  7034 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:06.069790  6773 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 5: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:06.070570  7008 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:06.071537  6953 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:06.071272  7008 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.071966  6953 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:06.072942  6953 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.074143  7033 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:06.074846  7033 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:06.077152  7033 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:06.077987  7008 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:06.030406  7026 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:06.081427  7033 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:06.084064  5947 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:06.032255  6766 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:06.091354  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:05.960007  7037 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.054391  5946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570"
dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:43756
I20260709 12:25:06.092751  5946 raft_consensus.cc:606] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:06.088595  7040 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.093726  6762 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:25:06.094137  7037 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:06.084954  5694 catalog_manager.cc:5697] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 2, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 2 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:06.086911  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:06.087666  7008 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:06.087670  7023 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.087868  5693 catalog_manager.cc:5697] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 1 to 2, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 2 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:06.089255  7033 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:06.089804  7028 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:06.097605  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:06.113281  7018 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:06.100288  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:06.112917  5693 catalog_manager.cc:5697] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:06.116533  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 167936us
W20260709 12:25:06.117268  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:06.031595  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 293098us
I20260709 12:25:06.122197  6611 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:06.123286  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:06.124445  6611 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:06.132524  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 81282.6us
I20260709 12:25:06.136226  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.140383  6611 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:06.142976  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 202515us
I20260709 12:25:06.143954  5987 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:06.144788  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:06.110427  7043 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.081459  7026 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:06.114813  7030 raft_consensus.cc:3055] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:06.159644  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:06.161279  5947 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:06.159824  5867 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:06.119307  7050 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.142555  7001 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:06.175355  6762 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:06.176700  5947 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:06.176539  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:06.178234  5946 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:06.174346  7028 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:06.179417  7028 raft_consensus.cc:697] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:06.180290  7028 consensus_queue.cc:237] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.180893  6953 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:06.182541  7040 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:06.187287  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:06.155094  7004 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:06.194265  7024 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.194860  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:06.195233  7004 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:06.195744  6768 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:25:06.196282  7004 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.148557  7037 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:06.197571  7037 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:06.198582  7037 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.200011  7001 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:06.163070  7053 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.201187  7001 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.202224  7053 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.167903  6979 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.204669  7053 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:06.204571  6979 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.163962  7043 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.180645  7033 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.161046  7030 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:06.196789  7024 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:06.148272  7052 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.189158  6793 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:06.191965  7023 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:06.208596  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:06.212350  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:06.224606  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:06.225016  6766 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:06.212467  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:06.225416  5949 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 3: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:06.214671  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:06.215979  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.221993  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:06.142434  7013 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 12:25:06.230612  7031 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:06.231196  6979 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:06.231667  5874 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 3: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:06.228749  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:06.226457  7052 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:06.233225  6981 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:06.222761  7044 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.235555  7044 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:06.235970  6981 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:06.210073  6948 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.237584  7052 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:06.237429  7030 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.238189  6948 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.212749  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.174146  7038 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 12:25:06.229382  7035 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:06.178676  7050 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.229803  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:06.253163  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:06.238937  7030 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:06.229012  6756 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:06.239480  6773 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:25:06.245980  5873 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:06.245915  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:06.253528  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:06.201385  6609 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:06.269918  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:06.270567  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:06.271351  5867 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:06.273166  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.270833  5871 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:06.276481  7043 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:06.277217  7058 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.277366  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.277841  7058 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.278272  6785 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
W20260709 12:25:06.278972  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 77724.6us
I20260709 12:25:06.231462  6757 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:06.274889  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:06.281580  6018 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:06.262423  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:06.282856  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:06.287834  5987 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:06.288691  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.279815  6017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df"
dest_uuid: "6add331326ca47038308a447bf798fa7"
mode: GRACEFUL
new_leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
 from {username='slave'} at 127.0.0.1:48972
I20260709 12:25:06.290132  6017 raft_consensus.cc:606] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Received request to transfer leadership to TS 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:06.285118  7033 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:06.290719  6017 raft_consensus.cc:614] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:06.291266  7033 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.292847  6953 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:06.290059  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.310056  6018 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:06.313122  5990 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:06.322890  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:06.323764  5947 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:06.324368  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:06.289060  7013 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:06.286651  7052 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:06.279897  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:06.301375  7043 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:06.304216  7033 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:06.295338  6953 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.280233  7058 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:06.254904  7035 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:06.335901  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.336573  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:06.302091  7041 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.302328  6951 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:25:06.345899  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 66113us
I20260709 12:25:06.346554  7033 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:06.342913  7043 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.325086  7023 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:06.359634  7035 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:06.319998  6775 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:06.354139  5867 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:06.372670  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:06.376099  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 59999.8us
I20260709 12:25:06.347903  7044 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:06.382227  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 66455.6us
I20260709 12:25:06.390600  5943 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:06.390981  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:06.391980  5874 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:06.347736  6953 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:06.397419  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.368511  6948 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:06.356962  7052 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:06.390249  5871 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:06.382226  7043 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:06.415294  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:06.382560  7023 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.415699  7028 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:06.417284  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:06.370321  7041 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.417964  7032 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.418982  6768 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:06.419787  6757 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:06.422346  6018 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:06.370194  6951 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.390232  6999 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:25:06.425753  7023 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:06.441855  7028 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.442518  7028 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.433049  7060 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:06.444422  7060 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.444782  7060 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:06.445309  7044 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:06.445868  7052 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.437717  7032 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.447571  6979 raft_consensus.cc:3055] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:06.448028  6979 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:06.429692  7033 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:25:06.456727  7019 raft_consensus.cc:3055] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:06.461810  7019 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:06.462831  7019 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:06.463569  7041 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:06.464170  7019 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:06.466166  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.466950  6768 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:06.448997  6979 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:06.468323  5988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.016s	sys 0.018s
W20260709 12:25:06.468909  5988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.017s	sys 0.018s
I20260709 12:25:06.472159  7038 raft_consensus.cc:993] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: : Instructing follower 1fc57407d56241b38a94d543e073f8af to start an election
I20260709 12:25:06.472859  7038 raft_consensus.cc:1081] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Signalling peer 1fc57407d56241b38a94d543e073f8af to start an election
I20260709 12:25:06.450096  7050 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:06.488336  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 104625us
I20260709 12:25:06.489055  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:06.488584  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:06.490757  7050 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:06.490973  6609 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:06.491657  7033 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:06.492449  7033 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:06.494776  5990 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:06.494920  7041 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.496276  7053 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.496464  7041 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.498071  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:06.503194  6773 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:43760
I20260709 12:25:06.503909  6773 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:06.504295  6773 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:06.506999  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:06.512640  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:06.517341  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:06.519207  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:06.519996  6022 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:06.520833  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:06.521760  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:06.523681  6018 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:06.496836  7053 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.526638  6022 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 5: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:25:06.527472  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.528249  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:06.526506  6948 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:06.530395  6021 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:06.532037  7032 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:06.532774  6023 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:06.529142  6611 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:06.525660  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:06.525789  6020 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:06.468398  6979 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:06.536994  6611 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:06.538130  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.535424  6018 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:06.539773  6020 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:06.541038  6016 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:06.541173  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:06.541509  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:06.541973  5871 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:06.542088  5874 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:06.542407  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:06.460444  6951 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:06.544154  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:06.545109  5874 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:06.517479  7050 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:06.553328  5871 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:06.550704  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:06.556770  7063 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:25:06.557046  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:06.559137  7069 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:06.559614  7041 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:06.560297  7028 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:06.559693  7069 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.560956  7069 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:06.536958  6948 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:06.563468  6948 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:06.545576  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.566304  7053 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:06.566725  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:06.565757  7074 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:06.568123  7074 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.568486  7074 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:06.548535  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:06.554831  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:06.566432  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:06.573722  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.575690  6766 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:06.579227  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:06.538877  7004 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:06.581187  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.574393  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.586017  6793 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:25:06.587525  5990 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:06.545346  7035 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:06.523738  6953 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.590890  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:06.592123  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.594630  6762 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 3: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:06.570552  7050 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.545511  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:06.554842  6768 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:06.614468  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:06.576221  7008 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:06.577086  7030 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:06.616654  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:06.577248  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:06.573189  7033 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:25:06.630980  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:06.632217  5947 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:06.620515  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:06.634593  5871 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:06.611227  7018 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:06.620726  5946 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:06.614645  7050 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:06.638116  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:06.639338  5944 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 2: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:06.611999  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:06.585669  7004 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:06.627743  5949 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 4: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:06.585819  7041 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:06.585719  7032 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.590792  7043 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.585354  7013 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:25:06.619673  7058 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:06.617432  7040 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:25:06.659117  6825 debug-util.cc:398] Leaking SignalData structure 0x7b08000616c0 after lost signal to thread 2070
I20260709 12:25:06.659654  7071 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:06.640733  7018 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.641765  7042 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:06.653258  7004 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:06.662039  6021 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:06.705258  6773 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:06.666584  7058 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.666837  7040 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.721118  6773 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:06.702215  7018 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:06.661090  7032 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:06.717269  7061 log.cc:927] Time spent T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:25:06.722664  7040 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:06.717963  7058 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:06.720108  6953 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:06.700551  7053 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
W20260709 12:25:06.758694  6825 debug-util.cc:398] Leaking SignalData structure 0x7b08000e2cc0 after lost signal to thread 5731
I20260709 12:25:06.751078  7030 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:06.775643  7041 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:06.812547  7041 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.814968  6756 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
W20260709 12:25:06.838065  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 216281us
I20260709 12:25:06.891412  6023 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:25:06.839651  6825 debug-util.cc:398] Leaking SignalData structure 0x7b0800256500 after lost signal to thread 5899
I20260709 12:25:06.895550  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:06.896608  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 280041us
W20260709 12:25:06.901944  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 47835.4us
I20260709 12:25:06.905933  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:06.906535  5874 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:06.896288  7061 log.cc:927] Time spent T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:25:06.910687  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:06.913748  5874 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 2: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:06.893326  6951 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.917728  6951 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:06.920802  6951 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:06.921545  7060 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:06.914752  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:06.926330  6825 debug-util.cc:398] Leaking SignalData structure 0x7b0800399120 after lost signal to thread 5973
W20260709 12:25:06.927743  6825 debug-util.cc:398] Leaking SignalData structure 0x7b0800106e60 after lost signal to thread 6048
W20260709 12:25:06.929104  6825 debug-util.cc:398] Leaking SignalData structure 0x7b0800289e80 after lost signal to thread 6829
W20260709 12:25:06.929383  6825 debug-util.cc:398] Leaking SignalData structure 0x7b08000ef920 after lost signal to thread 6831
I20260709 12:25:06.931381  6022 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:06.932655  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:06.926687  7032 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:06.934450  7055 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:06.934963  7055 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.935302  7055 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:06.955152  7060 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:06.941186  7043 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:06.938786  7032 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:06.941001  7024 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:25:06.968801  7080 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:06.901072  7030 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.912346  7042 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 12:25:06.993083  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:06.993846  6768 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:06.994057  7083 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:25:06.983122  7024 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.994812  7024 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:06.995028  7081 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.234s	user 0.002s	sys 0.000s
I20260709 12:25:06.998760  7069 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:06.980928  7074 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:07.001967  5867 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:07.002774  7023 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.554s	user 0.012s	sys 0.000s
I20260709 12:25:07.003161  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:07.003338  7023 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:07.004079  6778 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:07.004884  6015 raft_consensus.cc:2368] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4.
I20260709 12:25:07.004150  6768 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:07.007802  7040 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:07.010995  5695 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50588 (request call id 52) took 1311 ms. Trace:
I20260709 12:25:07.013025  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:07.014220  7008 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:07.014038  7013 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:07.014747  7008 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:07.015203  7008 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:07.017781  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:07.022904  6016 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:06.982641  7032 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.024473  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:07.029832  7013 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:07.030517  6953 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:07.031175  6953 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:06.994868  7067 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:07.033005  7067 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.035351  6019 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:06.988863  7060 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.048026  7001 raft_consensus.cc:3055] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:07.048640  7001 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:07.050508  7074 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:07.051261  7001 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.057544  7060 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:07.058255  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:07.059115  5947 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:07.040513  7071 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:07.040946  7032 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.061563  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:07.034528  6953 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:07.066704  7023 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:07.068033  7091 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:07.011577  5695 rpcz_store.cc:276] 0709 12:25:05.700020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.700207 (+   187us) service_pool.cc:224] Handling call
0709 12:25:07.010886 (+1310679us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:06.053287 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:06.185897 (+132610us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:06.185941 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:25:06.185959 (+    18us) tablet.cc:662] Decoding operations
0709 12:25:06.186368 (+   409us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:06.186425 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:06.186459 (+    34us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:06.186682 (+   223us) tablet.cc:701] Row locks acquired
0709 12:25:06.186701 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:25:06.186871 (+   170us) write_op.cc:270] Start()
0709 12:25:06.186999 (+   128us) write_op.cc:276] Timestamp: P: 1783599906186822 usec, L: 0
0709 12:25:06.187018 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:25:06.187865 (+   847us) log.cc:844] Serialized 534 byte log entry
0709 12:25:06.413978 (+226113us) op_driver.cc:464] REPLICATION: finished
0709 12:25:06.613325 (+199347us) write_op.cc:301] APPLY: starting
0709 12:25:06.618611 (+  5286us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:06.631501 (+ 12890us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:06.660482 (+ 28981us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:06.660840 (+   358us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:06.692167 (+ 31327us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:06.706660 (+ 14493us) write_op.cc:312] APPLY: finished
0709 12:25:06.742002 (+ 35342us) log.cc:844] Serialized 22 byte log entry
0709 12:25:06.774148 (+ 32146us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:06.778513 (+  4365us) write_op.cc:454] Released schema lock
0709 12:25:06.780108 (+  1595us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:06.836501 (+ 56393us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:06.838547 (+  2046us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":160355,"apply.run_cpu_time_us":3228,"apply.run_wall_time_us":198019,"num_ops":1,"prepare.queue_time_us":156178,"prepare.run_cpu_time_us":11885,"prepare.run_wall_time_us":216376,"raft.queue_time_us":124889,"raft.run_cpu_time_us":70228,"raft.run_wall_time_us":312358,"replication_time_us":194917,"spinlock_wait_cycles":72571136,"thread_start_us":357542,"threads_started":4,"wal-append.queue_time_us":70458}]]}
I20260709 12:25:07.010730  7004 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.052677  7001 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:07.061244  7067 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:07.076359  7058 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.077555  6023 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:07.082563  5867 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:45236
I20260709 12:25:07.074532  5838 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:07.084239  5867 raft_consensus.cc:606] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:25:07.092587  6017 raft_consensus.cc:2368] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4.
I20260709 12:25:07.093367  6951 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:07.094092  6951 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.094413  6951 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.094519  7040 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:07.093434  7060 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:07.091368  7069 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:07.080772  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:07.100483  6766 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:07.111855  7097 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:07.112556  7097 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:07.112959  7018 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.114974  7097 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:07.115341  7018 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:07.117028  7079 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.315s	user 0.004s	sys 0.000s
I20260709 12:25:07.117446  6021 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:25:07.067029  7035 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:25:07.087407  7091 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:07.130336  7058 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:07.139240  7071 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:07.140021  7071 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:07.124390  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 55271.4us
I20260709 12:25:07.141738  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:07.142280  7071 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:07.143748  7037 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:07.125067  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:07.108912  7013 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:07.154184  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:07.155162  6768 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.143766  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:07.161109  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:07.163780  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:07.166713  7050 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.004s
I20260709 12:25:07.167235  7050 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.168399  6020 raft_consensus.cc:2368] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4.
I20260709 12:25:07.169821  6951 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:07.173389  6611 leader_election.cc:400] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4.
I20260709 12:25:07.173684  6024 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 3: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:07.174069  6611 leader_election.cc:403] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:07.177201  6611 leader_election.cc:400] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4.
I20260709 12:25:07.177695  6611 leader_election.cc:403] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:07.178871  7096 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:07.159415  7013 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:07.181568  7096 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:07.177932  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:07.195061  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:07.195498  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:07.196138  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:07.197033  6020 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:07.196080  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:07.199191  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:07.189919  7058 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:07.202116  6785 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:07.202035  7058 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:07.202922  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:07.203672  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:07.204180  6768 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:07.190632  7037 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 4
W20260709 12:25:07.193329  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 49684.9us
I20260709 12:25:07.216666  6778 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 4: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:07.217955  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:07.208819  7044 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.218669  6778 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:07.221285  5915 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:07.219056  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:07.222507  7039 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:07.223059  7039 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:07.204824  7050 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:07.219837  6756 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.202934  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:07.223717  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:07.234338  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:07.226055  6757 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:07.226617  7024 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.240036  7055 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:07.232594  6020 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:07.244623  7004 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:07.243474  6024 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:25:07.265712  7053 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:07.254691  7095 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:25:07.267441  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:07.269093  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:07.170810  7091 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:07.228287  7058 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:07.252337  7033 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:25:07.276521  7058 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.276844  7033 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:07.277287  7033 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:07.277061  7058 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:07.277952  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:07.278702  5869 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.278353  7033 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.279502  7058 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:07.279824  7033 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:07.224094  7039 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:07.248176  7035 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:07.273908  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:07.282740  5940 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.275588  6778 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:07.271588  7050 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:25:07.269883  7095 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:07.292586  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:07.263660  7071 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:07.301468  5990 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:07.311671  7030 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:07.221922  7013 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:07.306919  7004 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:07.319171  7030 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:07.313021  7053 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:07.313645  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 44621.4us
W20260709 12:25:07.429340  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 79681us
I20260709 12:25:07.329403  6757 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:07.428066  6951 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:07.436069  7055 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:07.437389  6756 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:07.442425  7037 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4.
I20260709 12:25:07.445739  7044 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:07.313155  7071 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:07.449566  7071 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:07.459022  5943 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:07.459354  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:07.457317  6762 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:07.463387  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:07.436888  7030 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.466976  5944 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:07.487551  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:07.488279  5990 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:07.344412  7008 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:07.492890  5942 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:07.497375  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:07.499356  6766 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.435056  7053 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:07.493882  5948 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:07.494361  7060 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:07.497231  7008 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:07.549279  6766 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:07.497779  6768 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.562503  7060 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:07.563719  7060 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:07.567256  5990 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:07.568660  7055 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:07.569303  7055 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:07.570442  7055 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.572635  7053 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:07.606804  7008 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.607388  7008 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:07.606236  7001 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.003s
I20260709 12:25:07.609265  7058 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.581760  7024 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:07.500322  7050 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.110s
I20260709 12:25:07.603364  7096 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4.
I20260709 12:25:07.615845  7058 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:07.605995  7053 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:07.621460  7032 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.540s	user 0.016s	sys 0.000s
I20260709 12:25:07.636171  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:07.633821  7008 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:07.625447  7053 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.642256  6979 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.642160  7096 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:07.652972  7001 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:25:07.667043  7096 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:07.667480  7096 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:07.671639  5944 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:07.632239  7032 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.647687  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:07.676704  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:07.679870  5944 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.679091  6979 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:07.694640  5944 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:07.695930  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:07.696587  5944 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:07.704988  7001 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:07.706099  7001 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.707696  7032 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:07.684957  7055 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:07.729859  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.731957  7058 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:07.732535  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:07.726147  6793 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:07.732115  7044 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.720707  5944 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:07.709282  7001 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:07.742595  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:07.743672  5944 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:07.736747  6768 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
W20260709 12:25:07.835506  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 88303.7us
I20260709 12:25:07.735301  7055 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:07.694940  7096 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.861377  7040 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.765213  7030 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.862267  7040 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:07.863305  7030 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.867770  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:07.869686  7024 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:07.872045  7024 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:07.872551  7055 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:07.887181  7053 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:07.752568  7060 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.889338  7060 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:07.892596  7060 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:07.904263  7040 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:07.745280  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:07.740447  7044 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:07.872754  7024 raft_consensus.cc:697] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:08.261366  7108 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.113153  7113 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.126187  7008 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.141285  7040 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.174957  7060 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.288537  6979 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:08.315977  6022 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:08.320607  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:08.325410  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:08.327315  6020 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:25:08.336728  6020 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:08.340777  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:08.342401  6019 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 3: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:08.286751  7108 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:08.298538  7040 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:08.339330  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:08.289973  7113 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:08.349964  6020 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:08.293053  7008 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:08.330044  6979 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.334250  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:08.313179  7060 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:08.400094  7110 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 12:25:08.396811  6021 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:08.412806  7119 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.413653  7119 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:08.416607  7119 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:08.417944  7118 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:25:08.419175  7111 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.507s	user 0.000s	sys 0.002s
I20260709 12:25:08.418823  7118 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:08.419767  5873 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.145s	user 0.005s	sys 0.025s
I20260709 12:25:08.422164  7120 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.423018  7120 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:08.423837  7114 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:25:08.426512  7120 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:08.426723  7113 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:08.427460  7113 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.427376  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:08.428304  5945 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:08.428148  7113 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:08.428352  7120 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:08.423038  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:08.430557  6766 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:08.431663  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:08.435400  7108 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:08.436779  7008 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:08.437023  7044 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:08.344799  7024 consensus_queue.cc:237] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:08.440397  7115 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:08.440834  7091 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:08.441066  7115 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:08.443305  7119 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.443951  7119 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:08.452348  7118 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:08.456626  7119 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:08.373258  6979 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:08.393234  6778 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
W20260709 12:25:08.461287  5873 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.187s	user 0.008s	sys 0.026s
I20260709 12:25:08.470757  5990 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:08.483422  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 164802us
I20260709 12:25:08.491377  6020 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:08.490793  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:08.493274  6019 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:08.495623  6021 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:08.499361  6979 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:08.502359  7121 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.503011  7121 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:08.467221  7119 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.504525  7119 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:08.505278  7121 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:08.507334  7119 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:08.508478  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 434) took 1313 ms. Trace:
I20260709 12:25:08.509364  6793 rpcz_store.cc:276] 0709 12:25:07.194765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.195850 (+  1085us) service_pool.cc:224] Handling call
0709 12:25:08.508190 (+1312340us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54002a0dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:08.508354 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":986072320}
I20260709 12:25:08.516687  5990 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:08.521549  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:08.526937  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:08.527904  6793 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:08.508611  7113 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:08.533983  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:08.535179  7125 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:08.526782  7117 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:08.545423  7122 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:25:08.558674  5990 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:08.528949  7119 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.560207  6793 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:08.563405  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 604) took 1292 ms. Trace:
I20260709 12:25:08.563819  5873 rpcz_store.cc:276] 0709 12:25:07.267986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.273895 (+  5909us) service_pool.cc:224] Handling call
0709 12:25:07.274187 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.274415 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:08.475265 (+1200850us) log.cc:844] Serialized 14 byte log entry
0709 12:25:08.559835 (+ 84570us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:08.559888 (+    53us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:08.559936 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.560044 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.560170 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":265712,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"thread_start_us":1284351,"threads_started":3,"wal-append.queue_time_us":59632}
I20260709 12:25:08.545044  7113 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:08.567512  6793 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:08.557610  7117 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.568665  7117 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:08.560056  6979 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.565265  7108 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:08.561913  7125 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.562922  7024 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.577912  7125 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:08.566166  7119 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:08.571525  7121 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:25:08.576668  7108 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.584650  7108 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:08.586905  7125 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:08.578904  6979 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:08.589506  7133 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:08.590095  6979 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:08.567337  7122 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.580035  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:08.591177  5943 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:08.591023  7122 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:08.576486  7058 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.592384  7058 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:08.592927  7126 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:25:08.590152  7133 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:08.594950  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 588) took 1399 ms. Trace:
I20260709 12:25:08.595412  6024 rpcz_store.cc:276] 0709 12:25:07.195102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.195241 (+   139us) service_pool.cc:224] Handling call
0709 12:25:08.594741 (+1399500us) spinlock_profiling.cc:243] Waited 47.3 ms on lock 0x7b54003647d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:08.594884 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108804480}
I20260709 12:25:08.596228  7133 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:08.597012  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 433) took 1402 ms. Trace:
I20260709 12:25:08.597432  6762 rpcz_store.cc:276] 0709 12:25:07.194437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.194626 (+   189us) service_pool.cc:224] Handling call
0709 12:25:08.596838 (+1402212us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b5400006c54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:08.596952 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":575649536}
I20260709 12:25:08.598439  7058 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:08.582459  7131 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:08.601241  7131 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.601769  7131 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:08.602581  7013 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.603240  7013 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:08.607338  7119 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:08.572404  7113 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:08.617556  7113 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:08.610909  7024 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:08.624816  5947 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
W20260709 12:25:08.629935  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 67804.3us
I20260709 12:25:08.631529  6611 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:08.631661  7125 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.633273  7134 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:25:08.610630  7122 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:08.634670  7122 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:08.635187  7122 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.633929  7125 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:08.635743  7122 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:08.611104  7013 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:08.635295  7134 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:08.641345  7024 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:08.614358  7117 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:08.648404  7117 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:08.654938  5987 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.716s (SENT) [suppressed 324 similar messages]
I20260709 12:25:08.717911  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:08.719168  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:08.722079  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:08.722862  6024 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:25:08.723299  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:08.724237  6762 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 3: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:08.718195  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:08.727088  6022 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
W20260709 12:25:08.716434  7129 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.010s
W20260709 12:25:08.737923  7127 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.099s	user 0.009s	sys 0.000s
I20260709 12:25:08.716285  7130 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:08.726245  7117 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.741242  7013 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.763819  7141 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.764460  7141 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:08.759696  7125 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:08.768682  7141 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:08.770422  7122 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:08.773195  7122 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:08.774005  7013 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:08.798702  7013 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:08.761834  7091 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.777671  5946 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:08.732810  7131 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:08.768702  6775 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:25:08.752437  7117 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:08.771350  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 334) took 1312 ms. Trace:
I20260709 12:25:08.752557  7130 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:08.792389  5940 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.509s	user 0.019s	sys 0.012s
I20260709 12:25:08.781854  7108 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:08.794842  7060 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:08.807550  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 61) took 1348 ms. Trace:
I20260709 12:25:08.820214  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 448) took 1131 ms. Trace:
I20260709 12:25:08.827059  5946 rpcz_store.cc:276] 0709 12:25:07.457692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.457819 (+   127us) service_pool.cc:224] Handling call
0709 12:25:08.777209 (+1319390us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b5800030398. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:25:08.777410 (+   201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.805481 (+ 28071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.805760 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2318592}
W20260709 12:25:08.808446  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 249786us
I20260709 12:25:08.806471  6775 pending_rounds.cc:85] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 2
W20260709 12:25:08.811142  7112 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 1.109s	user 0.005s	sys 0.012s
I20260709 12:25:08.827978  6766 rpcz_store.cc:276] 0709 12:25:07.688634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.432581 (+743947us) service_pool.cc:224] Handling call
0709 12:25:08.433316 (+   735us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:08.433782 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:08.433818 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:08.438555 (+  4737us) log.cc:844] Serialized 26 byte log entry
0709 12:25:08.531368 (+ 92813us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:08.531431 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.531530 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.819906 (+288376us) raft_consensus.cc:1685] finished
0709 12:25:08.819934 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.820131 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2407,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":125,"thread_start_us":92264,"threads_started":1,"wal-append.queue_time_us":133871}
I20260709 12:25:08.834154  7137 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:08.808959  6757 rpcz_store.cc:276] 0709 12:25:07.458696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.460539 (+  1843us) service_pool.cc:224] Handling call
0709 12:25:07.494348 (+ 33809us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.767880 (+1273532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:08.767936 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:08.768070 (+   134us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:08.768140 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.770892 (+  2752us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540052a8d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:08.770972 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.771220 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2927848832}
I20260709 12:25:08.834908  7137 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:08.831193  6775 pending_rounds.cc:107] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:25:08.835875  6775 raft_consensus.cc:2889] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:08.836849  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 446) took 1293 ms. Trace:
I20260709 12:25:08.837148  7137 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:08.837203  6775 rpcz_store.cc:276] 0709 12:25:07.543541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.637707 (+ 94166us) service_pool.cc:224] Handling call
0709 12:25:08.767944 (+1130237us) spinlock_profiling.cc:243] Waited 5.59 ms on lock 0x7b580001f898. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:25:08.768223 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.836368 (+ 68145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.836782 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12863744}
W20260709 12:25:08.839684  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 184768us
I20260709 12:25:08.839732  5990 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:08.838706  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 436) took 1643 ms. Trace:
I20260709 12:25:08.845124  7124 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:25:08.812165  5940 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
W20260709 12:25:08.846884  5940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 553) took 1585 ms (client timeout 1741 ms). Trace:
I20260709 12:25:08.844986  6768 rpcz_store.cc:276] 0709 12:25:07.195364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.223327 (+ 27963us) service_pool.cc:224] Handling call
0709 12:25:08.838403 (+1615076us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54002a21d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:08.838597 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":604955904}
I20260709 12:25:08.805516  7091 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:08.847342  7131 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:08.858407  6023 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:08.861413  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:08.862289  6024 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:25:08.847700  7060 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:08.840438  7141 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:08.866873  7141 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:08.869330  7141 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:08.872979  7117 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:08.847479  5940 rpcz_store.cc:269] 0709 12:25:07.261374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.263458 (+  2084us) service_pool.cc:224] Handling call
0709 12:25:08.846742 (+1583284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.873653  7060 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:08.854803  7113 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.863718  7148 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:08.842949  7137 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:08.855262  7130 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:08.840644  7024 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:25:08.884117  7113 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:09.122987  7176 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.123944  7176 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:09.124738  7113 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:09.125255  7044 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.885501  7137 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:08.885860  7148 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:09.125877  7044 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:09.127853  7137 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:09.128263  7148 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:08.891553  7130 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:09.129242  7044 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:08.890892  7141 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:09.129413  7130 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:09.130100  7141 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:09.131703  7130 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:08.895087  7152 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:08.898454  5694 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40830 (request call id 146) took 2842 ms. Trace:
I20260709 12:25:09.131932  7044 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:09.132062  7141 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:08.903620  5840 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:08.893769  7108 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:08.928957  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:08.942203  7151 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:25:09.133940  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 230430us
I20260709 12:25:09.136274  7140 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:09.136677  7140 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.137024  7140 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:08.966437  7074 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:09.137537  7074 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:08.977362  7157 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.132462  5694 rpcz_store.cc:276] 0709 12:25:06.056111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.056308 (+   197us) service_pool.cc:224] Handling call
0709 12:25:08.898332 (+2842024us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:06.261215 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:06.477328 (+216113us) spinlock_profiling.cc:243] Waited 8.76 ms on lock 0x7b540077f7a8. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d903f9e1 00007f39d903e64c 00007f39d66cdd5d 00007f39d66b90a6 00007f39d66b8d58 00007f39d66bd9f1 00007f39d66bd9a9 00007f39d66bd939 00007f39d66bd901 00007f39d66bcb2d 0000000000388634
0709 12:25:06.477419 (+    91us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:06.477449 (+    30us) write_op.cc:435] Acquired schema lock
0709 12:25:06.477465 (+    16us) tablet.cc:662] Decoding operations
0709 12:25:06.477934 (+   469us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:06.477994 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:06.478020 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:06.478211 (+   191us) tablet.cc:701] Row locks acquired
0709 12:25:06.478222 (+    11us) write_op.cc:260] PREPARE: finished
0709 12:25:06.478405 (+   183us) write_op.cc:270] Start()
0709 12:25:06.478629 (+   224us) write_op.cc:276] Timestamp: P: 1783599906478351 usec, L: 0
0709 12:25:06.478654 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:25:06.623788 (+145134us) log.cc:844] Serialized 534 byte log entry
0709 12:25:06.810395 (+186607us) op_driver.cc:464] REPLICATION: finished
0709 12:25:06.900780 (+ 90385us) write_op.cc:301] APPLY: starting
0709 12:25:06.909326 (+  8546us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:06.928021 (+ 18695us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:06.928047 (+    26us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:06.928346 (+   299us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:06.928520 (+   174us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:06.928570 (+    50us) write_op.cc:312] APPLY: finished
0709 12:25:06.928794 (+   224us) log.cc:844] Serialized 22 byte log entry
0709 12:25:06.928966 (+   172us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:06.929061 (+    95us) write_op.cc:454] Released schema lock
0709 12:25:06.929105 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:06.993612 (+ 64507us) spinlock_profiling.cc:243] Waited 54.9 ms on lock 0x7b540077f954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c7ee28 00007f39d66b9581 00007f39d66b8d58 00007f39d66bd9f1 00007f39d66bd9a9 00007f39d66bd939 00007f39d66bd901 00007f39d66bcb2d 0000000000388634 0000000000388479
0709 12:25:06.993864 (+   252us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:06.993890 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":85614,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":80972,"apply.run_cpu_time_us":1817,"apply.run_wall_time_us":31608,"num_ops":1,"prepare.queue_time_us":105544,"prepare.run_cpu_time_us":13963,"prepare.run_wall_time_us":732631,"replication_time_us":322521,"spinlock_wait_cycles":146423936,"thread_start_us":131719,"threads_started":1}]]}
I20260709 12:25:09.139989  7074 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:08.976259  7155 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:08.989514  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:08.993050  7158 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:09.140826  7155 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:08.991978  7160 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.982756  7040 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:09.007787  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.005970  7134 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.012662  7091 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:09.014618  7162 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.132477  7152 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:09.019639  7166 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:09.032482  7159 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.035410  7163 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.034490  7164 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:07.319851  5869 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
W20260709 12:25:09.147946  5869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 607) took 1878 ms (client timeout 1549 ms). Trace:
I20260709 12:25:09.048055  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:09.050211  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.149883  5874 raft_consensus.cc:2368] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:09.147480  7163 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:09.051872  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:09.052611  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.151451  7159 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:09.054327  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.055325  7168 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:09.061998  6024 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:09.062949  7115 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:09.141191  7158 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:09.068710  7169 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:09.075590  7171 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:09.078677  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:09.080050  7173 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:09.082489  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.083884  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.151612  5870 raft_consensus.cc:2368] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:09.085180  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:08.886042  7060 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:09.098114  7172 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:08.854882  6611 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:09.143844  7162 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:09.183513  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 328694us
I20260709 12:25:09.112159  7174 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:09.118499  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.119730  7175 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.129597  7176 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:09.132635  7044 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:08.972950  7144 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.139842  7157 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:08.986126  7156 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.141896  7160 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:08.998891  7118 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:09.054193  5868 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:09.066301  5948 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:09.154422  5872 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:09.160274  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.177407  7173 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:09.204784  5874 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
W20260709 12:25:09.148514  5869 rpcz_store.cc:269] 0709 12:25:07.269113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.272736 (+  3623us) service_pool.cc:224] Handling call
0709 12:25:09.147815 (+1875079us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.179092  5946 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:09.148761  7108 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:09.184351  6611 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:09.189762  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 55189.5us
I20260709 12:25:09.185977  5940 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:09.155092  7168 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:09.182808  5945 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:09.177368  7169 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:09.210979  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 554) took 1534 ms. Trace:
I20260709 12:25:09.209280  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.216146  5948 rpcz_store.cc:276] 0709 12:25:07.669853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.065313 (+1395460us) service_pool.cc:224] Handling call
0709 12:25:09.065803 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.204246 (+138443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.204465 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.217273  7177 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.223745  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.195104  7153 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:09.203771  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.232038  5865 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:09.144790  7166 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:09.224077  7146 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:09.210759  7118 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:09.206853  5941 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:09.229986  7162 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:09.230734  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.231334  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.245141  5945 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:25:09.146049  7134 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:09.134276  7151 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:09.190459  7174 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:09.147542  7164 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:09.215313  7156 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:09.226658  7153 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:09.250233  5693 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40818 (request call id 145) took 3262 ms. Trace:
I20260709 12:25:09.185268  7171 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:09.257158  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.267356  5948 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:25:09.194708  7172 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:09.216394  7108 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:09.250113  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.262257  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.278456  5866 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:25:09.279139  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.279417  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.279752  7166 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:09.211104  7144 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:09.280553  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.265224  5873 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:09.281900  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.283075  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.283540  7044 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:09.284266  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.284552  7163 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:09.244247  7159 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:09.240209  7146 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:09.286259  7146 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:09.287006  7174 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:09.259927  5693 rpcz_store.cc:276] 0709 12:25:05.987792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.987966 (+   174us) service_pool.cc:224] Handling call
0709 12:25:09.250167 (+3262201us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:06.994333 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:06.994947 (+   614us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:06.994987 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:25:06.995006 (+    19us) tablet.cc:662] Decoding operations
0709 12:25:06.995635 (+   629us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:06.995701 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:06.995738 (+    37us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:25:06.996098 (+   360us) tablet.cc:701] Row locks acquired
0709 12:25:06.996114 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:25:06.996276 (+   162us) write_op.cc:270] Start()
0709 12:25:06.996408 (+   132us) write_op.cc:276] Timestamp: P: 1783599906996227 usec, L: 0
0709 12:25:06.996425 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 12:25:06.997193 (+   768us) log.cc:844] Serialized 916 byte log entry
0709 12:25:07.033495 (+ 36302us) op_driver.cc:464] REPLICATION: finished
0709 12:25:07.046580 (+ 13085us) write_op.cc:301] APPLY: starting
0709 12:25:07.046698 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:07.046840 (+   142us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:07.047016 (+   176us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:07.048387 (+  1371us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:07.048599 (+   212us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:25:07.048650 (+    51us) write_op.cc:312] APPLY: finished
0709 12:25:07.048895 (+   245us) log.cc:844] Serialized 28 byte log entry
0709 12:25:07.049149 (+   254us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:07.049255 (+   106us) write_op.cc:454] Released schema lock
0709 12:25:07.049300 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:07.062544 (+ 13244us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:07.064648 (+  2104us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2025286,"reported_tablets":3,"spinlock_wait_cycles":2944,"child_traces":[["op",{"apply.queue_time_us":3650,"apply.run_cpu_time_us":3193,"apply.run_wall_time_us":5185,"num_ops":2,"prepare.queue_time_us":838622,"prepare.run_cpu_time_us":3318,"prepare.run_wall_time_us":38828,"replication_time_us":36888}]]}
I20260709 12:25:09.264402  7118 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:09.288551  7157 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:09.272135  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:09.289728  5948 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:09.201313  7175 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:09.285498  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:09.292716  5866 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 3: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:09.296623  5873 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:09.266546  7153 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:09.223809  7177 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:09.268448  7173 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:09.280565  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.294610  7118 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:09.309152  5840 leader_election.cc:400] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4.
I20260709 12:25:09.309521  5840 leader_election.cc:403] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:09.304811  7004 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:09.311600  7118 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:09.312013  7004 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:09.313998  7118 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:09.314702  7004 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:09.315430  7004 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:09.315944  7004 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:09.318392  7004 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:09.321098  7141 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:09.318297  7171 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:09.330713  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.332947  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.302331  7155 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:09.341327  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.345022  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:09.352084  5948 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:09.353673  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.353735  5838 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:09.355049  5872 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:09.363585  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.362265  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:09.369572  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.371312  5948 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:25:09.369820  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.346951  7151 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:09.363147  7159 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:09.363332  7155 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:09.357797  5940 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:25:09.377763  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.374308  7152 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:09.370949  7144 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:09.384685  7172 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:09.373932  7148 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:09.381214  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.382759  7159 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:09.402231  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:09.394731  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.404958  5948 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:25:09.391330  7175 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:09.389559  7141 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4.
I20260709 12:25:09.409554  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.393275  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:09.393900  7180 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:25:09.393347  7164 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:09.392335  7148 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:09.411108  5948 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:25:09.394188  7108 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:09.415679  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.418347  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.391765  7156 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:09.401366  7168 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:09.420876  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.420637  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.423810  6766 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:25:09.429343  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.423511  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.432713  5948 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:09.390286  7134 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:09.420753  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.435818  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.436836  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.436318  7146 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:09.398850  7159 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:09.441605  5948 raft_consensus.cc:2410] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:09.438645  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.438058  7148 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:09.410580  7170 consensus_queue.cc:1059] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:25:09.440680  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.434063  7140 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:09.437356  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.443040  6766 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:09.445668  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.447507  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.437356  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.454470  6766 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:09.456516  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:09.460397  5948 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:09.457787  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:09.461488  5872 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:09.454540  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:09.456190  6785 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:09.462603  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.463948  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.456408  7108 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:09.477295  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.477486  7151 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:09.477578  7108 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:09.478283  7146 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:09.477336  6757 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 3: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:09.478690  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.479386  6785 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
W20260709 12:25:09.470395  5947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 60) took 2013 ms (client timeout 1563 ms). Trace:
I20260709 12:25:09.443843  7164 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:09.480011  5947 rpcz_store.cc:269] 0709 12:25:07.457312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.457480 (+   168us) service_pool.cc:224] Handling call
0709 12:25:09.470219 (+2012739us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540043f8d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:09.470331 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2594698752}
I20260709 12:25:09.456290  7134 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.466213  5867 raft_consensus.cc:2368] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3.
I20260709 12:25:09.478157  7151 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:09.485150  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.485917  7151 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:09.485981  5947 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:25:09.487221  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.487546  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.487876  5947 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:09.489054  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.479562  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.467862  7140 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:09.490671  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:09.490672  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.491518  5940 raft_consensus.cc:2368] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:09.491792  6757 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 4: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:09.492650  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:09.483886  7134 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:09.479606  7153 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:09.495285  7134 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:09.496177  7153 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:09.496297  7096 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:09.484333  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:09.484962  6775 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:25:09.492895  5940 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:09.492653  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.497854  7060 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:09.498383  7060 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:09.498668  5940 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:25:09.499151  5940 pending_rounds.cc:85] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 1
I20260709 12:25:09.499183  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.483850  7164 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:09.503698  6785 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:09.504529  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.505080  7108 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:09.505558  6017 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:09.515447  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.506029  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.517494  7060 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:09.517603  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.518548  6023 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
W20260709 12:25:09.508183  5941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 561) took 1822 ms (client timeout 2177 ms). Trace:
W20260709 12:25:09.519354  5941 rpcz_store.cc:269] 0709 12:25:07.685735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.928566 (+1242831us) service_pool.cc:224] Handling call
0709 12:25:09.507962 (+579396us) spinlock_profiling.cc:243] Waited 66.9 ms on lock 0x7b5400672d54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:09.508100 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153887104}
I20260709 12:25:09.520100  6019 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:09.520584  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.509085  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.522311  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:09.522485  6756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 335) took 2063 ms (client timeout 1991 ms). Trace:
I20260709 12:25:09.522959  5868 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:09.511083  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.521536  5941 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 3: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:09.509598  7108 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.514308  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.525319  7108 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:09.515117  7164 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:09.499513  5940 pending_rounds.cc:107] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 2.2
W20260709 12:25:09.522933  6756 rpcz_store.cc:269] 0709 12:25:07.460342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.462034 (+  1692us) service_pool.cc:224] Handling call
0709 12:25:09.508889 (+2046855us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b580001f898. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:09.522412 (+ 13523us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6549760}
I20260709 12:25:09.526717  5940 raft_consensus.cc:2889] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:09.527431  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.527729  7108 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:09.510000  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:09.528999  5869 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:09.529309  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:09.530975  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.531436  7024 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:09.532152  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.507521  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.532887  6756 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:25:09.514137  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.528901  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.534096  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.525462  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.534744  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:09.545984  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 63051.5us
I20260709 12:25:09.546257  6015 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:09.547248  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.547427  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:09.548019  6015 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:09.535449  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.529322  5872 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:09.536940  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.550715  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:09.551070  6024 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:25:09.551898  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.552285  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.552949  5872 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:09.538295  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.554690  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:09.555543  6020 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:09.538677  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.542258  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.535475  5915 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:09.557018  6018 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:09.550741  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.558046  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.558893  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:09.559386  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:09.559959  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:09.560319  6022 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:09.543998  5948 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:09.549386  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.561818  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:09.562850  5948 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:25:09.563576  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.554965  6611 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:09.556860  6019 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:09.567809  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:09.569943  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.570428  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:09.571084  6018 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:09.574441  5869 raft_consensus.cc:2410] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:09.575842  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:09.576041  5915 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:09.576944  5872 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:09.577270  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:09.578554  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.585058  5868 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:09.587615  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:09.588090  5915 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:09.591682  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:09.602214  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:09.619926  5915 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:09.592818  5697 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40812 (request call id 139) took 2960 ms. Trace:
I20260709 12:25:09.528455  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:09.506917  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.518692  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.606863  5872 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:09.567021  7157 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:09.569604  7130 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:09.610666  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:09.565675  5870 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:25:09.623032  6018 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:09.626678  6020 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:09.587045  7137 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:09.629052  5948 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:09.587275  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.607332  7172 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:09.565872  7164 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:25:09.623939  5697 rpcz_store.cc:276] 0709 12:25:06.575089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.609438 (+ 34349us) service_pool.cc:224] Handling call
0709 12:25:09.535647 (+2926209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2111704,"reported_tablets":1}
I20260709 12:25:09.586401  7141 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:09.608698  7171 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:09.606666  7071 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:09.578591  7184 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:09.690044  5872 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:09.594547  7175 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:09.704834  6020 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:25:09.708031  5874 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:25:09.713850  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 136623us
I20260709 12:25:09.725849  5840 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:09.691874  7170 raft_consensus.cc:2764] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:25:09.609277  7118 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.110s
I20260709 12:25:09.636353  7174 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:09.608938  7178 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:09.692670  7172 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.636056  7163 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:09.703806  7130 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:09.743986  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 189149us
I20260709 12:25:09.703902  7141 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:09.704190  7171 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:09.756088  6611 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:09.746683  7184 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:09.750154  7172 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:09.753741  7130 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:09.755354  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:09.753772  7141 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:09.756455  7171 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:09.751959  7163 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:09.773934  7184 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:09.788818  7153 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:09.827293  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 82124.6us
I20260709 12:25:09.832319  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:09.799476  7163 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:09.847023  7134 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:09.829185  7153 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.858417  5838 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:09.858877  7192 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:09.844717  6756 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 4: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:25:09.866739  5949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 557) took 2195 ms (client timeout 2168 ms). Trace:
I20260709 12:25:09.829293  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.829602  7184 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:09.863801  7151 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:09.872961  5949 rpcz_store.cc:269] 0709 12:25:07.670913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.866412 (+2195499us) service_pool.cc:211] Skipping call since client already timed out
0709 12:25:09.866610 (+   198us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:25:09.873380  7153 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:09.922945  6022 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:09.883001  7040 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:09.883494  7170 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:10.003950  7141 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:09.959563  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.966024  7201 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:09.967118  7195 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:25:09.970275  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:10.005225  6775 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:10.008385  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:10.005491  7201 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:09.929364  7134 raft_consensus.cc:697] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:09.978060  6793 raft_consensus.cc:2410] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:25:10.025704  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 146086us
I20260709 12:25:09.980822  6785 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:09.984905  7151 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:09.993964  7040 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:09.995694  5866 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:09.997505  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 610) took 1013 ms. Trace:
I20260709 12:25:09.959308  6021 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:10.004287  7170 raft_consensus.cc:697] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:10.055481  7170 consensus_queue.cc:237] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:10.006922  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:09.976287  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:10.058506  6773 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
W20260709 12:25:10.059427  6773 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 455) took 2158 ms (client timeout 2474 ms). Trace:
W20260709 12:25:10.059948  6773 rpcz_store.cc:269] 0709 12:25:07.901233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.329012 (+427779us) service_pool.cc:224] Handling call
0709 12:25:09.976016 (+1647004us) spinlock_profiling.cc:243] Waited 6.65 ms on lock 0x7b580001f898. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:10.059316 (+ 83300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15295360}
I20260709 12:25:10.007225  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:10.061528  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:10.062662  6019 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:10.063086  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:10.009402  6775 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 2: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:10.067117  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:10.068825  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:10.021090  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:10.070737  5947 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:10.070843  6024 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:10.073658  7134 consensus_queue.cc:237] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:10.073755  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:10.076428  6773 raft_consensus.cc:2376] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3. Re-sending same reply.
I20260709 12:25:10.077082  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:10.078480  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.079118  6775 raft_consensus.cc:2410] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:10.080118  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.080161  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:10.025451  7008 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.081533  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.078449  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:10.090210  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:10.092991  6773 raft_consensus.cc:2410] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:10.093786  5872 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 3: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:10.111219  6023 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:10.115084  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.053169  5867 rpcz_store.cc:276] 0709 12:25:08.983806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.983993 (+   187us) service_pool.cc:224] Handling call
0709 12:25:09.997194 (+1013201us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54002f99d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:09.997390 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":747242240}
I20260709 12:25:10.125682  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:10.120083  5987 connection.cc:256] RPC call timeout handler was delayed by -1.2049529199998688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
W20260709 12:25:10.129753  5987 connection.cc:256] RPC call timeout handler was delayed by -1.1968529199998557s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2177s for a response
I20260709 12:25:10.130897  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.135174  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.136008  5874 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:25:10.146485  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:10.117908  7096 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:10.151181  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:10.116413  6756 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:25:10.122440  7138 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:10.154605  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:10.078372  7172 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:10.081585  7163 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:10.160429  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.086241  7008 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:10.162328  5867 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 12:25:10.155424  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 399390us
I20260709 12:25:10.163194  6611 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:10.143185  6775 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:10.174096  5871 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:10.157577  7192 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:10.157044  7152 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:10.115958  5872 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:10.114562  7171 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:10.153126  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.181607  5941 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:10.152275  7184 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:10.183847  7171 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:10.184645  5949 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
W20260709 12:25:10.166697  5987 connection.cc:256] RPC call timeout handler was delayed by -1.1869529199998397s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21880000000000002s for a response
I20260709 12:25:10.178738  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:10.186710  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:10.164819  6609 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.023s	sys 0.059s
W20260709 12:25:10.187541  6609 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.024s	sys 0.059s
I20260709 12:25:10.187745  6611 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:10.187916  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 654449us
I20260709 12:25:10.165053  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:10.189463  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 146907us
I20260709 12:25:10.193154  5840 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:10.194083  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:10.158277  7096 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:10.180805  7192 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.159968  7130 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:10.165233  7138 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:10.218303  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:10.195482  7194 raft_consensus.cc:2764] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:25:10.185424  7204 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:10.224954  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.238719  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:10.227550  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:10.239814  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.240193  5946 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:10.240583  5867 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:10.192938  7193 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:10.193328  7108 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:25:10.191470  7120 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:10.242336  7095 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:10.242897  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.243454  7040 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:10.233222  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:10.244364  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.244596  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:10.219543  7184 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:10.245223  5871 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 2: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:10.245906  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:10.246580  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:10.215616  6022 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:10.246948  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:10.247412  7120 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:10.248068  7191 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:10.199156  7151 consensus_queue.cc:1059] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 12:25:10.242990  7095 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:10.214995  7192 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:10.281486  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.248072  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:10.274289  6018 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:10.277908  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:10.276290  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 569) took 1192 ms. Trace:
I20260709 12:25:10.296351  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.242331  7193 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:10.275272  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 567) took 1191 ms. Trace:
I20260709 12:25:10.341652  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:10.248443  7211 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:25:10.343667  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.344738  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:10.350288  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:10.346638  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.283413  7155 raft_consensus.cc:993] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: : Instructing follower 6add331326ca47038308a447bf798fa7 to start an election
I20260709 12:25:10.231774  7159 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:10.257805  7096 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:10.364207  7096 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:10.248795  7120 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:10.365731  7191 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:10.218151  7209 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:10.249140  7203 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:10.283303  7177 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:10.366993  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:10.247725  7108 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:10.353084  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.285656  7030 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:10.269273  7121 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:10.379799  7121 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:10.334784  5943 rpcz_store.cc:276] 0709 12:25:09.083685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.084517 (+   832us) service_pool.cc:224] Handling call
0709 12:25:09.084894 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:09.151449 (+ 66555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.151480 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:09.151955 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:25:09.211999 (+ 60044us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:09.212094 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.274729 (+1062635us) spinlock_profiling.cc:243] Waited 65.9 ms on lock 0x7b5400530f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:10.274803 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.274870 (+    67us) raft_consensus.cc:1685] finished
0709 12:25:10.274890 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.275192 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4358,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":159,"spinlock_wait_cycles":151674624,"thread_start_us":59612,"threads_started":1,"wal-append.queue_time_us":106077}
I20260709 12:25:10.382388  7121 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:10.382937  7033 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:10.383635  7033 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:10.384920  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:10.386111  6766 raft_consensus.cc:2410] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:10.386945  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:10.387938  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:10.388410  6766 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.331759  5944 rpcz_store.cc:276] 0709 12:25:09.084301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.084437 (+   136us) service_pool.cc:224] Handling call
0709 12:25:09.084755 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.243668 (+158913us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:09.364682 (+121014us) log.cc:844] Serialized 14 byte log entry
0709 12:25:09.370971 (+  6289us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:09.371035 (+    64us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:09.371086 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.275991 (+904905us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54008b01d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:10.276060 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.276219 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":149777,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":138,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":2533,"spinlock_wait_cycles":364878080,"thread_start_us":119739,"threads_started":2,"wal-append.queue_time_us":19044}
I20260709 12:25:10.285202  7138 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:10.251441  7130 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:10.256028  7172 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:10.258426  7201 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:10.254009  7037 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:10.391882  7037 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:10.394987  5940 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:10.395995  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 67) took 1293 ms. Trace:
I20260709 12:25:10.339283  7205 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 12:25:10.396436  5940 rpcz_store.cc:276] 0709 12:25:09.102248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.973457 (+871209us) service_pool.cc:224] Handling call
0709 12:25:10.394273 (+420816us) spinlock_profiling.cc:243] Waited 9.55 ms on lock 0x7b5800030398. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5ef8c 00007f39cfb5eef6 000000000036eb39 00007f39d7bd21e0 00007f39d7c9b0c4 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:25:10.394481 (+   208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.395624 (+  1143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.395916 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21957504}
I20260709 12:25:10.376061  7108 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.398479  7108 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:10.399453  6778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 454) took 2498 ms (client timeout 2102 ms). Trace:
I20260709 12:25:10.374177  7153 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:10.400090  6778 rpcz_store.cc:269] 0709 12:25:07.900935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.302161 (+401226us) service_pool.cc:224] Handling call
0709 12:25:10.399139 (+2096978us) spinlock_profiling.cc:243] Waited 3.58 ms on lock 0x7b5400004e54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:10.399317 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8237440}
I20260709 12:25:10.402768  7153 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:10.373987  7203 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:10.403872  7203 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:10.407505  6793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 347) took 1688 ms (client timeout 1710 ms). Trace:
W20260709 12:25:10.414594  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 228242us
W20260709 12:25:10.407984  6793 rpcz_store.cc:269] 0709 12:25:08.718764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.718894 (+   130us) service_pool.cc:224] Handling call
0709 12:25:10.407273 (+1688379us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400007654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:10.407436 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2892732672}
I20260709 12:25:10.384799  7033 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:10.418999  7033 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:10.410260  7209 raft_consensus.cc:697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:10.448873  6766 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
W20260709 12:25:10.450160  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 116219us
I20260709 12:25:10.451962  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:10.433503  7037 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:10.434875  7193 raft_consensus.cc:1081] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Signalling peer 6add331326ca47038308a447bf798fa7 to start an election
I20260709 12:25:10.459371  7095 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:10.452857  7205 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:25:10.463099  7209 consensus_queue.cc:237] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:10.483738  7192 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:10.491698  7192 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:10.282480  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:10.362152  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.541848  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 519) took 1400 ms. Trace:
I20260709 12:25:10.542725  5867 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:10.282076  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.542640  5869 rpcz_store.cc:276] 0709 12:25:09.136773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.599075 (+462302us) service_pool.cc:224] Handling call
0709 12:25:09.599537 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:09.918721 (+319184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.918753 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:09.950439 (+ 31686us) log.cc:844] Serialized 26 byte log entry
0709 12:25:10.158877 (+208438us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:10.158933 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.159033 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.247723 (+ 88690us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:10.536479 (+288756us) raft_consensus.cc:1685] finished
0709 12:25:10.536589 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.537328 (+   739us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1372,"raft.queue_time_us":136244,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":7730,"spinlock_wait_cycles":9540992,"thread_start_us":494670,"threads_started":2,"wal-append.queue_time_us":199322}
I20260709 12:25:10.545687  5870 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
W20260709 12:25:10.552007  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 99973.9us
I20260709 12:25:10.554149  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:10.557535  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:10.562716  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 261541us
W20260709 12:25:10.567835  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 320354us
I20260709 12:25:10.571162  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:10.454195  6778 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:25:10.575095  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 462) took 1056 ms. Trace:
I20260709 12:25:10.575577  6778 rpcz_store.cc:276] 0709 12:25:09.518768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.401480 (+882712us) service_pool.cc:224] Handling call
0709 12:25:10.430729 (+ 29249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.574307 (+143578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.574768 (+   461us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:10.580247  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 471) took 1059 ms. Trace:
I20260709 12:25:10.581270  6756 rpcz_store.cc:276] 0709 12:25:09.521034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.577806 (+1056772us) service_pool.cc:224] Handling call
0709 12:25:10.578802 (+   996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.579497 (+   695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.579538 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:10.579635 (+    97us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:10.579728 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.579824 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.579989 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:10.585050  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.586366  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.587392  6020 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:10.560748  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:10.580399  5987 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:10.589327  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:10.594810  6019 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:10.598284  7140 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.599742  7140 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:10.588007  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:10.606462  7120 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:10.607364  7120 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:10.607945  7120 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.608744  7155 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:10.610375  7205 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:10.611081  7205 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:10.611595  7205 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.614188  7050 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:10.616245  7095 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:10.617367  7030 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:10.618191  7030 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.618587  7030 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:10.618505  7203 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:10.630596  7140 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:10.632404  7095 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:10.632961  7095 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:10.633421  7095 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.637247  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 791) took 1542 ms. Trace:
I20260709 12:25:10.640832  7194 consensus_queue.cc:1059] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:10.616264  7161 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.639658  6611 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:10.653805  7130 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:10.655879  6019 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
W20260709 12:25:10.658111  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 97681.6us
I20260709 12:25:10.659373  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 798) took 1562 ms. Trace:
I20260709 12:25:10.660163  5949 rpcz_store.cc:276] 0709 12:25:09.096359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.927310 (+830951us) service_pool.cc:224] Handling call
0709 12:25:09.927587 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.191937 (+264350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.658679 (+466742us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54006761d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:10.659113 (+   434us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590691584}
I20260709 12:25:10.664598  7203 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:10.646046  7161 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:10.637845  7155 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:10.667490  7155 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:10.668151  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 524) took 1492 ms. Trace:
I20260709 12:25:10.667491  7161 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:10.669991  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.669601  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 73) took 1565 ms. Trace:
I20260709 12:25:10.675006  5941 rpcz_store.cc:276] 0709 12:25:09.103770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.982908 (+879138us) service_pool.cc:224] Handling call
0709 12:25:09.983388 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.182171 (+198783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.655415 (+473244us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54006725d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:10.669357 (+ 13942us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449985792}
I20260709 12:25:10.659475  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:10.637930  5948 rpcz_store.cc:276] 0709 12:25:09.094367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.710538 (+616171us) service_pool.cc:224] Handling call
0709 12:25:09.710916 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.749675 (+ 38759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.749728 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:09.749794 (+    66us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:09.749843 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.636433 (+886590us) spinlock_profiling.cc:243] Waited 38.4 ms on lock 0x7b540058fa54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:10.636545 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.636911 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88391040}
W20260709 12:25:10.684021  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44655.6us
I20260709 12:25:10.678229  5868 rpcz_store.cc:276] 0709 12:25:09.175674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.604480 (+428806us) service_pool.cc:224] Handling call
0709 12:25:09.698710 (+ 94230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.698992 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.699036 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:09.699114 (+    78us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:09.699162 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.699242 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.667603 (+968361us) spinlock_profiling.cc:243] Waited 93.5 ms on lock 0x7b540069e950. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:10.667907 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215156992}
I20260709 12:25:10.657394  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 564) took 1575 ms. Trace:
I20260709 12:25:10.691323  7165 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.693588  7040 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:10.697856  7209 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:10.698774  7209 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:10.699355  7209 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:10.700246  7040 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:10.701659  6611 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:10.691556  7203 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:10.701917  7050 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:25:10.710456  7037 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:10.736461  7165 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:10.721315  7203 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:10.738440  7203 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:10.711174  7134 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.711402  6019 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:10.719278  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:10.688751  5942 rpcz_store.cc:276] 0709 12:25:09.081766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.081921 (+   155us) service_pool.cc:224] Handling call
0709 12:25:09.082293 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.194275 (+111982us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.196102 (+  1827us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:09.200746 (+  4644us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:09.206392 (+  5646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.656663 (+1450271us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400673ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:10.656754 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.657054 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241967360}
I20260709 12:25:10.731495  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 367) took 1247 ms. Trace:
I20260709 12:25:10.757354  6785 rpcz_store.cc:276] 0709 12:25:09.483349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.117765 (+634416us) service_pool.cc:224] Handling call
0709 12:25:10.163508 (+ 45743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.176014 (+ 12506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.177707 (+  1693us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:10.179439 (+  1732us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:10.188684 (+  9245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.195617 (+  6933us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.730861 (+535244us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b5400472a50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:10.731187 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32331008}
I20260709 12:25:10.759538  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:10.760136  5970 debug-util.cc:398] Leaking SignalData structure 0x7b080027f060 after lost signal to thread 2070
I20260709 12:25:10.733611  7154 consensus_queue.cc:1059] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:10.735184  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 75939.1us
I20260709 12:25:10.737176  7037 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:10.710961  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:10.755110  6768 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 352) took 1337 ms (client timeout 1510 ms). Trace:
I20260709 12:25:10.746271  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.747334  6773 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:25:10.743259  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51546.2us
I20260709 12:25:10.765425  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.765807  7154 raft_consensus.cc:3055] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:10.767012  7120 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:10.769752  7161 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.741554  7008 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:10.786451  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 343) took 1286 ms. Trace:
I20260709 12:25:10.748692  7165 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:10.789398  6773 rpcz_store.cc:276] 0709 12:25:09.500515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.115404 (+614889us) service_pool.cc:224] Handling call
0709 12:25:10.725191 (+609787us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b580001f898. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:25:10.746374 (+ 21183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.782696 (+ 36322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.786352 (+  3656us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5562240}
I20260709 12:25:10.793818  6019 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:10.790530  7161 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:10.777228  6768 rpcz_store.cc:269] 0709 12:25:09.417991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.718419 (+1300428us) service_pool.cc:224] Handling call
0709 12:25:10.755010 (+ 36591us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:10.799993  7161 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:10.773790  6020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be"
dest_uuid: "6add331326ca47038308a447bf798fa7"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:48972
I20260709 12:25:10.802364  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 468) took 1281 ms. Trace:
I20260709 12:25:10.802604  6609 leader_election.cc:400] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:10.803207  6609 leader_election.cc:403] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:25:10.802381  6020 raft_consensus.cc:606] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:10.802944  6766 rpcz_store.cc:276] 0709 12:25:09.520279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.453887 (+933608us) service_pool.cc:224] Handling call
0709 12:25:10.456533 (+  2646us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:10.458020 (+  1487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.458053 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:10.572572 (+114519us) log.cc:844] Serialized 26 byte log entry
0709 12:25:10.687967 (+115395us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:10.692997 (+  5030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.696864 (+  3867us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.801786 (+104922us) raft_consensus.cc:1685] finished
0709 12:25:10.801820 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.802080 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3253,"raft.queue_time_us":1150,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":1958,"thread_start_us":104800,"threads_started":1,"wal-append.queue_time_us":177883}
W20260709 12:25:10.806504  7235 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:25:10.812441  7156 raft_consensus.cc:3055] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:10.813125  7156 raft_consensus.cc:740] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:10.814757  7159 raft_consensus.cc:3055] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:10.815258  7159 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:10.816494  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:10.817544  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:10.818972  7024 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:25:10.791039  5970 debug-util.cc:398] Leaking SignalData structure 0x7b08000454e0 after lost signal to thread 5731
I20260709 12:25:10.819705  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:10.820278  7158 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:10.821183  7024 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:10.802745  6785 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:10.827236  7169 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:10.853081  7205 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:10.854243  5970 debug-util.cc:398] Leaking SignalData structure 0x7b0800322ee0 after lost signal to thread 5899
I20260709 12:25:10.854701  6766 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:10.858104  7008 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:10.858534  7205 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:10.831483  7185 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.128s	user 0.003s	sys 0.001s
W20260709 12:25:10.834184  7196 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.912s	user 0.000s	sys 0.002s
I20260709 12:25:10.833990  6015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39"
dest_uuid: "6add331326ca47038308a447bf798fa7"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:48972
I20260709 12:25:10.861510  6015 raft_consensus.cc:606] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:10.861935  6015 raft_consensus.cc:614] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:10.799082  7037 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:10.786356  7154 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:10.805416  6609 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:10.846087  7040 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:10.886468  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:10.848604  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.888355  6018 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc"
dest_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:37112
I20260709 12:25:10.888396  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:25:10.889314  6018 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:10.889899  6018 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.849709  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:10.846203  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.890775  7154 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:10.852502  6785 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:25:10.893015  6785 pending_rounds.cc:85] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 2
I20260709 12:25:10.893536  6785 pending_rounds.cc:107] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:25:10.893617  7154 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.894029  6785 raft_consensus.cc:2889] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:10.895996  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:10.865947  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.843430  7024 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:10.898272  7024 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.866855  6768 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:10.816494  7159 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:10.902377  7141 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:10.844847  7134 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:10.899183  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:10.868284  7156 consensus_queue.cc:260] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:10.871218  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 337) took 1401 ms. Trace:
I20260709 12:25:10.910455  6775 rpcz_store.cc:276] 0709 12:25:09.469554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.083690 (+614136us) service_pool.cc:224] Handling call
0709 12:25:10.084057 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.172047 (+ 87990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.870705 (+698658us) spinlock_profiling.cc:243] Waited 9.6 ms on lock 0x7b5400003a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:10.871011 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22078848}
I20260709 12:25:10.912335  5990 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:10.912667  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:10.914366  7040 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:10.915015  6775 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 4: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:10.916528  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:10.873893  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:10.874526  7216 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.142s	user 0.004s	sys 0.000s
W20260709 12:25:10.917450  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 43775.4us
I20260709 12:25:10.918191  7040 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:10.919027  7040 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:10.919657  7040 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:10.872742  7030 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:10.876657  7209 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:10.879619  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:25:10.856431  5970 debug-util.cc:398] Leaking SignalData structure 0x7b08001ef000 after lost signal to thread 5973
I20260709 12:25:10.896804  6785 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:10.907099  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:10.908207  7134 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:10.860307  7108 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:10.921725  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.930400  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:10.931180  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 125993us
I20260709 12:25:10.930971  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.931761  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 601) took 1398 ms. Trace:
I20260709 12:25:10.957876  6021 rpcz_store.cc:276] 0709 12:25:09.533658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.542296 (+  8638us) service_pool.cc:224] Handling call
0709 12:25:09.929299 (+387003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.110367 (+181068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.931241 (+820874us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b5400160650. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:10.931522 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":738416896}
I20260709 12:25:10.931977  7138 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:10.932088  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:10.934504  7120 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:10.935514  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 472) took 1414 ms. Trace:
I20260709 12:25:10.922698  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:10.882661  7050 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:25:10.948328  7156 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:10.949044  6022 raft_consensus.cc:2368] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4.
I20260709 12:25:10.951611  7209 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:10.953533  7146 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:10.933290  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.960024  5869 raft_consensus.cc:2376] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3. Re-sending same reply.
I20260709 12:25:10.972204  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.903736  7203 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:10.981493  6611 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:10.921840  7095 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:10.948354  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:10.944322  6775 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:10.991050  6785 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:10.991751  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 813) took 1009 ms. Trace:
W20260709 12:25:10.991343  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.970006  6778 rpcz_store.cc:276] 0709 12:25:09.521294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.933825 (+1412531us) service_pool.cc:224] Handling call
0709 12:25:10.934492 (+   667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.934961 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.934998 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:10.935098 (+   100us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:10.935173 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.935252 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.935429 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:10.994432  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:10.995313  5873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 57) took 1948 ms (client timeout 1563 ms). Trace:
I20260709 12:25:10.995945  7191 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:10.998135  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:11.001983  6768 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:11.002236  7146 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:11.004828  7193 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:11.007366  7050 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:11.007931  7050 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:11.008316  7050 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:11.008661  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:11.011777  7095 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:10.995786  7108 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:11.012972  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:10.978518  7008 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:25:11.017505  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:11.038401  6757 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:11.039315  5948 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:25:10.995918  5873 rpcz_store.cc:269] 0709 12:25:09.047214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.047761 (+   547us) service_pool.cc:224] Handling call
0709 12:25:10.994860 (+1947099us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54004a0454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:10.995085 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266701184}
I20260709 12:25:11.043462  6773 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:11.046003  7203 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:11.033668  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:11.029552  7030 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:11.048947  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:10.995448  6778 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:11.027984  7160 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:11.023277  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:11.055332  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:11.057014  7163 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:11.059767  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:11.062398  7159 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:11.070186  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.103471  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 224084us
I20260709 12:25:11.112520  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:11.112617  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 450) took 1574 ms. Trace:
I20260709 12:25:11.114285  6017 rpcz_store.cc:276] 0709 12:25:09.538113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.542388 (+  4275us) service_pool.cc:224] Handling call
0709 12:25:11.048535 (+1506147us) spinlock_profiling.cc:243] Waited 7.52 ms on lock 0x7b5800480098. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:11.112521 (+ 63986us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17299328}
I20260709 12:25:11.114610  7194 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:11.115252  5942 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
W20260709 12:25:11.117956  6016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 443) took 1605 ms (client timeout 2077 ms). Trace:
W20260709 12:25:11.118526  6016 rpcz_store.cc:269] 0709 12:25:09.512778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.059205 (+1546427us) service_pool.cc:224] Handling call
0709 12:25:11.117859 (+ 58654us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:11.122012  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 127761us
I20260709 12:25:11.122048  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:11.123447  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:11.126031  7163 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:11.125233  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:11.130321  7163 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:11.130609  7146 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:11.131479  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 154056us
I20260709 12:25:11.133899  7203 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:11.134495  7203 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.134923  7203 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:10.992386  5947 rpcz_store.cc:276] 0709 12:25:09.830753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.071652 (+240899us) service_pool.cc:224] Handling call
0709 12:25:10.072072 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:10.072550 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.072588 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:10.640862 (+568274us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b2c001b1e88. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:25:10.668911 (+ 28049us) log.cc:844] Serialized 26 byte log entry
0709 12:25:10.778659 (+109748us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:10.778731 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.778821 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.781741 (+  2920us) raft_consensus.cc:1685] finished
0709 12:25:10.781768 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.840479 (+ 58711us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":494,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":115,"spinlock_wait_cycles":44865920,"thread_start_us":84135,"threads_started":1,"wal-append.queue_time_us":90473}
I20260709 12:25:11.140156  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:11.140745  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:11.141414  5871 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:11.142798  5970 debug-util.cc:398] Leaking SignalData structure 0x7b0800325fa0 after lost signal to thread 6048
W20260709 12:25:11.144380  5970 debug-util.cc:398] Leaking SignalData structure 0x7b08001065a0 after lost signal to thread 6829
I20260709 12:25:11.155782  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:11.159883  5990 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:11.161511  7194 consensus_queue.cc:1059] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:25:11.164769  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:11.165853  6022 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:11.159207  6018 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:11.169828  7151 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:11.170418  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:11.173269  7152 raft_consensus.cc:3055] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:11.183225  7152 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:11.184551  7152 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:11.186022  6018 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:11.186484  7152 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:11.176716  6757 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:11.178433  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:11.154567  7146 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:11.196835  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:11.181671  7053 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:11.190917  7151 raft_consensus.cc:697] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:25:11.206230  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 284790us
I20260709 12:25:11.204399  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:11.207412  7151 consensus_queue.cc:237] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:11.208102  5987 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:11.217579  6611 leader_election.cc:400] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4.
I20260709 12:25:11.218195  6611 leader_election.cc:403] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 12:25:11.219547  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 96314.1us
I20260709 12:25:11.227824  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:11.214891  7130 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:11.214277  7160 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:11.237142  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:11.225311  7244 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:11.241906  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:11.244683  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 466) took 1692 ms. Trace:
I20260709 12:25:11.221521  7053 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
W20260709 12:25:11.246184  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 67893.4us
I20260709 12:25:11.234416  7146 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4.
I20260709 12:25:11.247365  7095 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:11.246031  6793 rpcz_store.cc:276] 0709 12:25:09.519796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.416293 (+896497us) service_pool.cc:224] Handling call
0709 12:25:10.727351 (+311058us) spinlock_profiling.cc:243] Waited 8.23 ms on lock 0x7b580001f898. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:25:10.727901 (+   550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:10.931289 (+203388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.931318 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:10.931927 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 12:25:11.109288 (+177361us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:11.109335 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.110053 (+   718us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54005758d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:11.110109 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:11.208536 (+ 98427us) raft_consensus.cc:1685] finished
0709 12:25:11.208558 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.212627 (+  4069us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21676,"raft.queue_time_us":23154,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":157,"spinlock_wait_cycles":435485824,"thread_start_us":176871,"threads_started":1,"wal-append.queue_time_us":225002}
I20260709 12:25:11.249436  6018 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:11.243798  7244 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:11.231444  7024 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:11.260178  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:11.240206  5693 catalog_manager.cc:5697] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 1 to 2, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 2 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:11.264089  6015 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:11.257997  7244 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:11.271943  5693 catalog_manager.cc:5697] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 3, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 3 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:11.280927  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:11.265866  7030 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:25:11.287619  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 50557.2us
I20260709 12:25:11.275022  7160 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:11.287240  7030 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:11.293907  5693 catalog_manager.cc:5697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 3, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 3 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:11.310371  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68489.4us
W20260709 12:25:11.312317  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 54428.2us
I20260709 12:25:11.313922  7146 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:11.314702  5915 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:11.315228  7146 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:11.315820  7146 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:11.316862  7169 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:11.314329  7024 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:11.318092  7024 raft_consensus.cc:2764] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:25:11.322255  7134 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:11.328431  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:11.330950  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:11.331789  6793 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:11.332101  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:11.333012  5948 raft_consensus.cc:2376] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3. Re-sending same reply.
I20260709 12:25:11.317422  7169 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:11.335372  7169 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:11.336694  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:11.339697  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:11.340819  5948 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:11.302861  7030 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:11.336182  7203 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:11.316905  7146 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:11.344883  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:11.346493  5948 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:11.346603  7146 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:11.344252  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:11.347167  7154 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:11.347851  7154 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:11.350334  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:11.351490  7158 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:11.351763  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:11.352530  6016 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:11.355108  6023 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:11.357301  7160 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:11.358893  7206 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:11.307451  7095 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 12:25:11.362993  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:11.364002  5865 raft_consensus.cc:2376] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3. Re-sending same reply.
I20260709 12:25:11.365437  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:11.366372  5867 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:25:11.366757  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:11.367851  5866 raft_consensus.cc:2368] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:11.366496  7154 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:11.368105  5869 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:11.371083  5945 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:11.371886  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:11.366119  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:11.373477  5870 raft_consensus.cc:2368] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:11.375756  7206 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:11.376753  7169 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:11.383587  7203 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:11.384544  7095 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:11.436838  5940 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:11.387620  7040 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:11.388422  5874 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:11.389496  5693 catalog_manager.cc:5697] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 3, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 3 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:11.439808  5693 catalog_manager.cc:5697] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 3, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 3 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:11.391580  7172 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:11.391934  7050 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:11.397538  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:11.403995  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:11.408088  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:11.411893  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:11.406148  7206 consensus_queue.cc:1059] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:25:11.397066  7247 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.152s
I20260709 12:25:11.429888  7163 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:11.458416  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:11.462466  5874 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:25:11.450681  5871 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
W20260709 12:25:11.436337  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.445083  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 47674.7us
I20260709 12:25:11.465253  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:11.467293  7206 consensus_queue.cc:1243] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:25:11.465197  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:11.441515  5693 catalog_manager.cc:5697] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 3, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 3 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:11.459507  7040 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:11.458930  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:11.468084  6018 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:11.470163  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 190784us
I20260709 12:25:11.432746  7030 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:11.451048  7156 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:11.452540  6016 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:11.522104  5693 catalog_manager.cc:5697] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 3, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 3 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:11.534353  6611 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:11.535631  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:11.537104  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:11.517611  7154 raft_consensus.cc:3055] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:11.462396  7152 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 12:25:11.537402  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:11.540831  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:11.540882  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:11.542505  5944 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:11.518477  7130 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:25:11.544396  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:11.545256  5866 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:11.547001  5865 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:11.547138  7050 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:11.547863  5840 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:11.545414  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:11.549731  5693 catalog_manager.cc:5697] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 1 to 2, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 2 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:11.551052  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:11.530544  7156 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.533357  5870 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:11.555908  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:11.556289  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:11.558593  5869 raft_consensus.cc:2368] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4.
I20260709 12:25:11.563423  5693 catalog_manager.cc:5697] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 4 to 5, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 5 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:11.533380  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:11.565850  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:11.539868  7154 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:11.549105  7169 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:11.548919  7194 consensus_queue.cc:1059] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 12:25:11.560602  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:11.558907  7156 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:11.563867  7095 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:11.586366  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:11.589504  5873 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 1. Current term is 2. Ops: []
W20260709 12:25:11.590373  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.590667  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:11.574230  7193 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:11.597908  5874 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:11.617481  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 299791us
I20260709 12:25:11.627748  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:11.629014  5871 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:11.609246  7194 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.639357  7194 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:11.645823  5870 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:11.588043  7040 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:25:11.620034  7170 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.092s
I20260709 12:25:11.687726  7203 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:11.723389  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:11.698309  7108 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:11.728695  7108 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:11.731428  7249 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:25:11.691752  7172 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.732528  7172 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:11.734802  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:11.736614  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:11.655167  7154 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:11.739277  7154 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:11.739493  7159 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:11.740309  7159 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:11.744107  7024 raft_consensus.cc:993] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: : Instructing follower 1fc57407d56241b38a94d543e073f8af to start an election
I20260709 12:25:11.745038  7024 raft_consensus.cc:1081] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Signalling peer 1fc57407d56241b38a94d543e073f8af to start an election
I20260709 12:25:11.745301  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:11.766103  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 485235us
W20260709 12:25:11.780166  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 65029.9us
I20260709 12:25:11.795468  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:11.783090  5913 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:11.800242  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:11.762080  7160 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.802863  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:11.768031  7169 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.710389  7247 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.092s
I20260709 12:25:11.778551  7030 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:11.768244  7194 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:11.812616  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:11.771946  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:11.828917  7247 raft_consensus.cc:3055] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:11.829427  7247 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:11.829519  5940 raft_consensus.cc:2376] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3. Re-sending same reply.
I20260709 12:25:11.830582  7247 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:11.832516  7247 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:11.826033  5946 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:11.841909  7160 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:11.845150  7160 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:11.824714  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:11.825902  7169 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:11.827823  7250 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:11.845784  5944 raft_consensus.cc:2376] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4. Re-sending same reply.
I20260709 12:25:11.846884  7163 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:11.850020  7172 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:11.851068  7158 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.854856  5874 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:11.862174  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:11.863595  7151 raft_consensus.cc:3055] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:25:11.820441  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 424855us
I20260709 12:25:11.885484  5947 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:11.751258  7159 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:11.800927  7040 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
W20260709 12:25:11.898475  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 73938.4us
W20260709 12:25:11.902923  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 355124us
I20260709 12:25:11.904737  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:11.886483  7250 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:11.913132  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:11.905594  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:11.888620  7095 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:11.894688  7024 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:11.899390  7108 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:11.896595  7151 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:11.916491  7250 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:11.950747  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:11.903653  5941 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:11.954883  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:11.957363  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:11.894464  7158 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:11.915797  7163 consensus_queue.cc:1059] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:25:11.929210  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:11.973556  6022 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:11.969614  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:11.976212  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:25:11.986260  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 397095us
I20260709 12:25:11.937438  7030 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:25:11.960817  7251 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:12.015400  6757 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 4: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:25:11.990729  7244 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.700s	user 0.012s	sys 0.004s
W20260709 12:25:12.024811  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 438532us
I20260709 12:25:12.031064  7172 raft_consensus.cc:3055] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:12.031662  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:12.031739  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:12.012938  6017 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:12.032915  6757 raft_consensus.cc:2410] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:12.033056  6778 raft_consensus.cc:2393] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 2: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:12.013453  7024 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:11.974282  7146 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:12.031989  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:11.969954  7160 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:25:11.991649  7159 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.015945  6775 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:43762
I20260709 12:25:12.031126  6609 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:11.990285  7130 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:12.044298  6768 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:12.046185  7206 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:12.031620  7172 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:12.019896  7258 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:12.047312  7259 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.194s	user 0.000s	sys 0.002s
I20260709 12:25:12.049924  7151 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:12.038116  7244 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:12.036181  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.018141  7251 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:12.125245  7251 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.127549  7193 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:12.128360  7244 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:12.128253  7193 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:12.131577  7169 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:12.131938  7193 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:12.090137  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 307524us
W20260709 12:25:12.088879  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 230034us
I20260709 12:25:12.134245  6022 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:12.134589  7172 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:12.135344  5838 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:12.136264  7172 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:12.084120  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 179468us
I20260709 12:25:12.143445  7151 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.133248  5913 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:12.148150  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:12.083271  7130 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:12.074452  7108 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.152482  7271 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:12.074739  7206 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:12.082227  7258 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.086723  7152 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:12.016165  7095 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:12.086277  7134 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:12.122820  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 91847.8us
I20260709 12:25:12.069367  7146 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:12.174647  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:12.175562  5941 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:12.176118  7146 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:12.081326  7156 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:12.178912  6756 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:12.180053  7156 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:12.160600  7258 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:12.180737  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:12.149977  7275 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:12.035063  7158 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:12.181850  6756 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:12.076084  6775 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:12.182557  6775 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.183351  6756 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 2. Current term is 3. Ops: []
W20260709 12:25:12.154871  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.107935  7263 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.119769  7170 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.184768  7170 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:12.186008  7134 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.186614  7134 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:12.152519  7130 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:12.120183  7163 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:12.187227  7170 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:12.134435  6778 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:12.189365  6778 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:12.189945  7152 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:12.190209  7194 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:12.190778  7194 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:12.191144  7152 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:12.191694  7194 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:12.192250  6778 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:12.193113  7194 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:12.172859  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:12.193903  5913 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:12.184711  7263 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:12.172472  5942 raft_consensus.cc:3055] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:12.194726  5942 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:12.195879  6768 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:12.195763  5942 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:12.196544  7263 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:12.194049  6785 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:25:12.176942  7206 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:12.196684  7158 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:12.153498  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:25:12.199342  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.197552  5913 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:25:12.169574  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 203581us
W20260709 12:25:12.202158  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 166929us
I20260709 12:25:12.204535  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:12.153206  7271 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.251734  5838 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:12.253201  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:12.257527  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:12.259207  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 61754.5us
I20260709 12:25:12.259338  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:12.259522  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 55122.9us
I20260709 12:25:12.261422  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:12.196938  5942 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.197921  7206 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.152872  7108 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:12.276217  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 122955us
I20260709 12:25:12.249850  6762 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:12.261570  7271 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:12.252658  7263 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:12.262521  7266 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:12.197996  7279 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.078s
I20260709 12:25:12.273458  7275 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:25:12.321074  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 366253us
I20260709 12:25:12.285650  7146 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:12.321614  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 152489us
I20260709 12:25:12.293751  7008 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:12.293432  7169 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:12.323100  7160 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:12.306730  6017 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:12.293422  7024 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:12.311237  7266 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:12.354521  6775 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:12.366837  5840 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:12.368803  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:12.381021  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:12.354629  7289 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:25:12.323946  7275 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:12.372001  5942 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:12.385651  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:12.330279  7008 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:25:12.387449  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 116964us
I20260709 12:25:12.363901  7160 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:12.412132  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.417755  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 140511us
I20260709 12:25:12.380957  7266 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:12.420639  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:12.421434  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 88706.7us
I20260709 12:25:12.412592  7008 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:12.338692  7265 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.087s
I20260709 12:25:12.406076  7275 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.416097  7160 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:12.376983  7146 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:12.449071  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:12.452669  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.436076  6756 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:12.460228  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:12.465453  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:12.475116  6019 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:12.407255  7250 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:12.480278  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:12.430749  7170 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:25:12.482159  6022 raft_consensus.cc:2393] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:12.457589  6017 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:12.453244  7152 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:12.485267  6609 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:12.450572  7263 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:25:12.442967  7193 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
W20260709 12:25:12.480693  7154 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.741s	user 0.014s	sys 0.003s
I20260709 12:25:12.485539  5913 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:12.473912  7130 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:25:12.530371  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 45051.7us
I20260709 12:25:12.541996  6019 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:12.543846  5872 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:12.489037  7292 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:12.495164  6024 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:12.496250  7130 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:12.493012  7284 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:12.553900  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 68732.4us
I20260709 12:25:12.541055  7108 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:12.566932  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:12.549301  7300 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:25:12.555256  7256 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:25:12.529034  7156 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:25:12.570902  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:12.576218  6609 leader_election.cc:400] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:12.556046  7130 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.578233  7245 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.578830  7245 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:12.588718  7301 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 12:25:12.589991  7298 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:25:12.590379  7250 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:12.591779  7283 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:12.538872  7152 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:12.484360  7024 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:12.557360  7284 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.576637  6609 leader_election.cc:403] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:25:12.575466  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:12.604598  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 70461us
I20260709 12:25:12.606791  5913 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:12.608250  7303 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:12.608876  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:12.610198  7303 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:12.610925  5913 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:12.612171  7303 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:12.612943  5913 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:12.615454  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.621742  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 54473us
W20260709 12:25:12.622196  7304 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:25:12.623013  7170 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:12.623662  7170 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:12.624197  7282 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 12:25:12.626066  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:12.574442  7272 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:12.630054  6609 leader_election.cc:400] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:12.674789  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:12.675570  6609 leader_election.cc:403] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:12.676136  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 46248.4us
I20260709 12:25:12.677392  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:12.675078  5990 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:12.646878  7146 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:12.570864  7258 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:12.681819  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 69083.9us
I20260709 12:25:12.686026  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:12.659830  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.687345  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 200977us
I20260709 12:25:12.628458  7160 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:12.695258  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:12.645721  7267 raft_consensus.cc:2764] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260709 12:25:12.696679  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 235829us
W20260709 12:25:12.686805  7172 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.545s	user 0.013s	sys 0.000s
I20260709 12:25:12.594290  7300 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:12.694741  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:12.631143  7299 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.152s
W20260709 12:25:12.700451  7309 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:25:12.682520  7050 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:12.757601  6793 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:12.704370  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 97) took 1167 ms. Trace:
I20260709 12:25:12.709713  7170 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:12.715709  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:12.646479  7305 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:12.647905  7265 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 12:25:12.631924  7108 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:12.614414  7284 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:12.758426  5872 rpcz_store.cc:276] 0709 12:25:11.537313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.541221 (+  3908us) service_pool.cc:224] Handling call
0709 12:25:11.558857 (+ 17636us) spinlock_profiling.cc:243] Waited 7.4 ms on lock 0x7b58002e2f98. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:25:11.563549 (+  4692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.544440 (+980891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.703974 (+159534us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b54001441d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:12.704282 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2222719488}
W20260709 12:25:12.721199  7194 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.528s	user 0.014s	sys 0.000s
I20260709 12:25:12.738692  7163 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:12.732367  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:12.722930  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 92042.9us
W20260709 12:25:12.731161  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 54008.5us
W20260709 12:25:12.736932  7291 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:12.760713  7272 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:12.769805  7306 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:25:12.769449  7272 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:12.771983  7272 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:12.739094  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 53149.7us
W20260709 12:25:12.737818  7296 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:25:12.772716  7130 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:12.696988  7308 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:12.713135  7008 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:12.723663  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 260083us
I20260709 12:25:12.776239  5913 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:12.776705  7130 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:12.778076  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:12.737617  7245 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:12.786623  7299 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:12.788174  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:12.788897  7299 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:12.790287  7108 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:12.785313  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:12.791047  7160 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:12.793044  7308 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:12.793493  7308 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:12.793875  7308 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:12.774891  7050 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:12.796159  7050 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:12.797760  7270 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:25:12.775591  7251 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.646s	user 0.010s	sys 0.002s
I20260709 12:25:12.807852  6609 leader_election.cc:400] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4.
I20260709 12:25:12.807471  7305 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:25:12.811386  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:12.806938  7008 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:12.802170  7146 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:25:12.813978  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:12.802279  7270 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:12.815555  7251 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:12.815886  7096 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:12.816692  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:12.793833  7152 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:12.809463  6609 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:12.818954  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:12.739203  7271 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:12.830308  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:12.827713  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:12.833748  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:12.830479  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 50665.8us
W20260709 12:25:12.827750  7134 consensus_meta.cc:287] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.641s	user 0.009s	sys 0.005s
I20260709 12:25:12.841075  6020 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:12.836572  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:12.825192  7270 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.844805  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:12.846122  6757 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:12.842003  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:12.848440  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:12.849124  6019 raft_consensus.cc:3055] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:12.853188  6019 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:12.854928  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:12.855650  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:12.855136  6016 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:25:12.852450  7151 consensus_meta.cc:287] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.708s	user 0.008s	sys 0.007s
I20260709 12:25:12.857916  6017 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:12.852342  7134 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:12.859193  7251 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:12.860469  6768 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:12.835372  6756 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:12.839324  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:12.867053  7258 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:12.861856  6019 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:13.020702  6019 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:13.023582  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 159311us
I20260709 12:25:13.019296  6611 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:13.008582  5987 leader_election.cc:336] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Timed out: RequestConsensusVote RPC to 127.2.4.195:43329 timed out after 1.741s (SENT)
I20260709 12:25:13.028796  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:12.884611  7315 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:13.010336  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 154771us
I20260709 12:25:13.011983  7283 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:13.012264  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:13.012522  7303 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:12.847961  7300 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:12.854596  7160 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:13.036368  7008 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:13.041656  7315 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:13.042158  7008 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.042315  7315 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:13.042657  7008 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:13.013835  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 151468us
I20260709 12:25:12.866026  7152 consensus_queue.cc:1059] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:25:13.057195  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:13.013139  7134 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:13.017407  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 169033us
I20260709 12:25:13.014365  7096 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:13.057678  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:13.024169  7266 consensus_meta.cc:287] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.578s	user 0.012s	sys 0.007s
I20260709 12:25:13.066794  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:13.037163  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:13.024895  7158 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:25:13.062420  7275 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.007s
I20260709 12:25:13.066824  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:13.121748  6793 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:25:13.129796  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 63057.4us
I20260709 12:25:13.084286  6019 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
W20260709 12:25:13.127135  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 90043.6us
I20260709 12:25:13.127671  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 475) took 1180 ms. Trace:
I20260709 12:25:13.130858  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:25:13.135284  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.136216  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:13.138305  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:13.141122  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:13.142445  6016 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 2: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:13.041271  7316 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:13.028595  5945 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
W20260709 12:25:13.147842  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 80723.3us
I20260709 12:25:13.153561  6022 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 3: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:13.137413  5943 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:13.162858  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:13.165414  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:13.166955  5947 raft_consensus.cc:2410] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:13.132186  6018 rpcz_store.cc:276] 0709 12:25:11.947477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.947738 (+   261us) service_pool.cc:224] Handling call
0709 12:25:13.127438 (+1179700us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400172ad4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:13.127601 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2342491648}
I20260709 12:25:13.057528  7151 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.018147  7284 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:13.176329  5874 raft_consensus.cc:2376] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4. Re-sending same reply.
I20260709 12:25:13.174361  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:13.179724  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 48031.3us
I20260709 12:25:13.179996  7158 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:13.186514  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:13.186738  7158 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:13.187551  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:13.176049  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:13.035907  7146 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.098s
I20260709 12:25:13.116473  7247 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.179613  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:13.120256  6775 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:13.120416  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:13.027387  7265 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.117s
I20260709 12:25:13.179980  7305 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:13.187948  5947 raft_consensus.cc:2393] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 4: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:25:13.239961  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 384256us
I20260709 12:25:13.181807  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:13.179916  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:13.241317  5869 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:13.183975  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:13.183007  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:13.126746  7108 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:13.242622  5872 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:25:13.133249  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:13.134193  7193 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.179520  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:13.249282  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 118481us
I20260709 12:25:13.143976  7096 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:13.035327  7308 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:13.353878  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:13.356025  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:13.357143  5874 raft_consensus.cc:2393] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 3: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:13.033645  7311 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.138s
I20260709 12:25:13.042984  7303 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:13.128098  7266 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:13.347688  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 160209us
I20260709 12:25:13.353300  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:13.359727  5946 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:13.360749  5868 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 2: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:13.147086  7244 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 12:25:13.360601  6022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "428a50885681465a98089cb853b233b9"
dest_uuid: "6add331326ca47038308a447bf798fa7"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:48972
I20260709 12:25:13.395463  6022 raft_consensus.cc:606] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:13.396041  6022 raft_consensus.cc:614] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:13.364115  5865 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:13.364319  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:13.249282  7134 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:13.237900  7172 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:25:13.346832  5987 leader_election.cc:336] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Timed out: RequestConsensusVote RPC to 127.2.4.195:43329 timed out after 1.751s (SENT)
I20260709 12:25:13.367110  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:13.368419  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:13.364461  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:13.420692  5874 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:13.364655  5870 raft_consensus.cc:2368] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5.
W20260709 12:25:13.364959  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.118782  7283 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:13.422446  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:13.423286  5874 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:13.423972  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.397464  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:13.172333  7275 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:13.190183  7151 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:13.412606  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:13.429533  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:13.348608  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:43762 (request call id 487) took 1294 ms. Trace:
I20260709 12:25:13.246099  7247 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:13.425756  6609 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:13.439893  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:13.444208  5872 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 4: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:13.444769  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:13.432682  7289 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:25:13.446019  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:13.448697  5942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.054s	sys 0.086s
W20260709 12:25:13.451725  5942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.054s	sys 0.087s
I20260709 12:25:13.449832  5944 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 2: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:13.449676  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:13.420969  7134 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:13.358932  7193 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:13.439975  7271 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:13.558782  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:13.556666  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 111987us
I20260709 12:25:13.650588  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:13.665277  5874 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:13.446151  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:13.692273  7327 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.320s	user 0.001s	sys 0.000s
I20260709 12:25:13.693578  5868 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:13.443872  6775 rpcz_store.cc:276] 0709 12:25:11.961113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.969168 (+  8055us) service_pool.cc:224] Handling call
0709 12:25:13.041317 (+1072149us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b4c0005a200. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d0f4fa20 00007f39d0f60a29 00007f39d0f6113f 00007f39d0f252ab 00007f39d0f52f3c 00007f39d0f533fe 00007f39d11e088b 00007f39d2bdf495 00007f39d2c3030f 00007f39d2c7aa55 00007f39d7c4acd0
0709 12:25:13.246044 (+204727us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6273792}
I20260709 12:25:13.693679  5865 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:25:13.696425  7319 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.330s	user 0.001s	sys 0.000s
I20260709 12:25:13.699501  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:13.700917  5940 raft_consensus.cc:2410] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:13.644409  7096 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:13.702500  7323 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 12:25:13.708323  7266 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:13.709753  7264 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:13.710604  7264 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:13.711028  7151 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:13.712376  7321 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.686s	user 0.001s	sys 0.000s
I20260709 12:25:13.712630  7247 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:13.714164  7308 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:13.715094  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 375) took 1037 ms. Trace:
I20260709 12:25:13.715553  6773 rpcz_store.cc:276] 0709 12:25:12.677841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.735989 (+ 58148us) service_pool.cc:224] Handling call
0709 12:25:12.736475 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.736914 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.736939 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.737513 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.819999 (+ 82486us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:12.834903 (+ 14904us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.843508 (+  8605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.848162 (+  4654us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.714834 (+866672us) raft_consensus.cc:1685] finished
0709 12:25:13.714860 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.715020 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14736,"prepare.queue_time_us":10357,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":3219,"raft.queue_time_us":15602,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":2370,"thread_start_us":81946,"threads_started":1,"wal-append.queue_time_us":128832}
I20260709 12:25:13.718127  7329 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:13.718429  7270 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.858s	user 0.013s	sys 0.002s
I20260709 12:25:13.718783  7329 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:13.558883  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:13.720435  5872 raft_consensus.cc:2368] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5.
I20260709 12:25:13.724295  5871 raft_consensus.cc:2393] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 4: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:13.718889  7270 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:13.552356  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 103625us
I20260709 12:25:13.454242  7146 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:13.645911  7134 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:13.445983  7262 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.701s
I20260709 12:25:13.751782  6015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9"
dest_uuid: "6add331326ca47038308a447bf798fa7"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:48972
I20260709 12:25:13.777029  5838 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:13.750492  7264 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:13.547267  7024 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:25:13.695539  7325 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.406s	user 0.001s	sys 0.000s
I20260709 12:25:13.782567  6015 raft_consensus.cc:606] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:13.734877  6756 consensus_meta.cc:287] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.850s	user 0.014s	sys 0.000s
I20260709 12:25:13.544939  7265 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:13.645044  7328 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:13.788307  6015 raft_consensus.cc:614] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:13.783349  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:13.779104  7272 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.811237  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 632) took 1402 ms. Trace:
I20260709 12:25:13.813273  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:13.814247  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:13.787379  7311 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:13.787860  7203 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:13.823717  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:13.810281  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:13.788421  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:13.824504  5943 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:13.811862  6020 rpcz_store.cc:276] 0709 12:25:12.409282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.409447 (+   165us) service_pool.cc:224] Handling call
0709 12:25:12.409838 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.853148 (+443310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.810896 (+957748us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b54001072d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:13.811154 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":991250688}
I20260709 12:25:13.754422  6611 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:13.833314  6766 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.000s	sys 0.007s
I20260709 12:25:13.799150  6756 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:13.806350  7053 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:13.839143  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:13.734999  7206 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.820204  7311 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:25:13.834973  6766 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.001s	sys 0.007s
I20260709 12:25:13.798441  7265 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:13.840240  5949 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:13.824180  7203 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:13.852108  7203 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:13.801443  7264 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:13.837286  5940 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:13.856107  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 101792us
I20260709 12:25:13.806766  7328 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:13.834941  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 184476us
I20260709 12:25:13.874358  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:13.874882  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.821368  7272 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:13.855986  7311 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:13.884290  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 537472us
W20260709 12:25:13.885486  5869 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.638s	user 0.009s	sys 0.005s
I20260709 12:25:13.889863  5869 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:13.854879  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 377) took 1043 ms. Trace:
I20260709 12:25:13.855036  7265 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:13.876729  7267 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:25:13.910171  5987 leader_election.cc:336] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Timed out: RequestConsensusVote RPC to 127.2.4.195:43329 timed out after 1.762s (SENT)
I20260709 12:25:13.880519  7203 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:13.889711  7151 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.889062  7332 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:13.936789  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:13.847606  7206 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:13.846148  7053 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:13.950910  7335 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:25:13.951711  5987 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.195:43329 timed out after 1.942s (SENT) [suppressed 35 similar messages]
I20260709 12:25:13.957742  6609 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:13.958495  5870 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:25:13.956737  7151 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:13.967609  7053 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:13.964989  7206 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:13.970085  7303 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:13.977016  6778 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:13.980645  7315 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.938s	user 0.011s	sys 0.005s
I20260709 12:25:13.981272  7315 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:13.982935  6757 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:13.983894  7272 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:13.986393  7134 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:13.992442  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:13.993575  6020 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:13.995142  7053 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:13.996199  7053 raft_consensus.cc:3055] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:13.996585  7053 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:13.997107  7328 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:13.997727  7315 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:13.997427  7053 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:13.998015  7270 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:13.995275  5990 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:13.999344  7281 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:13.999811  7281 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.000137  7281 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:13.967331  5870 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:13.906824  7337 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:13.912762  7267 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:13.898384  6756 rpcz_store.cc:276] 0709 12:25:12.799945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.816362 (+ 16417us) service_pool.cc:224] Handling call
0709 12:25:13.843770 (+1027408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:13.982154  5865 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:14.024232  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:14.011678  7134 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:13.955338  7338 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.030612  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:14.032707  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:14.012290  7151 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:13.997141  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 490) took 1755 ms. Trace:
I20260709 12:25:14.003459  7332 raft_consensus.cc:697] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:14.004530  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 400) took 1144 ms. Trace:
W20260709 12:25:14.015314  7324 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
W20260709 12:25:14.046576  5947 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.626s	user 0.013s	sys 0.000s
W20260709 12:25:14.048434  5873 consensus_meta.cc:287] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.533s	user 0.012s	sys 0.001s
I20260709 12:25:14.070071  5873 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:14.028872  7146 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:14.042296  7331 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:14.035247  7053 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:14.030176  7193 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:14.071095  5947 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
W20260709 12:25:14.075956  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 45484.2us
I20260709 12:25:14.076664  5943 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:14.084255  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:14.088833  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:14.094174  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:14.091207  5913 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:14.099311  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 147698us
I20260709 12:25:14.076023  7146 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:14.084673  7326 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.253s	user 0.002s	sys 0.000s
I20260709 12:25:14.072510  7347 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:14.083196  7158 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.112835  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:14.129520  5870 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:14.137229  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 870) took 1318 ms. Trace:
I20260709 12:25:14.137821  5942 rpcz_store.cc:276] 0709 12:25:12.818871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.818994 (+   123us) service_pool.cc:224] Handling call
0709 12:25:12.827441 (+  8447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.834222 (+  6781us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.834255 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.840310 (+  6055us) log.cc:844] Serialized 26 byte log entry
0709 12:25:13.452259 (+611949us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:13.452318 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.452424 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:14.136885 (+684461us) raft_consensus.cc:1685] finished
0709 12:25:14.136917 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:14.137142 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17213,"raft.queue_time_us":8345,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":144,"thread_start_us":611448,"threads_started":1,"wal-append.queue_time_us":965662}
I20260709 12:25:14.145902  7290 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:14.146615  7290 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:14.148350  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:14.069715  7338 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:14.091177  7297 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
W20260709 12:25:14.151870  7339 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:25:14.153327  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 405) took 1335 ms. Trace:
I20260709 12:25:14.154080  7314 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:14.173020  5949 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:14.155997  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 100) took 1034 ms. Trace:
I20260709 12:25:14.159829  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:14.175416  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.175580  5874 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:14.156513  7008 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.113s	user 0.011s	sys 0.002s
I20260709 12:25:14.177452  7264 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:14.110303  7146 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:14.109745  7172 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:14.180045  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:14.173761  5946 rpcz_store.cc:276] 0709 12:25:13.121937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.122123 (+   186us) service_pool.cc:224] Handling call
0709 12:25:14.155780 (+1033657us) spinlock_profiling.cc:243] Waited 89.4 ms on lock 0x7b5400632754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:14.155933 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":205722496}
I20260709 12:25:14.059103  6768 rpcz_store.cc:276] 0709 12:25:12.242126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.680223 (+438097us) service_pool.cc:224] Handling call
0709 12:25:12.716714 (+ 36491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.860933 (+144219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.996778 (+1135845us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54002a0dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:13.997027 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329863040}
I20260709 12:25:14.176726  6766 rpcz_store.cc:276] 0709 12:25:12.818135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.851132 (+ 32997us) service_pool.cc:224] Handling call
0709 12:25:12.851438 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.862447 (+ 11009us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.862471 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.901909 (+ 39438us) log.cc:844] Serialized 26 byte log entry
0709 12:25:13.855602 (+953693us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:13.855653 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.855745 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:14.153102 (+297357us) raft_consensus.cc:1685] finished
0709 12:25:14.153123 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:14.153262 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7807,"raft.queue_time_us":1631,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":2801,"thread_start_us":837211,"threads_started":1,"wal-append.queue_time_us":990028}
W20260709 12:25:14.182993  7341 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:25:14.131011  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:14.194849  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:14.195547  7336 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.196075  7050 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.187549  7008 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:14.180962  7335 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:14.223192  5865 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:14.202301  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:14.225767  7335 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:14.227684  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 107) took 1095 ms. Trace:
I20260709 12:25:14.227906  7244 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:14.228809  5865 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:14.205329  6020 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:14.213593  5871 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 2: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:14.215224  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:14.176515  7314 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:14.219642  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:14.188012  7264 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.238394  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:14.224371  5940 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:14.225736  7008 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:14.201879  5945 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:14.151899  7297 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:14.234114  5874 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:14.250883  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:14.249557  7330 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:14.252823  7330 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:14.136495  7347 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:14.214102  7024 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:14.242717  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:14.206559  7172 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:14.244323  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:14.261596  5871 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:14.247501  7264 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:14.264698  5871 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:14.238983  5870 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
W20260709 12:25:14.270229  7346 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:25:14.271504  5870 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:14.272573  5872 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:14.276721  5874 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:14.277314  6766 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:14.288456  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:14.265775  7305 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.013s	user 0.011s	sys 0.004s
I20260709 12:25:14.257359  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 596) took 1085 ms. Trace:
I20260709 12:25:14.231927  7050 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:14.309078  5913 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:14.299273  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:14.249542  7297 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:14.302883  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:14.311841  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:14.312429  6022 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:14.259331  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 597) took 1089 ms. Trace:
I20260709 12:25:14.266039  7251 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:14.306665  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:14.308077  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.320490  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:14.313021  6020 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:14.262356  7024 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:14.306032  7163 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:14.307356  5940 rpcz_store.cc:276] 0709 12:25:13.170011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.732252 (+562241us) service_pool.cc:224] Handling call
0709 12:25:14.255300 (+523048us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:14.306327  7305 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.306244  7279 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:25:14.239132  7336 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:14.320067  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:14.332367  7170 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:25:14.369082  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 598) took 1198 ms. Trace:
I20260709 12:25:14.305194  7357 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:25:14.331894  7345 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.323060  7151 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.320051  7251 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:14.308151  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:14.317374  5945 rpcz_store.cc:276] 0709 12:25:13.170357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.365734 (+195377us) service_pool.cc:224] Handling call
0709 12:25:14.257661 (+891927us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:14.365785  5941 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:25:14.344183  7163 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:14.366519  7024 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:14.367504  7312 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:14.386919  5945 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:14.372833  7345 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:14.388497  5945 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:14.389297  7163 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:14.369715  5943 rpcz_store.cc:276] 0709 12:25:13.170690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.098394 (+927704us) service_pool.cc:224] Handling call
0709 12:25:14.098821 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.101371 (+  2550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.101436 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:14.110669 (+  9233us) log.cc:844] Serialized 26 byte log entry
0709 12:25:14.218690 (+108021us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:14.260965 (+ 42275us) log.cc:844] Serialized 14 byte log entry
0709 12:25:14.261245 (+   280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.261330 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:14.368768 (+107438us) raft_consensus.cc:1685] finished
0709 12:25:14.368800 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:14.369002 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1897,"prepare.queue_time_us":40483,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":52,"raft.queue_time_us":2474,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":35522,"thread_start_us":76710,"threads_started":1,"wal-append.queue_time_us":174714}
I20260709 12:25:14.311380  7331 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:14.314780  7193 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.382722  6768 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:25:14.389962  7163 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:14.078656  7267 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:14.103924  7332 consensus_queue.cc:237] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:14.394138  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 441260us
I20260709 12:25:14.396463  7338 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:14.245435  7290 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:14.426640  7290 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:14.383106  7305 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:14.341323  7172 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:14.362498  7096 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:14.434080  7096 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:14.434973  7345 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:14.437218  7251 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:14.438946  7267 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:14.439513  7169 raft_consensus.cc:3055] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:14.361969  7265 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:14.424041  7163 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:14.425220  5945 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:14.271579  7314 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:14.437446  7305 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:14.447018  7163 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:14.448251  7281 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:14.448963  7305 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:14.374475  7170 consensus_queue.cc:1243] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:25:14.405036  7331 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:14.413831  7193 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:14.068830  6785 rpcz_store.cc:276] 0709 12:25:12.816639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.816801 (+   162us) service_pool.cc:224] Handling call
0709 12:25:12.817144 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.817575 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.817605 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.827276 (+  9671us) log.cc:844] Serialized 26 byte log entry
0709 12:25:13.006250 (+178974us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:13.012455 (+  6205us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.031348 (+ 18893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.031454 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.938230 (+906776us) raft_consensus.cc:1685] finished
0709 12:25:13.938267 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.960996 (+ 22729us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34998,"prepare.queue_time_us":173,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":13124,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":10401,"thread_start_us":166363,"threads_started":1,"wal-append.queue_time_us":296610}
W20260709 12:25:14.450773  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 219267us
I20260709 12:25:14.440078  7169 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:14.379412  7151 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:14.479625  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:14.480192  7361 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:14.480288  7024 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:14.439024  7251 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:14.467753  7311 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.560s	user 0.013s	sys 0.001s
I20260709 12:25:14.430598  7330 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:14.482371  7281 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:14.370069  7297 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:14.497562  7361 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:14.513834  7264 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:14.515134  7335 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:14.515503  7361 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:14.502424  7244 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.508823  7313 raft_consensus.cc:3055] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:14.511601  7151 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:14.519400  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:14.498414  7169 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:14.502774  7311 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:14.521795  7336 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:14.534752  7169 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:14.535115  7244 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.527410  7313 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:14.532807  7297 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:14.572626  7264 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:14.578320  7146 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:14.526327  7347 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:14.580017  7193 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:14.531802  7050 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:14.580137  7265 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:14.230001  5865 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:14.228212  5866 rpcz_store.cc:276] 0709 12:25:13.132512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.134612 (+  2100us) service_pool.cc:224] Handling call
0709 12:25:13.374316 (+239704us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:13.451766 (+ 77450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.545375 (+ 93609us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:13.722108 (+176733us) log.cc:844] Serialized 26 byte log entry
0709 12:25:13.945042 (+222934us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:14.109055 (+164013us) log.cc:844] Serialized 14 byte log entry
0709 12:25:14.173019 (+ 63964us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.187239 (+ 14220us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:14.227407 (+ 40168us) raft_consensus.cc:1685] finished
0709 12:25:14.227434 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:14.227612 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9359,"prepare.queue_time_us":187902,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":26374,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":93879,"thread_start_us":294702,"threads_started":2,"wal-append.queue_time_us":307458}
W20260709 12:25:14.581305  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 61969.3us
I20260709 12:25:14.606853  7331 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:14.602998  7265 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:16.146507  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:14.611243  7146 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:14.595605  7347 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.614550  7284 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:16.147399  7265 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:16.147609  7347 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:16.150132  7265 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:14.614650  7305 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:16.151178  7305 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:16.151224  7347 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:16.152268  7305 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:16.153167  7385 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:16.153841  7305 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:16.154304  5987 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:16.155036  7385 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:16.155849  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:16.156546  7385 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:14.623736  7311 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:14.625485  7283 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:14.782835  7332 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:15.046119  7108 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.110168  7267 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:16.159307  7332 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:16.159698  7108 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:16.160017  5987 connection.cc:256] RPC call timeout handler was delayed by -1.9351267170000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
W20260709 12:25:16.160660  5987 leader_election.cc:336] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.503s (SENT)
I20260709 12:25:16.161029  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:16.161731  7332 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:16.162004  7108 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:16.162197  5987 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.535s (SENT)
W20260709 12:25:15.311591  7160 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.176s	user 0.003s	sys 0.026s
I20260709 12:25:16.162675  5987 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:16.162922  7160 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.036s	user 0.004s	sys 0.027s
I20260709 12:25:16.163251  7160 raft_consensus.cc:697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:15.333669  7373 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:15.359608  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:15.362150  7378 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:16.164319  7373 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:16.164541  7160 consensus_queue.cc:237] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:16.164950  7378 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:16.166966  7373 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:15.338742  7371 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:16.167341  7378 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:16.165493  7385 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:16.167608  7373 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:15.360882  7158 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:16.167780  7371 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:16.168522  7158 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:16.170938  7158 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:16.171058  7371 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:15.362533  5873 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 2: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:25:16.174382  5873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 571) took 2119 ms (client timeout 2121 ms). Trace:
W20260709 12:25:16.174888  5873 rpcz_store.cc:269] 0709 12:25:14.054694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.080246 (+ 25552us) service_pool.cc:224] Handling call
0709 12:25:16.174054 (+2093808us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54007350d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:16.174278 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2913687936}
I20260709 12:25:15.363186  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:16.178552  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:16.179684  6778 raft_consensus.cc:2393] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 3: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:16.180835  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:15.363873  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:15.364568  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:15.365236  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:15.367627  7379 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:16.185781  7379 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:16.164907  6756 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:15.369427  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:25:15.389429  5947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.047s	user 0.025s	sys 0.035s
W20260709 12:25:15.757218  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.399058  6793 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:16.124176  5867 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:25:16.116292  5946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.861s	user 0.024s	sys 0.018s
W20260709 12:25:15.562100  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.575228  5865 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:14.609372  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:16.192600  5867 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:25:15.458329  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.562064  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:16.111899  5913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.800s	user 0.160s	sys 0.193s
I20260709 12:25:16.132510  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:16.195629  5946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.941s	user 0.024s	sys 0.018s
W20260709 12:25:16.099632  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:16.197783  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 828393us
I20260709 12:25:15.362388  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:16.200953  7305 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:16.211243  7305 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:16.211705  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:15.432169  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:15.531742  7320 log.cc:927] Time spent T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Append to log took a long time: real 2.503s	user 0.005s	sys 0.030s
W20260709 12:25:15.332170  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.322698  7372 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.216574  7372 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:16.217814  7158 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.218498  7158 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:16.220150  7372 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:15.361431  7275 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:16.221139  7158 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:15.458523  7156 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:16.232461  7156 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:16.234782  7156 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:16.235347  7156 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.235803  7156 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:16.236742  5944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 855) took 4024 ms (client timeout 1885 ms). Trace:
I20260709 12:25:16.212654  7305 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:16.202198  5947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.857s	user 0.027s	sys 0.036s
W20260709 12:25:16.238217  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 1.91759e+06us
I20260709 12:25:16.238798  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 604) took 2245 ms. Trace:
W20260709 12:25:16.201856  5913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.890s	user 0.161s	sys 0.194s
I20260709 12:25:16.239485  7278 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:25:16.239890  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 1.9307e+06us
I20260709 12:25:14.621796  7313 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:16.160033  7267 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:16.203999  5868 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 568) took 2296 ms (client timeout 2708 ms). Trace:
I20260709 12:25:16.194567  7385 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:16.252130  5868 rpcz_store.cc:269] 0709 12:25:13.907723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.907955 (+   232us) service_pool.cc:224] Handling call
0709 12:25:14.609160 (+701205us) spinlock_profiling.cc:243] Waited 3.4 ms on lock 0x7b58002e2f98. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:16.203934 (+1594774us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7813888}
I20260709 12:25:16.106673  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:16.121783  7382 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.143987  7096 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:16.204469  7146 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:14.630357  7053 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.544s	user 0.015s	sys 0.000s
I20260709 12:25:14.587872  7264 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:16.205590  6609 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:25:16.267367  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 61848.3us
I20260709 12:25:16.205847  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:15.260942  7368 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:15.552222  7329 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:16.201993  5867 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:15.554481  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 96) took 2509 ms. Trace:
I20260709 12:25:16.207743  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:16.273303  6756 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 5: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:16.272372  7368 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:16.274470  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:15.708385  7300 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.221830  7275 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:14.606226  7351 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:16.282701  7379 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:16.282655  7351 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:16.284384  6762 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:25:15.358827  6785 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:16.225611  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:16.226318  7311 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.221861  7373 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:15.739552  7134 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.227414  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.296269  5943 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:16.227816  6773 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 3: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:16.230412  7386 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.230566  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:16.231688  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:16.257838  7158 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
W20260709 12:25:16.237200  5944 rpcz_store.cc:269] 0709 12:25:12.212313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.212436 (+   123us) service_pool.cc:224] Handling call
0709 12:25:16.236461 (+4024025us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400673ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:16.236637 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2729771776}
I20260709 12:25:16.222961  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:16.239254  5947 rpcz_store.cc:276] 0709 12:25:13.993563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.257788 (+264225us) service_pool.cc:224] Handling call
0709 12:25:14.269073 (+ 11285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:14.284692 (+ 15619us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:14.308483 (+ 23791us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.238445 (+1929962us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.238503 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:16.238548 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.238618 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.238736 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25889,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":64,"thread_start_us":1896391,"threads_started":1,"wal-append.queue_time_us":276759,"wal-append.run_cpu_time_us":1541,"wal-append.run_wall_time_us":1029848}
I20260709 12:25:16.242412  5840 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:16.243973  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.202617  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 499) took 3075 ms. Trace:
W20260709 12:25:16.307065  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 64745.8us
I20260709 12:25:16.307287  6793 rpcz_store.cc:276] 0709 12:25:13.120663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.147046 (+ 26383us) service_pool.cc:224] Handling call
0709 12:25:15.398537 (+2251491us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b580001f898. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:25:15.398814 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.195758 (+796944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.196046 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75542400}
I20260709 12:25:16.210033  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:16.252532  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 90351.8us
I20260709 12:25:16.238217  5940 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:16.269529  5942 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:16.280050  5867 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:16.282208  7300 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:16.264896  5945 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:16.222981  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.319841  5949 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:16.285821  7313 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:16.328259  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 118306us
I20260709 12:25:16.301019  7311 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:16.301963  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.301095  7386 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:16.275671  7096 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:25:16.309734  5987 leader_election.cc:336] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.585s (SENT)
I20260709 12:25:16.302822  7373 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:16.318042  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:16.213090  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:16.352066  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.321354  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:16.325654  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:16.295456  7351 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:16.334930  5943 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:25:16.343497  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.298739  7156 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:16.317606  7130 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:16.347393  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:16.354835  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.282313  7275 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:16.262347  7382 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:16.357182  6022 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:25:16.272289  5941 rpcz_store.cc:276] 0709 12:25:13.044971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.053400 (+  8429us) service_pool.cc:224] Handling call
0709 12:25:13.221381 (+167981us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.386740 (+1165359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.554126 (+1167386us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400602a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:15.554369 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2582662400}
I20260709 12:25:16.258719  7385 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:16.360189  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.406785  5942 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:16.407899  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.342535  7378 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.332084  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:16.403301  6020 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:16.329023  7053 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:25:16.405383  6022 pending_rounds.cc:85] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 2
I20260709 12:25:16.369501  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.438556  6022 pending_rounds.cc:107] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:25:16.438980  5944 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:16.406103  5943 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 4: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:16.408803  5942 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:16.412070  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.303859  7134 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:16.334872  7331 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:16.369231  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:16.439009  6022 raft_consensus.cc:2889] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:16.441679  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 72511.2us
I20260709 12:25:16.443567  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:16.342970  7283 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:16.345832  7284 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:16.408267  7156 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:16.431793  7378 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:16.434458  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:16.448379  7392 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:25:16.464229  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 91754.4us
W20260709 12:25:16.505492  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 43938.8us
I20260709 12:25:16.450215  7303 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:16.450623  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:16.530957  6023 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:16.531872  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 501) took 1145 ms. Trace:
I20260709 12:25:16.532241  6773 rpcz_store.cc:276] 0709 12:25:15.386271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.450176 (+1063905us) service_pool.cc:224] Handling call
0709 12:25:16.531794 (+ 81618us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.361403  7329 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.533964  7329 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:16.372368  5948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 99) took 3298 ms (client timeout 2153 ms). Trace:
I20260709 12:25:16.536079  5949 raft_consensus.cc:2376] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2. Re-sending same reply.
I20260709 12:25:16.452520  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:16.364670  7292 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:16.434696  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:16.357951  7158 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:16.461586  5947 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:16.368892  7130 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.537699  6766 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:25:16.538509  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:16.411810  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:16.540974  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:16.574775  6015 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:16.458717  7284 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:16.515503  7154 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:16.460314  6756 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:16.530126  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 220394us
I20260709 12:25:16.447101  7096 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:16.356148  7377 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:16.571180  6773 raft_consensus.cc:2410] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:16.572212  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:16.612617  5987 leader_election.cc:336] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.615s (SENT)
I20260709 12:25:16.613381  6762 raft_consensus.cc:2410] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:16.613343  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:16.572841  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 500) took 1187 ms. Trace:
W20260709 12:25:16.535785  5948 rpcz_store.cc:269] 0709 12:25:13.074399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.074676 (+   277us) service_pool.cc:224] Handling call
0709 12:25:13.810110 (+735434us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b5800030398. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:16.350830 (+2540720us) spinlock_profiling.cc:243] Waited 2.45 s on lock 0x7b540058fa54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:16.372265 (+ 21435us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5643542528}
I20260709 12:25:16.522341  7396 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:16.578346  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 143759us
I20260709 12:25:16.627413  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:16.580744  7371 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:16.586004  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.588447  7314 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:16.590416  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.594267  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 606) took 2575 ms. Trace:
I20260709 12:25:16.595917  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.644779  5943 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:25:16.580137  7284 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:16.545046  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.604031  7160 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:16.429755  7278 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.149s
I20260709 12:25:16.533980  7393 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:16.614495  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 107074us
I20260709 12:25:16.614322  6793 rpcz_store.cc:276] 0709 12:25:15.385906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.571893 (+1185987us) service_pool.cc:224] Handling call
0709 12:25:16.572273 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.572481 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.572509 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.572556 (+    47us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:16.572587 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.572640 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.572782 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29312}
W20260709 12:25:16.615504  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.574194  7130 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:16.646315  7264 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:16.666401  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 59991.1us
W20260709 12:25:16.667519  5987 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.669s (SENT)
W20260709 12:25:16.668332  5987 connection.cc:256] RPC call timeout handler was delayed by -1.8442267170003106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2037s for a response
W20260709 12:25:16.668686  5987 connection.cc:256] RPC call timeout handler was delayed by -1.8118267170002582s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2073s for a response
W20260709 12:25:16.668987  5987 connection.cc:256] RPC call timeout handler was delayed by -1.785726717000216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
W20260709 12:25:16.669275  5987 connection.cc:256] RPC call timeout handler was delayed by -1.7767267170002015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
I20260709 12:25:16.579339  6775 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 2: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:16.464341  7156 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:16.636498  5947 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:16.639278  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:16.568550  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:16.597652  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.612159  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.655053  6766 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:16.659114  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.629966  6778 raft_consensus.cc:2393] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 2: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:16.610067  7267 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:16.576172  7292 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.089s
I20260709 12:25:16.629848  7371 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:16.659570  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 504) took 1271 ms. Trace:
I20260709 12:25:16.661535  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:16.460994  7331 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:16.662588  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:16.669723  5987 connection.cc:256] RPC call timeout handler was delayed by -1.7686267170001884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
I20260709 12:25:16.460806  7283 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:16.607700  6757 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:16.631048  7377 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:16.671743  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.691996  7371 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:16.684643  7373 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:16.687885  6773 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:16.690641  7406 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.693604  7283 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:16.700949  6015 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
W20260709 12:25:16.702831  6757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 412) took 2409 ms (client timeout 2884 ms). Trace:
W20260709 12:25:16.703318  6757 rpcz_store.cc:269] 0709 12:25:14.293053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.295604 (+  2551us) service_pool.cc:224] Handling call
0709 12:25:16.369971 (+2074367us) spinlock_profiling.cc:243] Waited 3.04 ms on lock 0x7b580001f898. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:16.702715 (+332744us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6982016}
I20260709 12:25:16.703939  7156 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:16.662642  7160 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:16.705098  6757 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:16.706372  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 511) took 1151 ms. Trace:
I20260709 12:25:16.706664  7160 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:16.687354  7311 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:16.694046  5948 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:25:16.709172  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.687359  7378 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:16.644268  5940 rpcz_store.cc:276] 0709 12:25:14.019080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.362684 (+343604us) service_pool.cc:224] Handling call
0709 12:25:16.237964 (+1875280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.328910 (+ 90946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.594046 (+265136us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b5400673ed0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:16.594201 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4302471424}
I20260709 12:25:16.711314  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 506) took 1323 ms. Trace:
I20260709 12:25:16.712327  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.713212  5940 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:16.710487  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:16.714692  5948 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:16.695703  7331 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:16.717083  7378 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:16.695530  7373 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:16.717832  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:16.718001  7378 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:16.718614  7373 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:16.636372  7154 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:16.696252  5941 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:16.720090  7154 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:16.720749  7378 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:16.721206  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:16.721721  5942 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:25:16.721933  5941 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:16.723169  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 104) took 1169 ms. Trace:
I20260709 12:25:16.674897  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:16.723765  5949 rpcz_store.cc:276] 0709 12:25:15.553842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.722277 (+1168435us) service_pool.cc:224] Handling call
0709 12:25:16.722645 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.722822 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.722848 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.722889 (+    41us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:16.722920 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.722967 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.723114 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:16.724364  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 49616.1us
I20260709 12:25:16.725260  5949 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:16.725889  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.694123  5943 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 2: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:16.671140  7393 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:16.726470  5987 connection.cc:256] RPC call timeout handler was delayed by -1.7686267170001884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
I20260709 12:25:16.727823  7096 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:16.723237  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 107) took 1168 ms. Trace:
I20260709 12:25:17.003093  5942 rpcz_store.cc:276] 0709 12:25:15.554726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.722583 (+1167857us) service_pool.cc:224] Handling call
0709 12:25:16.722827 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.722978 (+   151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.722999 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.723032 (+    33us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:16.723057 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.723107 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.723188 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.728781  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.005899  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:17.007200  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.007795  6024 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:25:17.008035  7275 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:17.008642  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:16.730414  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.010810  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.011703  6019 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:17.012724  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:16.730569  7382 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:16.731160  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.016083  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:16.731323  7368 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:16.731776  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:17.017895  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 286187us
I20260709 12:25:17.018955  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:17.019876  7413 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:16.733183  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:16.733614  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 108) took 1175 ms. Trace:
I20260709 12:25:17.022183  5949 rpcz_store.cc:276] 0709 12:25:15.558469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.724735 (+1166266us) service_pool.cc:224] Handling call
0709 12:25:16.725088 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.733352 (+  8264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.733554 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.670346  7396 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:17.014503  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.023926  7396 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:17.026463  7396 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:16.700271  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 505) took 1312 ms. Trace:
I20260709 12:25:17.047528  6773 rpcz_store.cc:276] 0709 12:25:15.387432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.687277 (+1299845us) service_pool.cc:224] Handling call
0709 12:25:16.687677 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.700021 (+ 12344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.700216 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.965169  7329 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:17.048784  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:16.693979  6793 rpcz_store.cc:276] 0709 12:25:15.387142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.657477 (+1270335us) service_pool.cc:224] Handling call
0709 12:25:16.657930 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.658183 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.658222 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.658284 (+    62us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:16.658329 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.658405 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.658703 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.737154  6022 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:16.727340  7311 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:16.738961  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.059461  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:17.064749  6793 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:16.743230  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:16.573777  7158 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:17.066668  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:17.068286  7158 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:16.744932  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 109) took 1185 ms. Trace:
I20260709 12:25:16.700649  7406 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:17.069048  5943 rpcz_store.cc:276] 0709 12:25:15.559257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.743881 (+1184624us) service_pool.cc:224] Handling call
0709 12:25:16.744275 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.744516 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.744547 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.744621 (+    74us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:16.744655 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.744718 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.744858 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:17.071040  7406 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:16.731643  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 386) took 1367 ms. Trace:
I20260709 12:25:16.747267  7371 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:17.072546  6775 rpcz_store.cc:276] 0709 12:25:15.364180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.364312 (+   132us) service_pool.cc:224] Handling call
0709 12:25:16.731338 (+1367026us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54005294d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:16.731538 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":634910976}
I20260709 12:25:17.074231  7371 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:16.748816  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 114) took 1186 ms. Trace:
W20260709 12:25:17.074709  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:16.756881  5987 connection.cc:256] RPC call timeout handler was delayed by -1.7443267170001491s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21480000000000002s for a response
W20260709 12:25:17.075973  5987 connection.cc:256] RPC call timeout handler was delayed by -1.7087787640002716s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1869s for a response
W20260709 12:25:17.076421  5987 connection.cc:256] RPC call timeout handler was delayed by -1.7083267170000909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21880000000000002s for a response
I20260709 12:25:16.759335  5867 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
W20260709 12:25:17.076745  5987 connection.cc:256] RPC call timeout handler was delayed by -1.67682671700004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
I20260709 12:25:17.076897  5867 pending_rounds.cc:85] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 2
I20260709 12:25:16.763309  5869 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:16.733446  7300 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:16.732478  7134 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:16.719295  7373 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:17.080822  5940 rpcz_store.cc:276] 0709 12:25:15.562034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.747790 (+1185756us) service_pool.cc:224] Handling call
0709 12:25:16.748202 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.748426 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.748458 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.748524 (+    66us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:16.748554 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.748615 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.748745 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:17.082504  7373 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:16.773398  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.084182  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:16.760448  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.086040  5871 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:17.086987  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.087618  5871 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:17.088578  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:16.792119  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.089443  5868 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:17.090380  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:17.091009  5868 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:17.085417  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:16.801095  5874 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:17.093314  5874 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:16.803403  6020 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:17.093657  7292 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:16.686980  7385 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:16.808120  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.094714  5871 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:16.816944  5948 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:17.074821  5947 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:16.804790  7409 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.096722  7409 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:17.099363  7409 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:16.825304  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:16.827608  5946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.004s	sys 0.010s
W20260709 12:25:16.831233  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 203877us
I20260709 12:25:16.815593  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:16.832373  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:16.711922  6785 rpcz_store.cc:276] 0709 12:25:15.387712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.693257 (+1305545us) service_pool.cc:224] Handling call
0709 12:25:16.710599 (+ 17342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.710853 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.710897 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.710948 (+    51us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:16.710992 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.711088 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.711240 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.841979  7377 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:16.843556  7386 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:16.847561  6778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 388) took 1482 ms (client timeout 1910 ms). Trace:
I20260709 12:25:16.848913  7393 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:16.854063  5941 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:16.855129  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 502) took 1468 ms. Trace:
W20260709 12:25:16.855254  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 43904.1us
W20260709 12:25:16.854957  5945 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.520s	user 0.002s	sys 0.010s
W20260709 12:25:16.870810  7313 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.535s	user 0.015s	sys 0.001s
I20260709 12:25:16.886868  7369 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:16.888482  6611 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:16.698755  7299 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:16.927930  7130 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:16.930289  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 651117us
I20260709 12:25:16.673503  6762 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:16.948346  7154 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:16.967823  7402 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:16.706851  6757 rpcz_store.cc:276] 0709 12:25:15.555487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.704249 (+1148762us) service_pool.cc:224] Handling call
0709 12:25:16.704662 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.705573 (+   911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.706303 (+   730us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.959384  7395 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:16.688405  7278 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:17.009327  6024 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 1.
I20260709 12:25:17.030378  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.030408  5694 catalog_manager.cc:5697] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:17.032904  6768 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.003s	sys 0.016s
I20260709 12:25:17.033963  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:17.034719  5944 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:17.035862  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.050891  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:17.050664  7396 consensus_queue.cc:1059] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:17.049851  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:17.052359  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.061589  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.071763  5693 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50588 (request call id 53) took 6837 ms. Trace:
I20260709 12:25:17.076011  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:17.077126  5987 connection.cc:256] RPC call timeout handler was delayed by -1.67682671700004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
I20260709 12:25:16.696030  5865 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:17.078572  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.077266  5867 pending_rounds.cc:107] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:25:17.079411  5949 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:16.793290  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.134228  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.115049  5913 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:17.122395  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:17.178987  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.110597  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:17.240708  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 125753us
W20260709 12:25:17.241218  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 191447us
I20260709 12:25:17.210183  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:17.236945  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 348529us
W20260709 12:25:17.238505  6609 connection.cc:256] RPC call timeout handler was delayed by -1.4658704240000588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1966s for a response
I20260709 12:25:17.376504  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 615) took 1054 ms. Trace:
W20260709 12:25:17.376868  6609 connection.cc:256] RPC call timeout handler was delayed by -1.2975704240002415s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21530000000000002s for a response
I20260709 12:25:17.377018  5944 rpcz_store.cc:276] 0709 12:25:16.322084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.505799 (+183715us) service_pool.cc:224] Handling call
0709 12:25:17.376343 (+870544us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b5400467dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:17.376456 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":893064064}
I20260709 12:25:17.378575  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:17.370916  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:17.341363  5865 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 570) took 3294 ms (client timeout 2223 ms). Trace:
I20260709 12:25:17.237142  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 503) took 1850 ms. Trace:
I20260709 12:25:17.380048  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:17.238871  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.238624  5867 raft_consensus.cc:2889] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:17.380982  5873 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
W20260709 12:25:17.339157  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.241981  5913 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:17.354607  5694 catalog_manager.cc:5697] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 2 to 3, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 3 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:25:17.237068  5869 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:17.243476  6015 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:17.236443  6766 rpcz_store.cc:276] 0709 12:25:15.386570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.609313 (+1222743us) service_pool.cc:224] Handling call
0709 12:25:16.644388 (+ 35075us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.854493 (+210105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.855024 (+   531us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:17.355011  5868 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
W20260709 12:25:17.341614  5946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.081s	user 0.014s	sys 0.014s
I20260709 12:25:17.237972  6019 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 2: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:17.238325  5871 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:17.239971  6016 raft_consensus.cc:2410] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:17.237525  5942 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:17.237425  6022 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:17.379612  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:17.385022  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 142201us
I20260709 12:25:17.468956  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 620) took 1145 ms. Trace:
I20260709 12:25:17.383993  5694 catalog_manager.cc:5697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:17.470717  5942 rpcz_store.cc:276] 0709 12:25:16.323812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.004164 (+680352us) service_pool.cc:224] Handling call
0709 12:25:17.237271 (+233107us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:17.467642 (+230371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.468682 (+  1040us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b54004678d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:17.468878 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535092608}
W20260709 12:25:17.469120  5987 connection.cc:256] RPC call timeout handler was delayed by -1.2403267170002437s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27080000000000004s for a response
I20260709 12:25:17.470137  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:17.471936  5987 connection.cc:256] RPC call timeout handler was delayed by -1.0850787640001727s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2562s for a response
I20260709 12:25:17.406056  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.472751  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:17.468459  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 106) took 4336 ms. Trace:
I20260709 12:25:17.474748  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 603) took 3481 ms. Trace:
W20260709 12:25:17.475899  7316 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
W20260709 12:25:17.311832  7262 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
W20260709 12:25:17.473824  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 231928us
W20260709 12:25:17.618467  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 238899us
I20260709 12:25:17.611474  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.381058  7158 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:17.388461  6611 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:17.380298  6762 rpcz_store.cc:276] 0709 12:25:15.386845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.643574 (+1256729us) service_pool.cc:224] Handling call
0709 12:25:16.673223 (+ 29649us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:17.236818 (+563595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.237079 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:17.387143  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:17.472440  6018 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
W20260709 12:25:17.268435  6768 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.983s	user 0.008s	sys 0.021s
I20260709 12:25:17.474706  5867 rpcz_store.cc:276] 0709 12:25:13.132223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.132367 (+   144us) service_pool.cc:224] Handling call
0709 12:25:13.183331 (+ 50964us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.124021 (+2940690us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b54001a5a20. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bfadc0 00007f39d2c851c0 00007f39d2c86231 00007f39d2c82d8f 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c
0709 12:25:17.468033 (+1344012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.468348 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":599848192}
W20260709 12:25:17.646823  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 176329us
I20260709 12:25:17.374413  7300 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.240878  5945 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
W20260709 12:25:17.621510  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.237025  7402 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:17.385985  7264 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:17.624931  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:17.620292  6766 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:17.622298  6015 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 3: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:25:17.622164  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.237493  7395 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:17.237118  7278 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:17.101363  7284 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:17.677819  7284 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:17.240986  5693 rpcz_store.cc:276] 0709 12:25:10.234628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.234796 (+   168us) service_pool.cc:224] Handling call
0709 12:25:17.071666 (+6836870us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:12.484051 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:12.705080 (+221029us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:12.705160 (+    80us) write_op.cc:435] Acquired schema lock
0709 12:25:12.705200 (+    40us) tablet.cc:662] Decoding operations
0709 12:25:12.706475 (+  1275us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:12.706579 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:12.706608 (+    29us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:25:12.707601 (+   993us) tablet.cc:701] Row locks acquired
0709 12:25:12.707669 (+    68us) write_op.cc:260] PREPARE: finished
0709 12:25:12.707868 (+   199us) write_op.cc:270] Start()
0709 12:25:12.708047 (+   179us) write_op.cc:276] Timestamp: P: 1783599912707793 usec, L: 0
0709 12:25:12.708085 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:25:12.708893 (+   808us) log.cc:844] Serialized 3591 byte log entry
0709 12:25:15.530435 (+2821542us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:16.432200 (+901765us) op_driver.cc:464] REPLICATION: finished
0709 12:25:16.858217 (+426017us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b18000b1880. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:25:16.858462 (+   245us) write_op.cc:301] APPLY: starting
0709 12:25:16.858586 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:16.858862 (+   276us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:16.858924 (+    62us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:16.861230 (+  2306us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:16.861606 (+   376us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 12:25:16.861758 (+   152us) write_op.cc:312] APPLY: finished
0709 12:25:16.862240 (+   482us) log.cc:844] Serialized 70 byte log entry
0709 12:25:16.862483 (+   243us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:16.862641 (+   158us) write_op.cc:454] Released schema lock
0709 12:25:16.862708 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:16.865041 (+  2333us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:16.865083 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2415,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":100637,"apply.run_cpu_time_us":4107,"apply.run_wall_time_us":6540,"num_ops":9,"prepare.queue_time_us":392764,"prepare.run_cpu_time_us":17901,"prepare.run_wall_time_us":366532,"raft.queue_time_us":216379,"raft.run_cpu_time_us":12368,"raft.run_wall_time_us":73241,"replication_time_us":3723902,"spinlock_wait_cycles":47583488,"thread_start_us":489482,"threads_started":4,"wal-append.queue_time_us":145292}]]}
I20260709 12:25:17.112589  7371 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.364s
I20260709 12:25:17.237689  7299 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:17.680294  7299 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:17.681547  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.682142  7402 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:17.683288  7402 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:17.647379  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:17.685974  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:17.686614  5945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 102) took 2114 ms (client timeout 1924 ms). Trace:
W20260709 12:25:17.687049  5945 rpcz_store.cc:269] 0709 12:25:15.553014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.073275 (+520261us) service_pool.cc:224] Handling call
0709 12:25:17.664476 (+1591201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:17.687958  7369 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:17.688786  7393 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:17.689208  7393 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.689563  7393 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:17.689595  7130 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:17.476651  7368 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.194s
I20260709 12:25:17.375461  7134 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:17.105479  7292 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.311738  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 113) took 1750 ms. Trace:
I20260709 12:25:17.633476  5871 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:17.659543  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:17.649775  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 177099us
I20260709 12:25:17.650570  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.380837  7351 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.704537  5941 rpcz_store.cc:276] 0709 12:25:15.561661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.720840 (+1159179us) service_pool.cc:224] Handling call
0709 12:25:17.269677 (+548837us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:17.704879  7292 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:17.714290  7351 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:17.713023  6018 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:17.717897  6018 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:17.624784  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:17.718111  6785 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:25:17.718914  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 94243.8us
I20260709 12:25:17.719501  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.720305  5871 raft_consensus.cc:2368] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:17.720382  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:17.721068  6785 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:17.658234  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.718824  7303 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:17.722028  7303 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:17.721833  7377 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:17.722528  7303 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:17.722748  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.723487  5869 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:17.724112  7377 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:17.724448  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.725224  5869 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:17.660595  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.664419  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:17.726202  5872 raft_consensus.cc:2368] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5.
I20260709 12:25:17.726972  5874 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:17.726883  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.728344  5874 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:17.648484  7158 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:17.728915  7158 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:17.381503  7400 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:17.729866  7400 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.730252  7400 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:17.380151  5865 rpcz_store.cc:269] 0709 12:25:14.017326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.102309 (+ 84983us) service_pool.cc:224] Handling call
0709 12:25:15.574816 (+1472507us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b58000617d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bdf0aa 00007f39d2c61447 00007f39d2c79fe7 00007f39d2c7b7de 00007f39d2c8727c 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c
0709 12:25:17.251848 (+1677032us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443540224}
I20260709 12:25:17.618544  7329 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:17.667356  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:17.731585  7329 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.731997  7329 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:17.665520  7300 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:17.732626  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:17.381539  6775 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:17.733419  6773 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:17.667929  6793 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:17.734452  7300 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:17.734925  6775 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:17.668583  5838 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:17.736388  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 67900.5us
I20260709 12:25:17.738026  7154 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:17.738372  5838 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:17.738576  7154 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:17.664419  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:17.738617  7303 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:17.739569  7154 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:17.631840  7096 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:17.741657  7303 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:17.672946  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.742704  6020 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:17.612725  7412 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:25:17.743628  6756 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.116s	user 0.010s	sys 0.003s
I20260709 12:25:17.743783  7412 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:17.744148  6756 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
W20260709 12:25:17.744912  6756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 509) took 2353 ms (client timeout 2465 ms). Trace:
I20260709 12:25:17.620036  7305 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:17.666913  7315 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:17.238844  6778 rpcz_store.cc:269] 0709 12:25:15.365518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.180439 (+814921us) service_pool.cc:224] Handling call
0709 12:25:16.847172 (+666733us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b5400007b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:16.847448 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":821656320}
I20260709 12:25:17.675314  7278 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:17.678438  7313 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.475188  5946 rpcz_store.cc:276] 0709 12:25:13.993239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.227099 (+233860us) service_pool.cc:224] Handling call
0709 12:25:14.227494 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.227903 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.227935 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:14.254715 (+ 26780us) log.cc:844] Serialized 26 byte log entry
0709 12:25:16.200704 (+1945989us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:16.259977 (+ 59273us) log.cc:844] Serialized 14 byte log entry
0709 12:25:17.474407 (+1214430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.474495 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:17.474538 (+    43us) raft_consensus.cc:1685] finished
0709 12:25:17.474551 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:17.474690 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90319,"prepare.queue_time_us":56781,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":95228,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"thread_start_us":3102334,"threads_started":3,"wal-append.queue_time_us":907417,"wal-append.run_cpu_time_us":1834,"wal-append.run_wall_time_us":1034074}
I20260709 12:25:17.671762  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:17.624508  7373 raft_consensus.cc:3055] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:17.695402  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:17.699424  5915 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:17.778376  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 79054.7us
I20260709 12:25:17.779306  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:17.779791  6015 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 2: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:25:17.745334  6756 rpcz_store.cc:269] 0709 12:25:15.391872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.296992 (+905120us) service_pool.cc:224] Handling call
0709 12:25:17.744832 (+1447840us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:17.650199  7414 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:17.783146  7303 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:17.618844  5694 catalog_manager.cc:5697] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:25:17.712960  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.713440  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:17.785357  5873 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:17.722972  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:17.718716  6018 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:17.721149  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:17.786218  6766 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:17.786417  5871 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:17.741882  7096 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.390740  5866 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:17.797290  6018 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:17.704977  7396 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 12:25:17.746616  5987 leader_election.cc:400] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5.
I20260709 12:25:17.802857  5987 leader_election.cc:403] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:25:17.748334  6773 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:17.729485  5874 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:17.721519  5913 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:17.804473  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 83019.2us
I20260709 12:25:17.805164  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:17.669618  7264 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:17.702574  7134 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.741475  7130 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:17.820859  7278 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:17.752617  7386 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:25:17.781334  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:17.798167  7096 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:17.860992  6016 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:17.815469  7264 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:17.750387  7315 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.866111  7315 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:17.861667  6016 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:17.872112  6016 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:17.873307  6016 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:17.783602  7303 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:25:17.670801  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 45982.8us
I20260709 12:25:17.885466  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:17.787915  7284 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:17.793751  6045 debug-util.cc:398] Leaking SignalData structure 0x7b08000a6920 after lost signal to thread 2067
I20260709 12:25:17.777561  7373 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:17.811187  7371 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:17.816978  7377 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:17.905441  7377 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.906080  7377 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:17.811287  7409 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5.
I20260709 12:25:17.817736  7134 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:17.774245  7418 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.052s	user 0.000s	sys 0.005s
I20260709 12:25:17.821936  7130 raft_consensus.cc:697] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:17.831552  7395 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:17.911649  7130 consensus_queue.cc:237] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:17.912310  7395 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:17.833519  7408 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:17.836843  5874 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:25:17.838963  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 123157us
I20260709 12:25:17.915223  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:17.718112  7351 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:17.809827  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 652) took 1048 ms. Trace:
I20260709 12:25:17.753115  7305 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:17.781035  7313 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:17.882053  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 493625us
I20260709 12:25:17.887519  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:17.887444  5873 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
W20260709 12:25:17.892894  6045 debug-util.cc:398] Leaking SignalData structure 0x7b08000a6a40 after lost signal to thread 2070
I20260709 12:25:17.888643  7303 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:17.924468  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.928195  6016 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:25:17.961203  6016 pending_rounds.cc:85] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 2
I20260709 12:25:17.880048  7412 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:17.941095  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 53605.8us
I20260709 12:25:17.890667  7300 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:17.965020  7300 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:17.965473  7300 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:17.899274  7396 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:17.966758  7396 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:17.967677  7396 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:17.968889  7396 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:17.940608  7409 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:17.899672  7371 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:17.970805  7371 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:17.970237  7373 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:17.971969  7373 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:17.936664  5866 rpcz_store.cc:276] 0709 12:25:16.761879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.772049 (+ 10170us) service_pool.cc:224] Handling call
0709 12:25:17.802321 (+1030272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:17.937150  7284 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:17.960450  6611 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:17.978572  7369 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:17.979346  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:17.980600  7369 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:17.964004  7299 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:17.961715  6016 pending_rounds.cc:107] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:25:17.963073  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:17.962311  5987 leader_election.cc:400] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5.
I20260709 12:25:17.951498  6018 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:17.981225  7400 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:17.960915  7351 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:17.981556  6611 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:18.032094  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 50648.6us
I20260709 12:25:18.002355  5943 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:18.001513  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:18.038271  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:18.040387  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:18.042141  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:18.011945  5867 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 2: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:17.997793  6016 raft_consensus.cc:2889] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:18.057497  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:18.054798  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 885) took 1462 ms. Trace:
I20260709 12:25:18.060923  5943 rpcz_store.cc:276] 0709 12:25:16.592372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.070174 (+477802us) service_pool.cc:224] Handling call
0709 12:25:17.070518 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.043938 (+973420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.054561 (+ 10623us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b54003084d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:18.054738 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2142432512}
W20260709 12:25:18.063907  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 325576us
I20260709 12:25:17.988567  7369 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:18.002771  7368 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:18.068783  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.069397  5838 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:18.000031  7284 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:18.015617  6045 debug-util.cc:398] Leaking SignalData structure 0x7b0800269220 after lost signal to thread 5731
W20260709 12:25:18.072902  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 291594us
W20260709 12:25:18.074771  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 263013us
I20260709 12:25:18.079610  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:18.075218  5838 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:18.075114  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:18.007085  7305 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:18.052281  7409 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:17.997392  6785 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:18.055001  7396 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:18.082084  5838 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:18.080425  5913 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:18.013761  7351 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:18.004006  7402 consensus_queue.cc:1059] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 12:25:18.052039  7420 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:18.011906  7400 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:18.020370  7262 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.012966  7313 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:18.003788  7299 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:18.018925  7371 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:18.085268  7278 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:18.058552  7386 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:18.096786  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:18.095041  7156 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:17.990836  7300 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:18.009064  7329 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:18.011395  7393 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:18.109527  7420 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:18.106434  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 143444us
I20260709 12:25:18.169972  7371 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:18.173080  7134 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:18.174417  7393 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:18.175915  7134 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:18.110244  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.073761  7414 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:18.178037  7414 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:18.179165  7329 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:18.179044  7414 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:18.180657  5867 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:45236
I20260709 12:25:18.181314  5867 raft_consensus.cc:606] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:18.181660  5867 raft_consensus.cc:614] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:18.181877  7311 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:18.182633  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.183590  5943 raft_consensus.cc:2410] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:18.184016  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:18.161558  6045 debug-util.cc:398] Leaking SignalData structure 0x7b08002fbc60 after lost signal to thread 5899
I20260709 12:25:18.116972  7305 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:25:18.114444  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:18.193840  6766 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:18.086544  7385 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:18.014357  7292 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:18.118132  5987 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:18.195171  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 77108us
I20260709 12:25:18.196194  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:18.197391  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:18.198326  5942 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:18.119648  7278 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.076509  7315 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:18.097437  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:18.021219  7158 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:18.106069  7351 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:18.095719  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:18.112941  7130 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:18.085573  7284 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:18.120573  7412 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:18.129745  7299 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:18.117159  7262 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:18.222256  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 141920us
W20260709 12:25:18.114177  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 407146us
I20260709 12:25:18.223994  5990 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:18.188171  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:18.121835  7156 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.192466  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.228652  5946 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:18.228760  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:18.117007  7264 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:18.203420  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.157333  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:18.209733  7373 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:18.210567  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:18.164973  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:18.215741  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.237356  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:18.187654  6045 debug-util.cc:398] Leaking SignalData structure 0x7b080032b160 after lost signal to thread 5900
I20260709 12:25:18.207162  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:17.984113  7377 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:18.217085  7430 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:18.218384  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:18.183612  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.183271  7311 raft_consensus.cc:3055] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:18.227533  7385 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:25:18.247764  7292 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:18.247977  7385 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:18.248514  7385 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:18.248797  7303 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:18.154811  7420 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:18.194830  7096 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:18.222028  7299 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:18.221940  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:18.249593  7385 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:18.254370  7315 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:18.255303  7385 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:18.255406  7315 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:18.250509  7299 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.262048  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:18.263797  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:18.264523  6778 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:18.207587  5865 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:18.204841  7278 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:18.208835  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 111573us
I20260709 12:25:18.231523  5942 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:18.232366  7134 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:18.234776  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:18.213726  7316 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.217223  7329 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:18.239629  5949 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:18.241093  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:18.170426  7300 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:18.231520  7130 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.244462  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:18.318830  5868 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:18.267948  5944 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:18.271256  5874 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
W20260709 12:25:18.326145  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 93739.9us
I20260709 12:25:18.257555  7402 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:25:18.330276  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 99430.2us
I20260709 12:25:18.329178  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:25:18.245088  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.187934  7425 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 12:25:18.251639  7096 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:18.246976  7311 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:25:18.334122  6045 debug-util.cc:398] Leaking SignalData structure 0x7b08001e2740 after lost signal to thread 5973
W20260709 12:25:18.332623  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.334546  7425 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:18.206369  7404 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.335086  7425 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:18.334910  7311 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:18.335330  7329 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.335318  7404 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:18.336458  7311 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.336272  7329 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:18.334565  6045 debug-util.cc:398] Leaking SignalData structure 0x7b0800144500 after lost signal to thread 5974
I20260709 12:25:18.262342  7264 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:18.264820  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 380) took 2908 ms. Trace:
I20260709 12:25:18.225173  5990 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:18.338630  7329 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:18.339123  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 114044us
I20260709 12:25:18.339610  7305 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.321522  7316 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:18.340147  7305 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:18.243031  5867 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:18.342008  7316 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:18.315047  6766 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:18.342352  7305 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:18.342615  7373 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:18.343235  7373 raft_consensus.cc:697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:18.342756  7134 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.344616  7134 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:18.344244  7373 consensus_queue.cc:237] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:18.266598  5915 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:18.346204  7329 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:25:18.346354  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 79857.2us
I20260709 12:25:18.346658  7329 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:18.231842  7156 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:18.347133  7329 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.347995  5915 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:18.348896  7305 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:18.349303  7305 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:25:18.270586  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.295483  7369 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:18.349920  7369 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:18.349767  7305 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:18.350960  7369 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:18.352078  7369 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:18.283736  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:18.353693  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.354553  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.354861  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:18.355620  6778 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:25:18.336097  7425 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:18.246544  5866 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:25:18.338316  6045 debug-util.cc:398] Leaking SignalData structure 0x7b080038c760 after lost signal to thread 6048
I20260709 12:25:18.231020  7386 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:18.309386  7158 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:18.258561  7430 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:18.252151  7377 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:18.252418  7303 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:18.310679  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:18.370050  7425 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:18.360780  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.390762  5868 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:18.332617  7402 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:18.362578  5949 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:18.366262  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:18.355319  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:18.395175  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:18.369081  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 123424us
I20260709 12:25:18.395897  5874 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:18.396234  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:18.396872  6018 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:18.297384  7262 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:18.397845  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:18.377030  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 45488.2us
I20260709 12:25:18.375066  7386 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:18.375080  6045 debug-util.cc:398] Leaking SignalData structure 0x7b080015c0e0 after lost signal to thread 6049
W20260709 12:25:18.401980  6045 debug-util.cc:398] Leaking SignalData structure 0x7b08001fe4a0 after lost signal to thread 6829
I20260709 12:25:18.402118  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:18.363838  7435 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:18.388731  6785 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:18.306701  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:18.385929  7430 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:18.349350  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:18.360925  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:18.392557  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 63482.1us
W20260709 12:25:18.407720  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 58459.3us
I20260709 12:25:18.392922  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:18.392546  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:18.364956  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:18.407884  6609 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:18.396229  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:18.405246  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:18.323841  7130 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:18.409412  6020 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:18.410208  7373 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:18.393153  7393 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.386932  5871 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
W20260709 12:25:18.412546  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51736.6us
I20260709 12:25:18.391616  7404 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:18.418351  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:18.396716  7300 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:18.408485  5915 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:18.394697  7402 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:18.413087  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:18.403424  7386 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:18.416888  5913 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:18.404769  6045 debug-util.cc:398] Leaking SignalData structure 0x7b080022c100 after lost signal to thread 7095
I20260709 12:25:18.402750  7284 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:18.403831  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:18.415247  7303 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:18.411348  7130 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:18.454545  6020 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:18.457020  6018 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:18.411370  7373 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:18.425917  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:18.415827  7435 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:18.414551  6775 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:18.437877  7305 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:18.440512  7156 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:18.445092  7368 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:18.421610  7158 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:25:18.481359  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:18.419189  7377 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:18.427690  6757 raft_consensus.cc:2393] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 5: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:18.423067  7443 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:18.429942  7412 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:18.486325  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:18.514359  7426 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:25:18.515997  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:18.516664  7373 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:18.484942  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 77129.4us
I20260709 12:25:18.446040  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:18.426617  7427 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:18.493574  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:18.481841  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.495486  5869 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 3: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:18.496840  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:18.498677  5947 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:18.499909  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:18.557221  5942 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:25:18.557705  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 57968.3us
W20260709 12:25:18.559085  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.504537  7300 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:18.564486  5913 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:18.570647  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.571211  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.572907  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.423313  7393 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:18.512563  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.513866  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.522388  6773 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:18.514389  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:18.524788  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:18.505187  7351 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:18.583295  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 58638.2us
I20260709 12:25:18.584853  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:18.529719  7130 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:18.535230  7435 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:18.473755  7439 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:18.537055  7299 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:18.538599  7409 raft_consensus.cc:3055] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:18.542995  7156 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:18.338181  6768 rpcz_store.cc:276] 0709 12:25:15.356535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.356770 (+   235us) service_pool.cc:224] Handling call
0709 12:25:15.357177 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.284970 (+927793us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:17.669200 (+1384230us) log.cc:844] Serialized 14 byte log entry
0709 12:25:18.161563 (+492363us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:18.162592 (+  1029us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:18.167599 (+  5007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.264460 (+ 96861us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b5400004454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:18.264521 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.264749 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":574788,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":62,"raft.queue_time_us":27759,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":5685,"spinlock_wait_cycles":2133089664,"thread_start_us":1794471,"threads_started":3,"wal-append.queue_time_us":442241}
I20260709 12:25:18.548516  7305 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:18.548764  7420 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:18.549827  7313 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:18.571070  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.685544  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:18.686304  5943 raft_consensus.cc:2393] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 2: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:18.528910  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.576640  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:18.576716  6609 connection.cc:256] RPC call timeout handler was delayed by -2.2150677670001642s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2089s for a response
I20260709 12:25:18.577747  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:18.462852  6778 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:18.574347  5944 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
W20260709 12:25:18.582259  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 166324us
W20260709 12:25:18.690260  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.588653  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 95175.8us
I20260709 12:25:18.691561  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:18.592597  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:18.692659  6018 raft_consensus.cc:2410] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:18.692430  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:18.533082  7329 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:18.476442  7284 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.120s
W20260709 12:25:18.631726  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.622169  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.602248  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:18.603228  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.660547  6773 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:18.660778  6793 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:18.606418  6611 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:18.753015  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 146698us
I20260709 12:25:18.657217  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:18.662369  7351 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:18.596508  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:18.563915  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:18.506287  7371 consensus_queue.cc:1059] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.089s
I20260709 12:25:18.621042  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:18.554852  7412 raft_consensus.cc:697] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:18.766024  7371 raft_consensus.cc:3055] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:18.574400  7377 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:18.751624  7435 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:25:18.508852  7134 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:18.689786  7313 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.766464  7371 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:18.774003  7313 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:18.774998  6785 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:18.775028  7444 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:18.775846  7444 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:18.765472  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:18.580022  5838 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:18.581427  7154 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:18.777915  7154 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.778308  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:18.778656  7421 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:18.779227  5944 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:18.781164  6785 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:25:18.778342  7154 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:18.778386  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 198455us
I20260709 12:25:18.783689  5838 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:18.784045  7420 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:18.785005  5838 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:18.786011  7445 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:18.787037  7445 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:18.787680  7445 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 5 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:18.796392  7393 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:18.686328  5873 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:18.798686  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:18.799469  5873 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:18.696435  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:18.799960  7329 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:18.800480  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:18.800799  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 104422us
I20260709 12:25:18.801339  5873 raft_consensus.cc:2368] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:18.802906  7134 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:18.803416  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:18.773149  7402 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:18.803418  7299 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:18.808468  5874 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
W20260709 12:25:18.817291  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.819432  5838 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:18.697659  5943 raft_consensus.cc:2368] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:18.601841  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.749159  5947 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:18.750325  5942 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 2: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:18.602373  7395 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:18.589134  6756 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:18.601420  7400 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:18.656424  7443 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:18.753125  6762 raft_consensus.cc:2393] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 4: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:18.753883  6766 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:18.889038  5942 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:18.582392  7278 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:18.659044  7439 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:18.759038  7455 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 12:25:18.758816  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:18.684559  7369 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.602144  5945 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:18.698837  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:18.764518  7430 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:18.765370  7300 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:18.923753  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:18.767256  7377 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:18.766855  7412 consensus_queue.cc:237] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:18.687239  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.748591  7403 consensus_queue.cc:1059] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:25:18.696689  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:18.766029  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 145079us
I20260709 12:25:18.774858  7371 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:18.823932  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.666299  5868 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:18.577104  7414 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:18.596800  7158 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:18.877794  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 74449.1us
I20260709 12:25:18.979212  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:18.683972  7409 raft_consensus.cc:740] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:18.880424  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:18.595919  7442 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:18.663275  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:18.756525  7305 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:18.995462  7443 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:18.996847  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:18.998062  5869 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 4: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:18.998284  7409 consensus_queue.cc:260] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:18.998965  7430 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:18.960755  6775 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:19.000515  7278 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:18.888098  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:18.773366  7386 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:18.929826  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 236998us
I20260709 12:25:18.788859  7445 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:18.879567  7316 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:19.007330  5949 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:19.007457  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:19.003417  5865 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:19.010398  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:18.893272  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 205552us
I20260709 12:25:19.012492  5944 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
W20260709 12:25:19.012602  6609 connection.cc:256] RPC call timeout handler was delayed by -2.0800677669999459s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2239s for a response
W20260709 12:25:18.933935  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44742.6us
I20260709 12:25:19.014894  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:19.016007  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:18.926537  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 239474us
W20260709 12:25:18.896483  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.872269  7427 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 12:25:18.932993  5942 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:19.028065  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.888716  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.029522  5943 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:25:18.890921  7385 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.004s
I20260709 12:25:18.940076  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:19.036723  5874 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:19.037185  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:19.038000  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:18.769120  7292 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:18.941862  5867 raft_consensus.cc:2368] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3.
I20260709 12:25:18.777019  7444 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:18.880935  7425 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 12:25:18.927831  7395 raft_consensus.cc:697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:18.928046  6017 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:19.057479  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:18.923677  5941 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:19.062093  5868 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:18.933185  7303 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:18.958781  6766 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:19.062500  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:19.063218  6024 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:18.930564  7439 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:18.958509  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:18.940609  7329 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:18.963896  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.085011  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 126940us
I20260709 12:25:19.090433  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:19.035131  7316 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:18.985419  7414 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:18.973095  5840 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.556s (SENT) [suppressed 13 similar messages]
W20260709 12:25:18.957495  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.981072  6757 raft_consensus.cc:2410] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:25:18.981506  7435 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:18.981879  7284 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:18.980149  5866 raft_consensus.cc:2393] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 3: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:18.939862  7311 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:25:18.954675  6768 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:18.987776  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:19.143630  5873 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:19.081442  7438 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:25:18.994668  7426 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:19.005513  5947 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:19.005550  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:18.940564  7369 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:19.148572  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 143125us
I20260709 12:25:18.973423  5940 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:19.150463  7369 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:19.060585  5949 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:19.059244  6609 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:19.134944  6775 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:25:19.065768  6016 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:19.026397  7371 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:19.103848  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:19.097585  7430 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:19.095122  7292 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:19.140087  6766 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:19.144675  6021 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:19.144505  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 115073us
I20260709 12:25:19.146196  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 416) took 1071 ms. Trace:
I20260709 12:25:19.150120  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:19.097357  7456 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:25:19.116354  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:19.167044  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 107887us
I20260709 12:25:19.192613  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:19.192803  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:18.980134  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:19.193718  7471 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:19.194428  7435 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:19.194662  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:19.195318  7435 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:19.195848  7435 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:19.195744  7471 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:25:19.176522  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.198653  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 218586us
I20260709 12:25:19.199465  5915 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:19.173276  7432 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:19.156150  6775 pending_rounds.cc:85] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 2
I20260709 12:25:19.185560  6762 rpcz_store.cc:276] 0709 12:25:18.074692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.074828 (+   136us) service_pool.cc:224] Handling call
0709 12:25:19.145992 (+1071164us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b54002a12d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:19.146136 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1456325376}
I20260709 12:25:19.154191  7311 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:19.188861  5913 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:19.178587  5873 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
W20260709 12:25:19.200855  7475 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:25:19.152339  7466 consensus_queue.cc:1059] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 12:25:19.212357  6609 leader_election.cc:400] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:19.212903  6609 leader_election.cc:403] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:25:19.196769  7471 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:19.202741  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:19.214501  7471 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:19.212771  7436 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:19.182191  7430 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:19.216626  7436 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.217103  7430 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:19.217128  7436 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:19.218995  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:19.221239  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:19.222635  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:19.223390  6766 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:19.224465  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:19.225392  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:19.225909  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:19.226101  6015 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:19.226723  6757 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 2: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:25:19.226886  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.223536  6756 raft_consensus.cc:2368] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4.
I20260709 12:25:19.228518  7421 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:19.229066  7421 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:19.230450  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:19.231434  6773 raft_consensus.cc:2393] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 4: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:19.233549  7421 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:19.198100  7403 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:19.236168  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:19.239320  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:19.240149  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:19.240622  6023 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:19.206252  6775 pending_rounds.cc:107] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:25:19.205839  7305 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:19.207363  7432 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:19.245253  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:19.204241  7461 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:19.238602  6778 raft_consensus.cc:2368] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4.
I20260709 12:25:19.253778  6018 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:19.208477  7311 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:19.184118  7395 consensus_queue.cc:237] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:19.240227  7403 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:25:19.248450  7478 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.094s	user 0.000s	sys 0.002s
I20260709 12:25:19.246609  6775 raft_consensus.cc:2889] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:19.235721  7351 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:25:19.248273  7305 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:19.201743  7316 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:19.255735  7432 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:19.255470  7369 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
W20260709 12:25:19.290225  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 95639.8us
I20260709 12:25:19.291536  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:19.280138  7403 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:19.292508  7476 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:25:19.266837  7386 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:19.259183  7292 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:19.304867  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:19.339612  6785 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 2: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:19.342373  7414 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:19.304644  7305 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:19.296998  7409 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:19.350528  6023 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:19.283964  7134 raft_consensus.cc:2764] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:25:19.320132  7262 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:19.321245  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 85166.8us
I20260709 12:25:19.266562  7373 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:19.284163  7474 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 12:25:19.275068  7479 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:19.337376  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:19.275923  7430 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:19.299836  7278 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:19.314433  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:19.375164  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:19.384020  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 46760.5us
I20260709 12:25:19.324497  7096 raft_consensus.cc:3055] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:19.384925  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:19.245900  7313 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:19.349845  6024 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:19.350530  7445 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.320842  7421 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:19.392769  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 419680us
I20260709 12:25:19.394784  6015 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:19.377753  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 298084us
I20260709 12:25:19.353969  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:19.396747  7466 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:19.397593  7466 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:19.398118  7466 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:19.398532  7466 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:19.388845  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:19.376528  7373 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:19.406514  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:19.411532  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:19.412744  5871 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:19.383025  7278 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:19.411886  7457 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:19.425333  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:19.394240  7369 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:19.336387  7311 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:19.384599  7479 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:19.373854  7439 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:19.413111  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:19.388077  7096 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:19.426594  5873 raft_consensus.cc:2368] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:19.414978  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:19.378407  7262 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:19.409155  7373 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:19.393447  7134 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:25:19.407508  7461 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.394377  7386 consensus_queue.cc:1059] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:25:19.462536  5871 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:19.451730  7457 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:19.463949  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:19.428457  6021 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 2: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:19.472651  7462 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.473191  7489 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:19.500909  7313 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:19.502187  5948 raft_consensus.cc:2368] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5.
I20260709 12:25:19.503379  7439 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:19.505038  7430 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:19.505347  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 672) took 1096 ms. Trace:
I20260709 12:25:19.505810  6016 rpcz_store.cc:276] 0709 12:25:18.408812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.412806 (+  3994us) service_pool.cc:224] Handling call
0709 12:25:19.505124 (+1092318us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b5400364cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:19.505275 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1473413888}
I20260709 12:25:19.407228  7421 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:19.442394  7480 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
W20260709 12:25:19.470242  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 57274.9us
I20260709 12:25:19.522599  5915 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:19.469520  7154 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.687s	user 0.011s	sys 0.003s
I20260709 12:25:19.524413  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:19.418272  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:19.459265  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:19.424399  7264 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 12:25:19.483026  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:19.477530  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:19.548003  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:19.548569  5871 raft_consensus.cc:2376] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5. Re-sending same reply.
I20260709 12:25:19.477402  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:19.483529  7134 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:19.402616  7445 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:19.533656  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 148779us
I20260709 12:25:19.474248  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:19.481927  7311 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:19.467149  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:19.543215  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.468422  5940 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3"
dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:43756
W20260709 12:25:19.519941  7464 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:25:19.555310  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:19.555702  5873 raft_consensus.cc:2368] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5.
W20260709 12:25:19.559101  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 76164.1us
I20260709 12:25:19.561395  6609 leader_election.cc:400] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:19.562772  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:19.565300  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 88057.4us
I20260709 12:25:19.566783  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:19.478124  7486 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.480520  7262 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:19.576149  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:19.542760  7439 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:19.546826  7489 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:19.611218  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 44470.1us
I20260709 12:25:19.581259  6609 leader_election.cc:400] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:19.583377  7470 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:19.614488  7470 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:19.615309  7430 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.615408  7470 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:19.615899  7430 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:19.616847  7470 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:19.583827  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:19.573172  7156 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:19.573225  7491 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:19.544188  7278 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:19.588950  6611 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:19.631062  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 129) took 1042 ms. Trace:
I20260709 12:25:19.497704  7479 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:19.591540  7445 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:19.591801  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:19.593912  7414 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:19.595206  6768 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:19.577750  5940 raft_consensus.cc:606] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:19.596038  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:19.593354  5915 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:19.542554  7154 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:19.597740  7264 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:19.498380  7462 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:19.606283  6775 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 3: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:19.664618  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:19.481583  7457 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:19.641220  7468 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.757s	user 0.000s	sys 0.008s
I20260709 12:25:19.606974  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 124) took 1030 ms. Trace:
I20260709 12:25:19.608913  7436 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:19.612377  6609 leader_election.cc:403] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:25:19.668881  5869 rpcz_store.cc:276] 0709 12:25:18.576232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.576367 (+   135us) service_pool.cc:224] Handling call
0709 12:25:19.606657 (+1030290us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b5400027954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:19.606889 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":713474816}
I20260709 12:25:19.482223  7461 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:19.490491  7373 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:19.671242  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 90137.5us
I20260709 12:25:19.612475  5840 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:19.672744  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 60372.2us
I20260709 12:25:19.631654  5949 rpcz_store.cc:276] 0709 12:25:18.588422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.596141 (+  7719us) service_pool.cc:224] Handling call
0709 12:25:19.630783 (+1034642us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54006725d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:19.630959 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":577687424}
I20260709 12:25:19.674752  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:25:19.627902  6017 consensus_meta.cc:287] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.529s	user 0.007s	sys 0.006s
W20260709 12:25:19.642204  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 48925.2us
W20260709 12:25:19.640854  5865 consensus_meta.cc:287] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.630s	user 0.014s	sys 0.001s
I20260709 12:25:19.643386  7134 raft_consensus.cc:2764] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:25:19.643929  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:19.630465  7386 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:19.643424  7158 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:19.625432  7489 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:19.626801  7305 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:19.711659  5946 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:19.659935  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:19.714329  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 54505.8us
I20260709 12:25:19.716037  7421 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:19.716236  7430 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:19.717188  7430 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:19.686327  7484 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.718410  7484 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:19.720992  7484 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:19.663136  7462 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:19.613173  7262 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:19.677871  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:19.685257  7156 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:19.681171  5872 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 4: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:19.634686  7473 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:19.682927  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 483364us
I20260709 12:25:19.639235  7432 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:19.684999  6017 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
W20260709 12:25:19.732996  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.665383  6775 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:25:19.674638  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:19.659195  7435 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:19.580741  7486 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:19.685819  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:19.650112  5940 raft_consensus.cc:614] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:19.650956  7278 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:19.638612  7414 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:19.699421  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:19.689466  5915 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:19.751579  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 62233.4us
W20260709 12:25:19.682493  7371 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.508s	user 0.013s	sys 0.000s
I20260709 12:25:19.752767  7371 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:19.717661  7430 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:19.662505  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:19.597365  7466 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:19.753830  7430 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:19.754402  5870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 131) took 1176 ms (client timeout 1529 ms). Trace:
W20260709 12:25:19.754909  5870 rpcz_store.cc:269] 0709 12:25:18.578284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.595250 (+ 16966us) service_pool.cc:224] Handling call
0709 12:25:19.662268 (+1067018us) spinlock_profiling.cc:243] Waited 4.23 ms on lock 0x7b58002e2f98. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:19.754321 (+ 92053us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9730304}
I20260709 12:25:19.755707  7466 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:19.660113  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 436) took 1054 ms. Trace:
I20260709 12:25:19.679183  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:25:19.777926  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 98852.1us
I20260709 12:25:19.685638  7264 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:19.690894  5941 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: [3.3-4.4]
I20260709 12:25:19.711773  7489 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:19.736579  7496 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.737012  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:19.791548  5942 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:25:19.637202  7471 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:19.737488  7154 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:19.737680  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:19.687099  5865 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:19.690708  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:19.741891  5947 raft_consensus.cc:2368] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3.
I20260709 12:25:19.707077  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:19.798090  5948 raft_consensus.cc:2368] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3.
I20260709 12:25:19.723476  7400 consensus_queue.cc:1059] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:25:19.709118  7386 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:19.756949  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 57625.2us
I20260709 12:25:19.757753  6016 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:19.758002  5915 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:19.808079  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 50169.9us
I20260709 12:25:19.762965  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:19.765158  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:19.775797  7402 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:19.731645  7473 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:19.736244  7484 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.780232  6609 leader_election.cc:400] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5.
I20260709 12:25:19.730437  7156 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:19.788072  7420 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:19.756841  7414 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:19.777534  7313 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:19.780241  7445 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:19.808979  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.810871  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:19.806131  7471 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:19.831343  5915 leader_election.cc:400] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4.
I20260709 12:25:19.832227  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:19.816442  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:19.821007  6609 leader_election.cc:403] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:25:19.809245  5866 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:19.809502  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 134) took 1229 ms. Trace:
I20260709 12:25:19.821389  7473 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:19.777037  6768 rpcz_store.cc:276] 0709 12:25:18.605490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.753262 (+147772us) service_pool.cc:224] Handling call
0709 12:25:19.651357 (+898095us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:19.815879  7408 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:19.833606  5915 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:19.836158  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 56129.6us
I20260709 12:25:19.819952  5945 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:19.837728  5990 leader_election.cc:400] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4.
I20260709 12:25:19.839437  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:19.848505  5990 leader_election.cc:403] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:25:19.827167  7156 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:19.858592  6021 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4"
dest_uuid: "6add331326ca47038308a447bf798fa7"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:48972
I20260709 12:25:19.859494  6021 raft_consensus.cc:606] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:19.831840  7420 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:19.835062  7395 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:19.846405  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:19.796802  7496 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:19.902784  7466 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:19.908641  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:19.852509  7373 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:19.841747  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:19.838729  7499 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:19.879617  7461 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:19.838846  5865 rpcz_store.cc:276] 0709 12:25:18.579072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.009426 (+430354us) service_pool.cc:224] Handling call
0709 12:25:19.807953 (+798527us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:19.850328  5915 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:19.879546  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:19.839219  7436 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:19.928570  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:19.934406  6768 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:19.934834  7461 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:19.840834  7262 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:19.846583  7457 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5.
I20260709 12:25:19.906203  7420 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:19.851946  7432 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:19.914471  6762 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:19.916436  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 70110.2us
I20260709 12:25:19.927325  7400 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:19.919111  7305 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:19.863904  7402 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4.
I20260709 12:25:19.838479  7412 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:25:19.902015  7396 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:25:19.931110  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 80898.8us
I20260709 12:25:19.967746  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:19.825616  7484 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:19.908823  7466 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:19.927359  7443 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:19.917016  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 75391.4us
I20260709 12:25:19.942962  6021 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:19.944896  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:19.948774  7483 raft_consensus.cc:3055] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:19.861948  7439 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:19.958557  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 683) took 1574 ms. Trace:
I20260709 12:25:19.955444  5695 catalog_manager.cc:5697] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:19.992564  5867 rpcz_store.cc:276] 0709 12:25:18.384496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.384850 (+   354us) service_pool.cc:224] Handling call
0709 12:25:19.958199 (+1573349us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b540083f254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:19.958434 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1226006272}
I20260709 12:25:19.991472  5874 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:19.982036  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:19.996585  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:20.000811  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:20.003842  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:20.010268  5874 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 6: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:20.011577  5869 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:20.014251  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:20.013541  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:20.019855  6762 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:19.978446  7351 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:20.023901  5840 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:19.978818  7436 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.027657  5874 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:20.028788  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:19.927178  7471 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:20.032656  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:19.992321  7483 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:19.978260  7396 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:20.009399  7402 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:20.068670  7402 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.068950  7305 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.069947  7305 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:20.070537  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:20.071523  5870 raft_consensus.cc:2368] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4.
I20260709 12:25:20.043697  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:20.073333  7311 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.003s
I20260709 12:25:20.095016  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:20.045396  7462 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:25:20.062757  7504 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 12:25:20.063037  7430 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:20.069264  7402 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:20.072829  7483 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.073454  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:20.115337  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:20.045011  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:20.113849  5942 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:20.074970  7484 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:20.075062  7305 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:20.004968  7470 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:20.076421  7385 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:20.079013  7466 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:20.081735  7386 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:20.082954  7426 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:20.085026  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:20.046125  7371 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.041949  7436 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:20.090477  6017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 186) took 1903 ms (client timeout 2107 ms). Trace:
W20260709 12:25:20.091660  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.092856  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:20.012818  7496 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:20.107970  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:20.111451  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:20.095562  7284 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.150197  7284 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.121115  7373 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:20.116005  6021 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:20.085513  7351 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:20.123495  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:20.131608  6762 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:20.130911  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:20.115778  7508 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 12:25:20.172537  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 49087.1us
I20260709 12:25:20.173445  6609 leader_election.cc:400] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5.
I20260709 12:25:20.173775  6609 leader_election.cc:403] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:20.112828  7462 raft_consensus.cc:697] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:25:20.131170  7403 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.775s	user 0.007s	sys 0.007s
I20260709 12:25:20.178916  5946 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:20.143299  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 676) took 1549 ms. Trace:
I20260709 12:25:20.145002  7426 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:20.153820  7371 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:20.089435  7409 consensus_meta.cc:287] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.715s	user 0.009s	sys 0.004s
I20260709 12:25:20.154951  5947 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:20.156220  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:20.215309  5948 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:20.216431  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:20.217923  7264 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3.
I20260709 12:25:20.219491  6020 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:20.153312  7284 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:20.217578  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:20.222256  5948 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
W20260709 12:25:20.227334  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.220686  7473 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:20.137027  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 101665us
I20260709 12:25:20.171643  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:20.234678  5942 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:20.235337  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 63812.1us
I20260709 12:25:20.172950  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:20.236167  6019 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:20.237378  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:20.238037  6021 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:20.238801  6021 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba"
dest_uuid: "6add331326ca47038308a447bf798fa7"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:48972
I20260709 12:25:20.240765  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:20.179690  6022 rpcz_store.cc:276] 0709 12:25:18.593401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.593544 (+   143us) service_pool.cc:224] Handling call
0709 12:25:18.681895 (+ 88351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.113607 (+431712us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:19.113703 (+    96us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:19.113773 (+    70us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:19.113835 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.142988 (+1029153us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b54005659d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:20.143043 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.143227 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":992141056}
I20260709 12:25:20.138111  7484 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.181761  7351 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:20.182399  5873 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:20.263569  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:20.264449  6762 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:20.182224  7414 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.175348  7373 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:20.189947  7489 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.242098  7311 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:20.194316  7462 consensus_queue.cc:237] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:20.196925  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:20.267679  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:20.289179  5948 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:25:20.198484  7513 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:25:20.295077  7473 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:20.295939  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 675) took 1703 ms. Trace:
I20260709 12:25:20.296581  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:20.296445  6020 rpcz_store.cc:276] 0709 12:25:18.592597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.592762 (+   165us) service_pool.cc:224] Handling call
0709 12:25:18.596822 (+  4060us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:20.219945 (+1623123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.295644 (+ 75699us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b5400764654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:20.295869 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3730797952}
I20260709 12:25:20.286808  7409 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:25:20.298135  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:20.287599  7264 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.301007  7264 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.301980  6023 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:20.303308  7414 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:20.304531  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 509) took 1070 ms. Trace:
I20260709 12:25:20.305039  6015 rpcz_store.cc:276] 0709 12:25:19.234500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.234707 (+   207us) service_pool.cc:224] Handling call
0709 12:25:19.235126 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.402362 (+167236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.304284 (+901922us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b5400763c54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:20.304462 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":366645504}
I20260709 12:25:20.305809  7400 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:20.172891  7496 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:20.307161  6015 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:20.307266  7496 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:20.200440  7096 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:20.201977  7396 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:20.146605  6017 rpcz_store.cc:269] 0709 12:25:18.186983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.195105 (+  8122us) service_pool.cc:224] Handling call
0709 12:25:20.090272 (+1895167us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b540041da54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:20.090413 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1550710144}
I20260709 12:25:20.147747  7156 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:20.205159  7403 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.178332  7517 raft_consensus.cc:2764] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:25:20.210227  5866 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:20.251583  6022 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:20.327251  6024 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:25:20.270895  5942 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:20.276486  5840 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:20.194555  5940 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:20.327764  6024 pending_rounds.cc:85] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 2
W20260709 12:25:20.328657  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 52278.2us
I20260709 12:25:20.329068  6024 pending_rounds.cc:107] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:25:20.329327  6024 raft_consensus.cc:2889] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:20.197482  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:20.248737  7439 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:20.303426  7264 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:20.201725  7442 raft_consensus.cc:3055] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:20.311709  5948 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
W20260709 12:25:20.311836  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 44257.3us
I20260709 12:25:20.337918  5948 pending_rounds.cc:85] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 1
I20260709 12:25:20.338442  5948 pending_rounds.cc:107] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:25:20.338765  5948 raft_consensus.cc:2889] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:20.333727  6778 raft_consensus.cc:2376] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3. Re-sending same reply.
I20260709 12:25:20.310521  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:20.340277  7403 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:20.285815  7421 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:20.341545  7329 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.341915  7305 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:20.310464  6768 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:20.342175  7329 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:20.317178  7299 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.371s	user 0.000s	sys 0.038s
I20260709 12:25:20.343379  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:20.318701  5943 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:20.334256  6018 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:20.344982  6768 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:20.310647  7501 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:20.345779  7479 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.250542  7484 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.346469  7479 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.336210  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:20.314873  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:20.340471  6766 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 4: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:20.341214  7421 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:20.292224  7311 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:20.343608  7299 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.400s	user 0.000s	sys 0.039s
W20260709 12:25:20.334781  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 137378us
W20260709 12:25:20.320539  6775 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.577s	user 0.007s	sys 0.006s
I20260709 12:25:20.355180  5987 leader_election.cc:400] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4.
I20260709 12:25:20.319819  7496 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:20.322155  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 413) took 1064 ms. Trace:
I20260709 12:25:20.331115  7439 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.323194  7432 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:25:20.334910  7313 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:20.375831  6020 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:20.377651  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:20.384119  6762 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:20.337713  7442 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:20.350220  7517 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:20.342092  7284 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:20.377728  6609 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:20.378504  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 60297.2us
I20260709 12:25:20.337343  7278 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:25:20.393510  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:20.397516  5869 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:20.349267  7402 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:20.402999  7489 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:20.403429  6021 raft_consensus.cc:606] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:20.403829  6021 raft_consensus.cc:614] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:20.357939  7096 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:20.404371  7154 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:20.404847  7371 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:20.400825  6757 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:20.409835  7484 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:20.401844  7436 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:20.411679  7486 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:20.363755  7305 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.377974  6775 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:20.414251  7305 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:20.366194  7400 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:20.385370  7439 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:20.366340  7311 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:20.386546  7430 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:20.388751  7496 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:20.423723  6609 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:20.355831  7316 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:25:20.416034  7517 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:20.415259  7442 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:20.461537  7442 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:20.461813  7517 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.391563  7313 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:25:20.471019  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68865us
I20260709 12:25:20.419580  7400 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.472084  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:20.420732  7368 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5.
I20260709 12:25:20.419989  7351 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:20.453605  7484 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:20.481348  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 57738.9us
I20260709 12:25:20.483708  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:20.484434  5944 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:20.377313  6785 rpcz_store.cc:276] 0709 12:25:19.257677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.340427 (+ 82750us) service_pool.cc:224] Handling call
0709 12:25:20.196740 (+856313us) spinlock_profiling.cc:243] Waited 4.69 ms on lock 0x7b580001f898. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:20.319813 (+123073us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10788096}
I20260709 12:25:20.500259  7426 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:20.501981  5949 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 3: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:20.502839  7426 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:20.479060  7400 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:20.458918  7315 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:25:20.476675  7496 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:20.487550  7096 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:20.468312  7404 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:25:20.490797  7420 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.418592  7284 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.508111  6609 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:20.492985  7462 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:20.523561  7496 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:20.530372  7154 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.531121  7154 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:20.533299  7154 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:20.533592  5913 leader_election.cc:400] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3.
I20260709 12:25:20.542016  5913 leader_election.cc:403] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:25:20.543198  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:20.543532  7371 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:20.515470  7420 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:20.547099  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:20.547524  7420 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:20.547942  5942 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:20.548054  7507 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.619s	user 0.002s	sys 0.002s
I20260709 12:25:20.548830  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:20.549508  5944 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:20.549872  7489 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:20.550164  7421 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:20.535173  7284 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:20.551023  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:20.551695  5943 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:20.552989  5948 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:20.553817  7156 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:20.555564  7264 raft_consensus.cc:3055] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:20.556025  7264 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:20.556876  7264 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.511260  7373 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:20.521772  7486 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:20.558485  7511 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:25:20.559233  7527 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:25:20.560853  7435 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:20.564559  5866 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:20.582247  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:20.550479  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:20.521157  7313 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:20.567911  5946 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:20.538574  5695 catalog_manager.cc:5697] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 1 to 2, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:20.550537  7489 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:20.489734  7484 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:20.574311  5694 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40818 (request call id 146) took 9401 ms. Trace:
I20260709 12:25:20.573009  7430 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:20.584414  5944 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 2: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:20.588187  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:20.593350  5942 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:20.567039  7402 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:20.597018  5695 catalog_manager.cc:5697] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 1 to 3, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 3 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:20.568847  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:20.578430  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:20.605880  5872 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:20.574838  7486 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.609237  5695 catalog_manager.cc:5697] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 3 to 4, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 4 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:20.607383  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:20.628841  5867 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:25:20.629283  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 143058us
W20260709 12:25:20.621980  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.597800  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:20.599817  7404 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.577045  7526 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:25:20.638172  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:20.612509  7486 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:20.644018  7371 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3.
I20260709 12:25:20.601308  5694 rpcz_store.cc:276] 0709 12:25:11.173013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.173931 (+   918us) service_pool.cc:224] Handling call
0709 12:25:20.537610 (+9363679us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:18.175455 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:18.176031 (+   576us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:18.176064 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:25:18.176080 (+    16us) tablet.cc:662] Decoding operations
0709 12:25:18.176664 (+   584us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:18.176720 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:18.176750 (+    30us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:25:18.177143 (+   393us) tablet.cc:701] Row locks acquired
0709 12:25:18.177162 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:25:18.177337 (+   175us) write_op.cc:270] Start()
0709 12:25:18.177465 (+   128us) write_op.cc:276] Timestamp: P: 1783599918177283 usec, L: 0
0709 12:25:18.177482 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 12:25:18.252200 (+ 74718us) log.cc:844] Serialized 1681 byte log entry
0709 12:25:18.662904 (+410704us) op_driver.cc:464] REPLICATION: finished
0709 12:25:19.145755 (+482851us) write_op.cc:301] APPLY: starting
0709 12:25:19.150403 (+  4648us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:19.184347 (+ 33944us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:19.209139 (+ 24792us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:19.214013 (+  4874us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:19.214301 (+   288us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:25:19.214372 (+    71us) write_op.cc:312] APPLY: finished
0709 12:25:19.215011 (+   639us) log.cc:844] Serialized 40 byte log entry
0709 12:25:19.215315 (+   304us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:19.215483 (+   168us) write_op.cc:454] Released schema lock
0709 12:25:19.215566 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:19.216247 (+   681us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:19.216306 (+    59us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":127257,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":450625,"apply.run_cpu_time_us":3711,"apply.run_wall_time_us":78774,"num_ops":4,"prepare.queue_time_us":275562,"prepare.run_cpu_time_us":18152,"prepare.run_wall_time_us":164892,"raft.queue_time_us":250678,"raft.run_cpu_time_us":15249,"raft.run_wall_time_us":275926,"replication_time_us":485213,"spinlock_wait_cycles":29056,"thread_start_us":781774,"threads_started":4,"wal-append.queue_time_us":74523,"wal-append.run_cpu_time_us":64877,"wal-append.run_wall_time_us":1907099}]]}
I20260709 12:25:20.647200  7483 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:20.630249  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:20.603467  7264 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:20.649081  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:20.650805  5987 leader_election.cc:400] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3.
I20260709 12:25:20.653347  5987 leader_election.cc:400] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3.
I20260709 12:25:20.654047  5987 leader_election.cc:403] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:25:20.610046  7435 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:20.658913  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:20.660131  7262 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:20.662050  7501 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3.
I20260709 12:25:20.666453  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:20.667294  6756 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:20.668915  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:20.670022  5940 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:20.656147  7529 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:20.672930  7529 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:20.673954  7529 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:20.674703  7154 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:20.675418  7154 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:20.688993  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:20.689824  6785 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:20.689872  7489 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:20.690866  7154 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:20.677749  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:20.691864  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 12:25:20.692476  7509 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.761s	user 0.003s	sys 0.003s
I20260709 12:25:20.693811  5872 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:20.612100  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:20.697041  5940 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:20.686340  6756 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:20.698468  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:20.679239  5947 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:20.702189  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:20.694406  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:20.608986  7430 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:20.694957  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:20.714905  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.715380  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:20.716931  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 58568.9us
I20260709 12:25:20.726227  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:20.727681  5872 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:20.649492  7402 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:20.655789  7511 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:20.734470  5872 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:20.738812  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:20.727979  5913 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:20.741112  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.729262  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:20.747491  5874 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:20.754676  5872 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:20.632648  7439 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.700843  7421 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:20.730521  7435 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:25:20.746520  7096 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:20.773491  7484 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:20.650065  7404 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:20.774461  7439 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:20.774745  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:20.774528  7400 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:20.815902  7400 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:20.776175  7435 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:20.821812  7435 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:20.822779  7435 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:20.824241  7435 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:20.737691  7501 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:20.825904  7501 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.826402  7501 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:20.827275  5941 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:20.828888  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:20.833601  7373 raft_consensus.cc:3055] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:25:20.834285  7514 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.786s	user 0.000s	sys 0.003s
I20260709 12:25:20.834296  7373 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:20.835500  7373 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.836784  7329 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:20.810657  7262 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:20.776690  7511 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:20.869688  6020 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:20.733054  7402 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:20.730969  7156 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:20.778803  7421 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:20.799755  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:20.875630  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 100968us
I20260709 12:25:20.876369  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:20.877970  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:20.879735  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:20.880260  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:20.882593  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:20.884918  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:20.853212  7264 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.769060  7368 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:20.895717  7368 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.896204  7368 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:20.768818  7284 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:20.799742  7371 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 12:25:20.906971  7479 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:20.809868  7311 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:20.813578  7531 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:20.775315  7403 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:20.853160  7373 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:20.866794  7511 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.891013  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:20.837728  7262 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:20.867846  7329 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:25:20.894835  7264 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.903465  7421 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:20.892673  7414 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:25:20.874722  7402 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:20.887207  7484 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:20.921455  7329 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:20.799386  7486 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:20.914021  7511 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:20.955662  7496 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:20.956383  7496 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:20.957559  7496 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:20.960182  7496 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:20.969548  7486 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:20.801568  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:20.800683  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:20.801535  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:20.812459  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:20.801847  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 80375.8us
I20260709 12:25:21.148494  5944 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
W20260709 12:25:21.018751  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 219956us
I20260709 12:25:20.762377  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.022754  5913 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:20.812184  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:21.164104  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 142042us
I20260709 12:25:21.164498  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:20.865142  6016 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:20.861515  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:21.028556  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:21.166260  7537 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:21.167196  7537 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:21.167551  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:20.862331  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:21.168373  5870 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:21.030159  6756 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 5. Ops: []
I20260709 12:25:21.032322  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:21.033079  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:21.034071  5947 raft_consensus.cc:2393] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 2: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:21.169204  7537 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:21.036024  7284 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:21.169903  6019 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:21.174616  7484 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:21.174985  7284 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:20.914115  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:20.913375  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:20.864925  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:20.883841  7156 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:21.033281  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:21.037557  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.177610  7284 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:21.040772  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:21.188326  6023 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:21.041795  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:21.190418  6773 raft_consensus.cc:2393] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 4: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:21.191751  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:21.191963  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:21.192736  6773 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:21.057958  7329 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:21.042070  7299 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:25:21.195043  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:21.044937  7311 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:21.044946  7264 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:21.049274  7300 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.199259  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:21.048478  7262 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:21.073448  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.078279  7292 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.201179  7292 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:21.201996  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.200815  7284 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:21.203403  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.204099  7292 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:21.101420  7414 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:21.204741  7300 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:21.205113  7414 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:21.208277  7414 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:21.178959  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:21.104254  7438 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:21.211489  7438 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:20.975292  7501 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:21.214437  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.216240  5873 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:20.979292  7511 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:21.217429  5865 raft_consensus.cc:2410] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:21.219769  7511 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:21.180433  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:21.167272  6017 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:21.222592  6021 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:20.819274  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 203) took 1219 ms. Trace:
I20260709 12:25:21.179548  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:21.225121  6762 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 3: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:21.224506  7402 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:21.182154  7479 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:25:21.182176  7484 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:21.181226  6015 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:21.229650  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:21.177116  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.233275  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:21.183573  7436 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
W20260709 12:25:21.038707  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.178874  5866 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:21.085695  6024 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:20.947757  7430 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:21.239245  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 83070.9us
I20260709 12:25:21.240008  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:21.240528  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:21.241834  7436 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:21.242633  6762 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:21.153613  7421 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:25:21.221324  6766 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 3: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:21.234292  5871 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 4: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:21.242751  7436 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:21.243578  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:21.217473  7404 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:21.181022  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:21.184906  5941 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 4: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:21.223852  6018 rpcz_store.cc:276] 0709 12:25:19.599548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.663884 (+ 64336us) service_pool.cc:224] Handling call
0709 12:25:19.664313 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:20.344934 (+680621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.818907 (+473973us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b5400661cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:20.819181 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1540195200}
I20260709 12:25:21.205924  7284 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:21.246865  7438 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:21.216670  7501 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:21.204193  5874 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:21.248224  5943 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:21.183161  6019 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:21.197378  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:21.256707  7278 raft_consensus.cc:2764] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:25:21.208146  7300 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:21.258502  6016 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 5: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:21.183889  7156 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:21.261313  5942 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:21.264709  5944 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:21.265039  7436 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:21.270494  7444 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:21.270648  7264 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.271350  7444 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:21.271750  7264 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:21.275086  7517 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:21.276847  7444 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:21.277503  7264 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:21.277227  7517 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:21.277748  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:21.279649  7329 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:21.280449  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.280372  7517 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:21.282980  5871 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:25:21.292764  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:21.293051  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:21.293217  7368 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:21.294900  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.295575  5869 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:21.296535  7368 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:21.298059  6773 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:21.298343  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 444) took 1024 ms. Trace:
I20260709 12:25:21.298974  6778 rpcz_store.cc:276] 0709 12:25:20.273659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.297836 (+ 24177us) service_pool.cc:224] Handling call
0709 12:25:21.297902 (+1000066us) spinlock_profiling.cc:243] Waited 32.9 ms on lock 0x7b54005294d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:21.298132 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75707776}
I20260709 12:25:21.302984  6019 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:21.305650  7329 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:21.306768  7369 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:21.307490  5990 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:21.303607  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.303846  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:21.308698  6024 raft_consensus.cc:2376] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4. Re-sending same reply.
W20260709 12:25:21.311630  7546 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.069s	user 0.002s	sys 0.000s
I20260709 12:25:21.314285  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:21.312855  6020 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:21.312724  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.310459  6766 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:21.313721  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:21.327241  6019 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:21.349468  5865 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:21.351711  7402 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:21.353724  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:21.354166  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.354882  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.355846  5870 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
W20260709 12:25:21.356839  7548 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.213s	user 0.001s	sys 0.001s
I20260709 12:25:21.356953  7501 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:21.357656  7501 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.358057  7501 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:21.358199  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:21.359040  5949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 132) took 1267 ms (client timeout 1511 ms). Trace:
W20260709 12:25:21.359500  5949 rpcz_store.cc:269] 0709 12:25:20.091138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.092582 (+  1444us) service_pool.cc:224] Handling call
0709 12:25:21.358569 (+1265987us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b540058f554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:21.358833 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":822816000}
W20260709 12:25:21.356999  7544 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.199s	user 0.000s	sys 0.005s
I20260709 12:25:21.364509  5871 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:21.371165  6015 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:21.375614  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:21.377749  6019 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:21.380503  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.386847  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:21.391659  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:21.392746  5949 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
W20260709 12:25:21.394752  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 154418us
I20260709 12:25:21.395467  7499 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.396214  7499 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:21.399178  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:21.401027  7371 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:21.403249  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:21.405061  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:21.405675  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:21.406567  6020 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:21.406687  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:21.407899  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:21.408689  5987 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:21.410390  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:21.381531  7403 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:25:21.417642  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 110219us
W20260709 12:25:21.418460  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.418605  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:21.419842  6017 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 6: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:21.419960  6024 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:21.422921  7486 consensus_queue.cc:1059] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:21.423128  7517 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:21.424401  7486 raft_consensus.cc:3055] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:21.424907  7486 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:21.408713  6020 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:21.424636  7517 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:21.431795  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:21.432503  7430 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:21.432983  6016 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:21.433748  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:21.434052  7299 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.434815  7299 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:21.435771  7484 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:21.429152  7284 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:21.439131  7457 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:21.439879  7457 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:21.442027  5947 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 6: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:21.441831  7438 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:21.450615  5944 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:21.433032  7430 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.463963  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:21.462281  5913 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:21.430330  7517 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:21.468291  7412 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.469609  7542 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.255s	user 0.002s	sys 0.001s
I20260709 12:25:21.470181  5838 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:21.473649  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:21.474066  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:21.474611  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:21.474851  7292 raft_consensus.cc:3055] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:21.477000  6022 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:21.458165  7484 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.460075  7278 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:21.444288  7438 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.482892  7368 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:21.485016  7369 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:21.486613  7264 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:21.489878  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:21.490989  7550 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.347s	user 0.002s	sys 0.000s
I20260709 12:25:21.491103  7444 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:21.492259  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 83662.5us
I20260709 12:25:21.493212  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:21.457825  7430 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:21.494447  5947 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:21.499025  5871 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:21.479233  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:21.503166  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 47960.5us
I20260709 12:25:21.454090  7299 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:21.507189  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:21.521003  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:21.503017  7292 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:21.513084  7438 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:21.535545  6021 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:21.552934  7450 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 12:25:21.555569  5867 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:21.555981  7529 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.558030  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:21.559859  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 89865.3us
I20260709 12:25:21.561022  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:21.562443  7292 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:21.563797  7444 consensus_queue.cc:1243] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:25:21.564486  7292 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:21.555346  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:21.567272  6020 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:21.568837  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:21.569878  7511 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:21.570408  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:21.570640  7511 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:21.570989  7501 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:21.572769  6017 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:21.572618  7511 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:21.521080  7404 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:21.577958  7404 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:21.578284  7402 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:21.578495  7404 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:21.578954  7402 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.579420  7402 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:21.580886  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:21.535240  7412 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:21.582031  7403 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:21.582576  7403 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:21.583029  7403 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:21.583928  7412 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:21.584918  7457 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:21.586858  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:21.588728  5915 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:21.521121  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:21.590428  7499 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:21.590394  7402 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:21.590723  7517 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:21.518741  7484 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:21.592373  7517 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:21.593755  7484 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:21.511130  7486 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:21.595889  7486 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:21.596418  5990 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:21.599436  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:21.602061  7373 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:21.603199  7373 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:21.605275  7373 consensus_queue.cc:1059] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:21.606354  7402 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:21.607017  7483 raft_consensus.cc:3055] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:21.607290  6609 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:21.607646  7483 raft_consensus.cc:740] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:21.609033  7483 consensus_queue.cc:260] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:21.610623  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 49800.5us
I20260709 12:25:21.613500  7156 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:21.613922  7489 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:21.614507  7156 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:21.546667  7278 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:21.616263  7483 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:21.621618  5990 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:21.631194  5840 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:21.646011  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 48712.5us
I20260709 12:25:21.557047  7529 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:21.630746  7414 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:21.678051  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51064.3us
W20260709 12:25:21.682602  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 140683us
I20260709 12:25:21.683856  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:21.682694  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:21.643424  7484 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:21.601126  7403 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:21.700063  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 78573.3us
I20260709 12:25:21.671598  7368 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:21.697521  6019 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:21.669490  7489 raft_consensus.cc:697] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:25:21.709800  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 198902us
W20260709 12:25:21.743359  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 50275.8us
W20260709 12:25:21.733467  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 144832us
I20260709 12:25:21.683826  7479 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:21.735580  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:21.718039  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 154) took 1037 ms. Trace:
I20260709 12:25:21.750852  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.751840  5871 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
W20260709 12:25:21.758306  7537 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.042s	sys 0.030s
W20260709 12:25:21.758834  7537 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.042s	sys 0.030s
I20260709 12:25:21.710449  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:21.726629  7300 raft_consensus.cc:3055] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:21.710568  7536 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:21.773061  5915 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:21.713811  7313 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:21.727679  7278 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:21.767036  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.721793  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:21.709702  7368 raft_consensus.cc:697] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:21.709856  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:21.724730  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:21.775252  5867 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:21.708712  7412 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 12:25:21.784560  5947 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:21.752413  7421 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:25:21.771966  7300 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:21.776819  7313 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:21.796949  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 94351.3us
I20260709 12:25:21.763710  7444 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:25:21.777187  7262 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:21.779237  7373 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:21.748920  5868 rpcz_store.cc:276] 0709 12:25:20.680097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.680843 (+   746us) service_pool.cc:224] Handling call
0709 12:25:20.681238 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:20.725904 (+ 44666us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:21.225273 (+499369us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b18002394a0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d667b3c7 00007f39d2c8d9a7 00007f39d2c9d0bf 00007f39d2c9d038
0709 12:25:21.323724 (+ 98451us) log.cc:844] Serialized 14 byte log entry
0709 12:25:21.547745 (+224021us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:21.554264 (+  6519us) log.cc:844] Serialized 26 byte log entry
0709 12:25:21.554542 (+   278us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:21.554904 (+   362us) log.cc:844] Serialized 14 byte log entry
0709 12:25:21.555163 (+   259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.555344 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:21.717505 (+162161us) raft_consensus.cc:1685] finished
0709 12:25:21.717536 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:21.717785 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94496,"prepare.queue_time_us":634126,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":106,"raft.queue_time_us":106270,"raft.run_cpu_time_us":413,"raft.run_wall_time_us":22669,"spinlock_wait_cycles":7782656,"thread_start_us":149800,"threads_started":2,"wal-append.queue_time_us":310011}
I20260709 12:25:21.779493  7536 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:21.728732  7426 consensus_queue.cc:1059] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 12:25:21.789489  7403 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:21.813763  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 112029us
I20260709 12:25:21.810577  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:21.808043  7537 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.816064  7537 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:21.819228  7537 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:21.808187  7499 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:21.823685  7351 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:21.825309  7351 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:21.825107  7536 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:21.838037  7501 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:21.805364  7264 consensus_queue.cc:1059] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:25:21.790514  7489 consensus_queue.cc:237] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:21.844693  5943 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:21.846446  7368 consensus_queue.cc:237] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:21.849844  7457 raft_consensus.cc:3055] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:21.804482  7278 consensus_queue.cc:1059] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:25:21.851563  7529 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:21.852970  7329 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.547s	user 0.004s	sys 0.011s
I20260709 12:25:21.853107  7479 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.850579  7565 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:25:21.854215  7426 raft_consensus.cc:3055] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:21.856105  5940 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:21.812952  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:21.858132  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:21.858000  7300 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:21.802230  7313 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:21.861438  7305 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.862246  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 93953.1us
I20260709 12:25:21.862697  7369 raft_consensus.cc:3055] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:21.864977  7450 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:25:21.866125  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.896992  5948 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:21.853996  5840 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:21.873880  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 111194us
I20260709 12:25:21.894862  5949 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:21.850446  7457 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:25:21.897889  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 87407.1us
I20260709 12:25:21.902457  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:21.907939  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 84165.2us
W20260709 12:25:21.911137  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 57260.8us
I20260709 12:25:21.913492  7457 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:21.899533  7305 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:21.915354  7457 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:21.916476  7305 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:21.918058  7479 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:21.920274  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.920848  7479 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:21.922009  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:21.922885  5866 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:21.923600  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:21.927016  7412 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.927767  7412 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:21.888808  7426 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:21.896581  7369 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:21.899744  7450 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:21.902004  7403 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:25:21.947839  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 51980.1us
I20260709 12:25:21.947089  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:21.934701  7300 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:21.896935  7438 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:21.963335  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.964753  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:21.955049  5913 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:21.938490  7262 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:21.967072  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:21.967760  7369 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:21.968135  7426 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:21.970077  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 71995.1us
I20260709 12:25:21.970672  7501 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:21.971040  7404 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:21.971423  7373 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:21.975835  7450 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:22.029326  7450 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:22.029691  7412 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:22.030287  7412 raft_consensus.cc:2764] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:21.953214  7486 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.983795  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 81443.2us
I20260709 12:25:22.019157  6757 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:25:22.021766  7491 consensus_queue.cc:1175] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" responder_term: 3 status { last_received { term: 3 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 3 index: 3 } }
W20260709 12:25:22.022998  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 58393.3us
I20260709 12:25:22.024375  7373 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:22.025350  7577 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:25:22.027732  7404 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:21.971305  7483 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.047173  7484 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:22.047353  7483 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:22.047912  7484 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:22.004618  7262 raft_consensus.cc:697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:22.041888  7403 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
W20260709 12:25:22.028339  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 92481us
I20260709 12:25:21.994974  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:22.064873  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:22.055859  6611 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:22.061400  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 106473us
I20260709 12:25:22.070477  5948 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
W20260709 12:25:22.056453  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 61631us
I20260709 12:25:22.053014  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:22.073336  7479 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:22.075402  6611 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:22.135913  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:22.056908  7571 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:25:22.149055  7369 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:22.049597  7483 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:22.150522  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 85747.7us
I20260709 12:25:22.156248  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:22.049983  7484 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:22.053436  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:22.055927  7426 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:22.055825  7351 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.099s
I20260709 12:25:22.134563  7517 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:22.153045  7436 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:22.056669  7574 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:22.054751  7262 consensus_queue.cc:237] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:22.159992  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 84684.3us
I20260709 12:25:22.162104  7496 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:22.134037  7402 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:22.139227  7299 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.139310  7479 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.057542  7438 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:22.056629  7444 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.100s
I20260709 12:25:22.072281  7486 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:22.173902  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 99205.6us
I20260709 12:25:22.139269  7404 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:25:22.057145  7570 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.109s
I20260709 12:25:22.059810  7421 consensus_queue.cc:1059] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.108s
I20260709 12:25:22.165678  7499 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:22.207806  6611 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:22.181322  7574 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:22.171996  7517 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.170015  7483 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:22.174425  7436 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.229995  7577 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:22.178468  7484 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:25:22.279832  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 116464us
I20260709 12:25:22.173751  7403 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 12:25:22.224413  7479 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:22.301508  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 93879.7us
I20260709 12:25:22.288275  7517 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:22.281839  7574 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:22.208078  7313 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:22.300854  7430 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.749s	user 0.010s	sys 0.004s
W20260709 12:25:22.331184  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 271280us
I20260709 12:25:22.295542  7436 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:22.324426  7414 consensus_meta.cc:287] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.000s
I20260709 12:25:22.337661  5915 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:22.232129  7299 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:22.272634  7402 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:22.338377  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:22.348994  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:22.350332  5871 raft_consensus.cc:2410] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:22.368973  5869 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:22.376479  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:22.308564  7484 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:22.407401  6020 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
W20260709 12:25:22.407594  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 258849us
W20260709 12:25:22.408905  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 77875.9us
I20260709 12:25:22.353524  7414 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:22.300484  7577 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:22.436659  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 82593.5us
I20260709 12:25:22.441028  6022 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:25:22.438772  6024 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:22.439162  6023 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:25:22.448201  6018 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:22.341408  7351 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 12:25:22.443123  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:22.449683  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:22.450254  5948 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:22.443892  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.448084  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:22.350063  7292 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:22.443445  6015 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 12:25:22.451280  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.453320  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 122373us
I20260709 12:25:22.443388  7373 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:22.455053  7586 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:22.456120  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.457496  7582 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:25:22.457842  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:22.458562  5947 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:22.460816  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:22.455716  7586 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:22.349728  7430 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:22.441847  7537 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:25:22.445204  7581 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 12:25:22.445204  7583 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:25:22.480329  6021 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:25:22.482247  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:22.440322  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:22.487697  5871 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:25:22.487525  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 45309.7us
I20260709 12:25:22.486515  6021 pending_rounds.cc:85] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 3
I20260709 12:25:22.442601  7580 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:22.495482  7491 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.444821  7587 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:22.498111  7596 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:22.498371  5941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 154) took 1166 ms (client timeout 1511 ms). Trace:
I20260709 12:25:22.500552  7430 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:22.482440  7373 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.452333  7292 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:22.501964  6017 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:25:22.461534  5945 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:22.504292  5946 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:22.509354  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:22.510881  7589 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:22.511413  7576 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:22.509605  7529 raft_consensus.cc:3055] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:22.511829  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:22.512809  7591 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:22.513535  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:22.462155  7511 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.515380  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 434) took 1226 ms. Trace:
I20260709 12:25:22.518870  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:22.519290  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.462632  7444 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:22.489543  6021 pending_rounds.cc:107] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:25:22.580417  6021 raft_consensus.cc:2889] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:22.522126  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:22.523247  7579 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 12:25:22.528123  7499 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:22.528084  7299 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:22.534029  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:22.482344  7305 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:22.572521  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:22.572732  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:22.485086  6778 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:25:22.577611  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68355.6us
I20260709 12:25:22.540123  7596 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:22.569715  7489 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:22.589596  6756 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 4: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:22.587210  7444 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:22.594720  6768 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:22.596726  6766 raft_consensus.cc:3055] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:22.599149  6023 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:25:22.604360  7577 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:22.578734  7589 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:22.605567  7313 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:22.587391  7499 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:22.606094  7305 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:22.580457  7511 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:22.607393  7484 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:22.572623  7529 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:22.607810  7576 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:22.608659  7592 raft_consensus.cc:3055] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:22.609474  7586 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:22.579748  7414 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:22.539971  7587 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.573184  7486 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:22.614377  5947 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:22.566234  7491 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:22.620819  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 48185.8us
I20260709 12:25:22.593185  6793 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:22.619693  6766 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:25:22.634272  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 145469us
I20260709 12:25:22.636265  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:22.642025  6611 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:22.655377  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:22.622515  7596 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:22.662218  6757 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:22.653023  7570 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:22.669041  6766 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:22.565958  5941 rpcz_store.cc:269] 0709 12:25:21.332252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.492853 (+160601us) service_pool.cc:224] Handling call
0709 12:25:22.498123 (+1005270us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b540058f554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:22.498290 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1878757120}
I20260709 12:25:22.653461  7592 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:22.682108  7529 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:22.683420  7529 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:22.683481  7592 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:22.685146  7592 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:22.654874  7496 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:22.687304  7491 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:22.687834  7576 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:22.672818  7368 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:22.689112  7368 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:22.675180  7486 consensus_queue.cc:1059] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:25:22.690100  7264 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:22.667101  7586 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.691179  7264 raft_consensus.cc:3055] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:22.691584  7264 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:22.691378  7586 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:22.692726  7264 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:22.693507  5949 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:22.694023  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:22.694132  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:22.694625  7457 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.779s	user 0.014s	sys 0.000s
I20260709 12:25:22.695370  7457 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:22.681975  7351 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:22.695887  7457 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:22.696465  7597 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:22.696821  7597 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:22.696949  7264 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:22.689524  7368 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:22.698734  7439 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 12:25:22.695492  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:22.653036  7292 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:22.701521  7292 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:22.696969  7457 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:22.702322  6766 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:22.701943  7589 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:22.702837  7457 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:22.694540  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:22.673215  7570 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:22.675487  7403 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:25:22.657907  7587 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:22.706386  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:22.707022  7573 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.497s	user 0.001s	sys 0.005s
I20260709 12:25:22.707365  5945 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:22.671211  7414 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:22.711052  7351 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.711534  7414 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:22.695279  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:22.711709  7351 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:22.712746  5943 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:22.712917  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:22.713909  7580 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:22.682456  7305 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:22.678674  7511 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:22.727973  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:22.714692  7486 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.588830  7373 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:22.768980  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:22.705448  7570 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:22.676059  7402 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:22.775746  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:22.735601  7299 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:22.781055  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 68264.2us
I20260709 12:25:22.777539  7486 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:22.784333  7486 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:22.785801  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:22.786928  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.787544  6757 raft_consensus.cc:2410] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:22.735880  7597 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:22.788160  7444 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:22.789143  7597 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:22.789759  5943 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:22.771293  5948 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:22.768996  7436 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:22.714466  7580 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:22.793095  7436 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:22.794164  7351 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:22.794574  7580 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:22.795342  7426 consensus_meta.cc:287] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.623s	user 0.012s	sys 0.000s
W20260709 12:25:22.790015  7590 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.349s	user 0.000s	sys 0.002s
I20260709 12:25:22.801380  7588 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.802145  7588 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:22.804723  7588 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:22.770660  7479 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:22.808099  7479 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:22.757324  7450 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:22.812867  5942 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:22.811533  6766 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:25:22.813951  6766 pending_rounds.cc:85] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 2
I20260709 12:25:22.814414  6766 pending_rounds.cc:107] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:25:22.814747  6766 raft_consensus.cc:2889] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:22.759388  7403 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:22.760522  7414 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:25:22.819775  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.826506  5949 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:22.707365  7491 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.093s
I20260709 12:25:22.835945  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:22.837160  6024 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
W20260709 12:25:22.838918  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 212666us
W20260709 12:25:22.832381  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 196218us
I20260709 12:25:22.812532  7313 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.849109  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:22.857223  6766 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:22.860190  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:22.813279  7576 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:25:22.843427  7300 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.863s	user 0.012s	sys 0.002s
I20260709 12:25:22.874066  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:22.874176  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:22.879261  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:22.880259  5948 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:22.844709  7404 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.844059  7436 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.836311  7414 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:22.883152  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 100607us
I20260709 12:25:22.834580  7278 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:22.895680  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:22.897900  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.869166  7351 raft_consensus.cc:2764] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:22.915259  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:22.916105  5869 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:22.845546  7403 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:22.919665  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:22.872522  7262 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.932578  6020 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:25:22.933764  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 170177us
W20260709 12:25:22.934664  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 69330.4us
I20260709 12:25:22.931465  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:22.866953  7313 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:22.917891  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:22.953425  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:22.886400  7576 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:22.954321  5948 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:22.930757  7444 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.955655  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:22.962262  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.964619  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:22.965962  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:22.964591  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:22.919224  7439 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:25:22.912216  7436 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:22.967425  5949 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:22.904754  7595 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.217s
I20260709 12:25:22.927047  7404 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:22.949564  7585 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:25:22.927048  7278 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:23.011997  7278 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:23.011997  7517 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.672s	user 0.008s	sys 0.004s
W20260709 12:25:23.013464  7604 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:25:23.014704  7574 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.662s	user 0.003s	sys 0.009s
I20260709 12:25:23.013926  7517 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.017057  7479 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:23.018404  5872 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:22.951295  7262 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:23.020505  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:23.020661  7483 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:23.021322  5873 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:23.022066  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.023034  5874 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:23.023097  7580 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:22.576719  6775 rpcz_store.cc:276] 0709 12:25:21.289377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.485819 (+196442us) service_pool.cc:224] Handling call
0709 12:25:21.486293 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:21.547259 (+ 60966us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:21.758508 (+211249us) log.cc:844] Serialized 14 byte log entry
0709 12:25:22.133962 (+375454us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:22.134075 (+   113us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:22.134175 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.514966 (+380791us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b54002a17d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:22.515067 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.515288 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27082,"prepare.queue_time_us":250952,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":40391,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":9506,"spinlock_wait_cycles":27684096,"thread_start_us":482787,"threads_started":3,"wal-append.queue_time_us":218623}
I20260709 12:25:23.024012  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:23.023789  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:23.030170  5873 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:22.950182  7402 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:22.983404  6766 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:22.990216  7439 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:22.991698  7414 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:22.995819  7404 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:22.966176  7351 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:22.968165  7300 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:25:22.998370  7403 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:22.998852  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.080901  7597 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:23.081760  7597 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.082396  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.083606  5865 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:22.999986  7576 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:23.002075  5945 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:23.002287  7436 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:23.002568  7501 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.970s	user 0.013s	sys 0.002s
I20260709 12:25:22.972206  7444 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:22.997395  7373 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:23.007534  7313 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:22.982735  7457 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:23.030910  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.024798  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:22.978749  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:23.017302  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:23.050631  5869 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:23.022060  7483 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:22.983937  7496 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:23.054383  7369 consensus_meta.cc:287] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.897s	user 0.013s	sys 0.000s
I20260709 12:25:23.054637  7402 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:23.056227  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.060509  7299 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:23.066504  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:23.049817  7439 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:23.001233  7587 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:23.024102  7580 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:23.123695  5949 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:23.123842  7580 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:23.125016  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:23.088182  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.085490  7608 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:23.086068  7457 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.126219  5870 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:23.126163  7608 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.126809  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.127719  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.127827  7457 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:23.128082  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:23.086896  5867 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:23.089015  7450 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:23.089507  7496 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.096086  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:23.130079  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:23.130628  5947 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:23.130271  7439 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:23.087061  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 108351us
I20260709 12:25:23.015173  7562 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:23.131331  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.131922  7562 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:23.086918  7368 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.132851  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:23.132937  7562 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.125200  7580 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.134231  7368 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:23.134469  7404 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.128741  5869 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:23.135142  7444 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:23.087284  7313 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:23.137727  5867 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 4: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:23.137989  7313 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.138473  7313 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.110106  7609 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:23.139242  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.139510  7609 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.111025  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:23.142541  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.142685  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:23.143045  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.143559  5941 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:23.143891  6766 raft_consensus.cc:2410] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 7 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:25:23.144057  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.144600  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:23.145267  5941 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:23.088336  7483 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:23.152449  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.153291  6022 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:25:23.115818  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 49433us
I20260709 12:25:23.131247  7496 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:23.114718  7501 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:23.128538  7608 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:23.129038  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.015203  7574 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.159752  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.160699  6015 raft_consensus.cc:2410] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:23.160890  7586 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:23.161017  5870 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:23.162287  7574 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:23.165005  7489 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.001s
I20260709 12:25:23.165536  7489 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:23.165831  7517 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:23.075898  7596 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:23.166589  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.167129  5871 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:23.170034  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.170529  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:23.170992  6766 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 7: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:23.084647  7597 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:23.111872  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 94630.9us
I20260709 12:25:23.145833  7576 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:23.146526  6775 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:23.146817  7598 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.147197  7402 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:23.125313  7587 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:23.147298  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.134294  7562 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:23.146569  7292 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:25:23.148805  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.214663  6018 raft_consensus.cc:2410] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:23.149590  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.215437  6020 raft_consensus.cc:2393] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:23.150002  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.149930  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.151085  7610 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:23.087392  7262 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:23.131683  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:23.111150  7607 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:23.152221  7609 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:23.135162  7404 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:23.085528  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.145401  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.171804  7499 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:23.169395  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.183773  5873 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:23.260332  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.259055  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:23.184288  5838 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:23.180629  5990 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:23.177037  7278 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:23.259032  7610 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:23.269857  7529 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.270874  5873 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:23.271158  6757 raft_consensus.cc:2410] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:23.187641  7597 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.272526  7596 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:23.273015  7597 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.275344  7597 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:23.188119  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.275848  5870 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3. Ops: []
W20260709 12:25:23.276803  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.131589  7439 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:23.196403  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.286371  6015 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:23.240705  6766 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:23.245286  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.134616  7580 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.210125  7484 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:23.258749  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.300982  6019 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:23.226055  6778 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
W20260709 12:25:23.265671  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 85105.1us
I20260709 12:25:23.241706  7402 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.258808  7592 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:23.192301  5869 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
W20260709 12:25:23.269920  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 85710.2us
I20260709 12:25:23.308228  5838 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:23.291487  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.293717  5867 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
W20260709 12:25:23.295075  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.312676  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:23.289227  6022 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 2.
I20260709 12:25:23.306615  7592 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:23.314833  5947 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:23.315860  7517 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:23.315505  7592 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:23.316290  7517 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.316433  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:23.317251  7501 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:23.317804  5941 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:23.262945  7609 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.317943  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.318591  7609 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:23.320932  7609 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:23.321774  5865 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 6: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:23.322353  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:23.323213  6019 raft_consensus.cc:2393] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:23.323083  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.323859  6757 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:23.324918  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.211257  7598 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:23.326284  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.316716  7517 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.324959  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.254678  7329 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:23.326941  6773 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:23.330297  5838 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:23.328238  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:23.331331  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.316754  7592 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.329293  5990 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:23.337006  5874 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:25:23.334299  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.332733  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:23.344198  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 379628us
I20260709 12:25:23.254205  7570 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:23.346987  5838 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:23.343945  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.351495  6019 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:23.349246  6762 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:23.329865  7292 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:23.305826  7589 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:23.378026  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.315541  7597 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:25:23.352730  7501 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:23.381695  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.350239  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.345985  6757 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:23.398977  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.400000  6757 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:23.400380  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.401153  6793 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:23.401275  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.404217  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:23.356343  7605 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:25:23.462224  5866 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:23.387404  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:23.389043  6766 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:23.389560  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:23.467432  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 80182.9us
I20260709 12:25:23.467667  6019 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:23.333191  7402 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:23.394979  7625 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.395540  7484 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:23.356755  7595 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:23.362874  7529 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:23.363322  7596 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:23.420881  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:23.388635  7501 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.424695  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:23.426916  6785 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:23.344782  7329 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.424497  7570 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:23.362386  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.381708  7262 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:25:23.365952  7313 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:23.390074  7610 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:23.425395  7404 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:23.410790  7598 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:23.378742  7292 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 12:25:23.432772  7499 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:23.443377  6793 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:23.419502  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.409793  6768 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:23.454411  5990 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:23.455612  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:23.444653  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.474506  7630 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:23.477320  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.482298  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:23.494277  5943 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:23.494891  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.497092  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.484074  7484 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.492959  7501 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.536082  5867 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:23.536798  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.474946  6757 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:23.540740  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.482548  7595 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.412858  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:23.545022  5944 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:23.548012  6023 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:25:23.517190  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.530714  6019 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:23.533381  6017 raft_consensus.cc:2393] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:23.508383  6766 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:23.535517  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.557693  6020 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:23.559142  6793 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:25:23.535951  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:23.540118  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 172315us
I20260709 12:25:23.560213  6016 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:23.561025  5838 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:23.541349  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 63376.9us
I20260709 12:25:23.542798  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:23.562680  6024 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:23.562713  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:23.563654  7587 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:23.544056  7486 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:23.566613  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 112331us
I20260709 12:25:23.559716  6793 pending_rounds.cc:85] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 2
I20260709 12:25:23.508658  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.567833  6793 pending_rounds.cc:107] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:25:23.544104  7483 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:23.490899  7625 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:23.573268  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.568120  6793 raft_consensus.cc:2889] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:23.570837  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:23.555190  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:23.575892  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:23.551205  7595 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.581074  6766 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:23.584368  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.551972  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:23.533030  7585 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:25:23.584972  6023 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
W20260709 12:25:23.590147  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 55263.4us
I20260709 12:25:23.595526  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:23.571229  7589 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:23.566212  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:23.599491  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:23.569118  7483 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:23.606014  6021 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:23.583122  7607 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:23.586879  7529 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:23.586114  7414 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:23.642283  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:23.589563  7605 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.672503  7605 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:23.673910  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.643254  5944 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:23.656807  5866 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:23.660466  7457 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:23.661996  5948 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:23.661422  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 65983.3us
W20260709 12:25:23.666340  7635 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 12:25:23.666652  7638 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:25:23.566323  7486 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.586648  7278 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.069s
I20260709 12:25:23.644703  7483 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:23.599591  7589 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.654665  7636 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.698585  7636 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:23.701015  7636 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:23.680641  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:23.702148  7368 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:23.703774  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:23.704028  7597 raft_consensus.cc:3055] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:23.677770  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:23.689075  5942 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:23.679762  7457 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.715700  7457 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.717965  7329 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:23.720368  7486 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:23.721992  7402 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:23.725837  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.726764  5867 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:23.727166  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:23.727864  6016 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:23.599735  5941 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:23.655117  7517 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:23.714687  7368 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:23.738663  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.714859  7585 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:23.739428  7517 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:23.727934  5874 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:23.731741  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.735023  7625 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:23.740926  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:23.743391  5944 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:23.706642  7589 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:23.746775  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.747551  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:23.748736  7450 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:23.760553  5947 raft_consensus.cc:2410] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:23.762933  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:23.765666  5874 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:23.768261  7313 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:23.768772  7536 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:23.769513  7536 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:23.770570  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 349768us
I20260709 12:25:23.772181  7536 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:23.772468  7284 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:23.773000  7284 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.773370  7284 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.774912  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:23.775238  7639 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:25:23.776821  7585 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:23.776528  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.712001  7597 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:23.778195  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:23.780382  7517 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:23.782028  7517 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.785261  7517 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:23.786842  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:23.721613  7486 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.056s
W20260709 12:25:23.767884  7562 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.551s	user 0.010s	sys 0.002s
I20260709 12:25:23.778005  7450 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.795507  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:23.780182  7585 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:23.797617  7605 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:23.795749  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 153563us
I20260709 12:25:23.809595  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:23.810894  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:23.811589  5866 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:23.813005  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:23.776479  7484 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.797178  7586 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:23.795168  7368 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.787760  5945 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:23.802457  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.802721  7585 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.803483  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:23.813830  7430 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:23.783543  7637 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.088s	user 0.000s	sys 0.004s
I20260709 12:25:23.796355  5944 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.818224  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:23.823904  6022 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 3: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:23.837823  7501 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.834332  7368 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:23.840337  7501 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:23.840600  7368 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:23.834954  7430 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:23.842084  7430 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.815073  7605 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:23.845943  7605 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:23.846172  7305 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:23.846827  7305 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:23.847500  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.848062  7305 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:23.830821  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 52729.7us
I20260709 12:25:23.849821  7436 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:23.850852  7484 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:23.851181  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:23.851471  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:23.852509  7586 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.853330  7586 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:23.823400  7630 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:23.846405  7605 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:23.855968  6019 raft_consensus.cc:2368] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6.
I20260709 12:25:23.855922  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.845968  7597 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:23.857914  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:23.858033  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 171) took 1407 ms. Trace:
I20260709 12:25:23.858630  6778 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:23.857049  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:23.845427  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:23.859398  6762 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.858067  7597 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:23.858125  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:23.865482  7586 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:23.856707  6768 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:23.816735  7450 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.866119  7586 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:25:23.867432  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 46452.4us
I20260709 12:25:23.867277  7586 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:23.868211  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:23.869412  7484 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.870255  7499 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:23.870762  7499 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.871203  7499 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.872308  6611 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:23.874505  5913 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:23.874572  7368 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:23.875286  7368 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.875665  7368 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:23.563205  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:23.870034  7484 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:23.882051  5913 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:23.883584  7436 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:23.884064  7436 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.884503  7436 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:23.885733  7587 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:23.886305  7587 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.876514  7608 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:23.892148  5915 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:23.893247  6021 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:23.894335  6020 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:23.892161  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:23.898075  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46693.6us
I20260709 12:25:23.832479  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:23.834359  6609 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:23.902117  6611 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:23.889925  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:23.902469  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 70055.3us
I20260709 12:25:23.904305  5990 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:23.903939  7576 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:23.903539  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 45542.7us
W20260709 12:25:23.906826  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.908337  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:23.909919  5840 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:23.910079  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 78236us
I20260709 12:25:23.912102  5867 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:23.918659  5944 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:23.919823  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:23.937196  5948 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.882735  5697 catalog_manager.cc:5697] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:23.937753  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:23.923418  5915 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:23.913354  6609 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:23.929919  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.929461  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.905954  7630 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:23.953727  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 45449.4us
I20260709 12:25:23.954319  5946 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:23.906569  7592 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:23.957042  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:23.939877  5868 raft_consensus.cc:2376] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7. Re-sending same reply.
I20260709 12:25:23.906545  7608 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:25:23.971366  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68040.2us
I20260709 12:25:23.982632  5840 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:23.982923  5990 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:23.907982  7501 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.990736  6611 leader_election.cc:400] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6.
I20260709 12:25:23.969548  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:23.922489  7484 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:23.929729  7369 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:23.971103  7613 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:23.978705  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:24.005861  5874 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:23.979560  7496 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:23.922362  7313 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 12:25:23.991216  6611 leader_election.cc:403] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:25:23.922374  7536 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:23.993968  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:23.923172  7486 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:23.926553  7262 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:25:24.001101  5695 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40812 (request call id 140) took 11641 ms (client timeout 15000 ms). Trace:
I20260709 12:25:24.001547  7596 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:23.971000  7439 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:24.035480  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 78004.5us
W20260709 12:25:24.036783  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 104215us
I20260709 12:25:24.037568  6611 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:24.037904  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:24.040398  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46549.3us
I20260709 12:25:24.040997  5948 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:23.952952  7630 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.975068  5697 catalog_manager.cc:5697] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:24.022516  6022 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:23.929515  7607 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 12:25:23.978123  7625 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:23.984014  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:24.021759  7496 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.042115  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:24.047379  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:24.054550  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:24.042456  5838 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:24.002360  7292 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.006240  7484 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.998323  7537 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.029929  7536 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.030180  7486 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.035401  7577 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:24.030241  7438 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.099426  5697 catalog_manager.cc:5697] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:24.003527  7501 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.067255  7630 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:24.078399  6017 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:24.056439  7439 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.001168  7457 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:24.080709  6020 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:24.036231  7369 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:24.065666  7598 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:24.074043  6775 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:24.066246  7625 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:24.079300  7414 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:23.858570  5940 rpcz_store.cc:276] 0709 12:25:22.451058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.463763 (+ 12705us) service_pool.cc:224] Handling call
0709 12:25:22.573350 (+109587us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:22.811692 (+238342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:22.813027 (+  1335us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:22.927814 (+114787us) log.cc:844] Serialized 52 byte log entry
0709 12:25:23.222052 (+294238us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:23.305070 (+ 83018us) log.cc:844] Serialized 14 byte log entry
0709 12:25:23.760533 (+455463us) spinlock_profiling.cc:243] Waited 7.62 ms on lock 0x7b18003007c0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb10c0a 00007f39d9250219 00007f39cfb98a40 00007f39cfb934e4 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287 00007f39d2c9e208
0709 12:25:23.777981 (+ 17448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.783279 (+  5298us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:23.783931 (+   652us) raft_consensus.cc:1685] finished
0709 12:25:23.794408 (+ 10477us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:23.857796 (+ 63388us) spinlock_profiling.cc:243] Waited 46.5 ms on lock 0x7b54006725d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:23.857934 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3677,"prepare.queue_time_us":41500,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":524689,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":255,"spinlock_wait_cycles":124452352,"thread_start_us":351958,"threads_started":2,"wal-append.queue_time_us":453341}
I20260709 12:25:24.035037  7529 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 12:25:24.143056  7607 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.130187  7284 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:24.122699  7577 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:24.150374  7607 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:24.151252  7577 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:24.152113  7284 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:24.152657  7607 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:24.154590  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:24.156858  5945 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:24.160483  7576 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:24.161243  7576 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:24.162102  7313 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:24.076989  7305 consensus_queue.cc:1059] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 12:25:24.166255  7607 raft_consensus.cc:3055] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:24.166764  7607 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:24.168344  7607 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.144364  7414 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.170222  7414 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.097199  7484 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:24.117693  7608 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:24.128129  7598 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:24.102591  7595 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:24.123473  7438 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:24.109072  5697 catalog_manager.cc:5697] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:24.082810  7329 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:24.091181  7562 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:24.082322  7496 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.087137  7610 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.088011  7536 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.098589  7486 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.077266  7588 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 12:25:24.120018  7439 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.146209  7625 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:24.147208  7649 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:25:24.208439  6773 raft_consensus.cc:2368] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4.
I20260709 12:25:24.224334  6762 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
W20260709 12:25:24.243769  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.242328  7587 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:24.273116  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 235262us
I20260709 12:25:24.315961  5697 catalog_manager.cc:5697] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:24.267095  7652 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.097s	user 0.002s	sys 0.001s
W20260709 12:25:24.194761  7644 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
W20260709 12:25:24.266960  7658 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.072s	user 0.002s	sys 0.000s
I20260709 12:25:24.201918  7457 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:24.266669  7651 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.056s	user 0.002s	sys 0.001s
I20260709 12:25:24.207252  7598 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:24.266870  7654 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.077s	user 0.001s	sys 0.006s
I20260709 12:25:24.214947  7438 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:24.216195  7562 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:24.363287  7562 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.215755  7329 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:24.366747  7329 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:24.194648  7484 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:24.315194  7653 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.124s	user 0.003s	sys 0.003s
I20260709 12:25:24.340859  7430 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:24.032378  5695 rpcz_store.cc:269] 0709 12:25:12.359742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.359994 (+   252us) service_pool.cc:224] Handling call
0709 12:25:21.323260 (+8963266us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b540077f7a8. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d65ed161 00007f39d66773af 00007f39d6678b11 00007f39d8b78fa1 00007f39d8b797ff 00007f39d8a38a1b 00007f39d8b4090e 00007f39d70012a4 00007f39d7001232 00007f39d7001161 00007f39d70010dc
0709 12:25:24.000892 (+2677632us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:21.803134 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:21.830741 (+ 27607us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:21.830795 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:25:21.830831 (+    36us) tablet.cc:662] Decoding operations
0709 12:25:21.831547 (+   716us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:21.831757 (+   210us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:21.831784 (+    27us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:25:21.832292 (+   508us) tablet.cc:701] Row locks acquired
0709 12:25:21.832341 (+    49us) write_op.cc:260] PREPARE: finished
0709 12:25:22.024526 (+192185us) spinlock_profiling.cc:243] Waited 35.9 ms on lock 0x7b2c0005c178. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158ba0a 00007f39d158b693 00007f39d158b631 00007f39d2cbe902 00007f39d2cbe6dd 00007f39d66b9243 00007f39d66b8d58 00007f39d66bd9f1 00007f39d66bd9a9 00007f39d66bd939
0709 12:25:22.024651 (+   125us) write_op.cc:270] Start()
0709 12:25:22.024917 (+   266us) write_op.cc:276] Timestamp: P: 1783599921873863 usec, L: 0
0709 12:25:22.024962 (+    45us) op_driver.cc:348] REPLICATION: starting
0709 12:25:22.031074 (+  6112us) log.cc:844] Serialized 1681 byte log entry
0709 12:25:22.505162 (+474088us) op_driver.cc:464] REPLICATION: finished
0709 12:25:22.882086 (+376924us) write_op.cc:301] APPLY: starting
0709 12:25:22.905094 (+ 23008us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:22.938692 (+ 33598us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:22.954506 (+ 15814us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:23.007830 (+ 53324us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:23.008078 (+   248us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:25:23.008141 (+    63us) write_op.cc:312] APPLY: finished
0709 12:25:23.008492 (+   351us) log.cc:844] Serialized 40 byte log entry
0709 12:25:23.008691 (+   199us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:23.008797 (+   106us) write_op.cc:454] Released schema lock
0709 12:25:23.008898 (+   101us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:23.009302 (+   404us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:23.009328 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":993915,"reported_tablets":8,"spinlock_wait_cycles":5129984,"child_traces":[["op",{"apply.queue_time_us":325284,"apply.run_cpu_time_us":3449,"apply.run_wall_time_us":152167,"num_ops":4,"prepare.queue_time_us":371232,"prepare.run_cpu_time_us":26020,"prepare.run_wall_time_us":382199,"raft.queue_time_us":274015,"raft.run_cpu_time_us":33056,"raft.run_wall_time_us":259675,"replication_time_us":480021,"spinlock_wait_cycles":82668928,"thread_start_us":739197,"threads_started":4,"wal-append.queue_time_us":180199,"wal-append.run_cpu_time_us":42402,"wal-append.run_wall_time_us":1803004}]]}
I20260709 12:25:24.346977  5697 catalog_manager.cc:5697] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:24.352074  7501 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:24.216744  7517 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.352650  7636 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:24.208492  7576 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:24.326798  7657 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.138s	user 0.004s	sys 0.000s
I20260709 12:25:24.345808  6609 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:24.390446  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 44729.5us
I20260709 12:25:24.389850  5697 catalog_manager.cc:5697] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:24.349905  7585 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.344480  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 441) took 1020 ms. Trace:
I20260709 12:25:24.324196  7595 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:24.332175  7599 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6.
W20260709 12:25:24.418370  7368 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.542s	user 0.012s	sys 0.000s
I20260709 12:25:24.418845  7368 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:24.420995  7368 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:24.404752  7630 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:24.423241  7597 consensus_meta.cc:287] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.562s	user 0.012s	sys 0.000s
I20260709 12:25:24.423429  7630 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:24.421308  7536 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:24.424928  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:24.426020  5940 raft_consensus.cc:2410] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:24.375258  7369 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:24.181586  7483 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:24.202854  7589 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.384667  7587 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:24.434438  7536 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:24.403321  7499 consensus_meta.cc:287] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.532s	user 0.012s	sys 0.000s
W20260709 12:25:24.407462  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.406265  6773 rpcz_store.cc:276] 0709 12:25:23.323633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.419117 (+ 95484us) service_pool.cc:224] Handling call
0709 12:25:24.333334 (+914217us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b5400007b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:24.344388 (+ 11054us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1611286272}
I20260709 12:25:24.411392  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:24.413612  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:24.415130  7605 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.560s	user 0.012s	sys 0.001s
I20260709 12:25:24.412791  7438 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:24.416443  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:24.400568  7450 consensus_meta.cc:287] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.534s	user 0.007s	sys 0.003s
I20260709 12:25:24.440579  5697 catalog_manager.cc:5697] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:24.389422  7636 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:25:24.429955  7436 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.005s
I20260709 12:25:24.454272  6015 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:24.463605  5697 catalog_manager.cc:5697] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:24.446532  7537 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:24.475109  7537 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.453176  7262 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:25:24.491557  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 449456us
I20260709 12:25:24.502662  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:24.427779  7585 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.460692  7438 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.505573  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:24.513566  6762 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:24.507318  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:24.509436  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:24.477595  7537 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:24.519577  5987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.014s	sys 0.013s
W20260709 12:25:24.520155  5987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.014s	sys 0.013s
I20260709 12:25:24.451121  7499 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.524998  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:24.524036  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:25:24.689936  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 165980us
I20260709 12:25:24.691200  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:24.479425  7636 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:24.622574  7439 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:24.516844  7438 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.589541  7597 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.512490  7574 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:24.526175  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:24.549551  7430 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:24.651681  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:24.590380  7589 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:24.547269  7262 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:24.522650  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:24.538689  7592 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:24.561513  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:24.526192  7630 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.480997  7436 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:24.642858  5913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.010s	sys 0.020s
I20260709 12:25:24.604121  7625 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:24.623477  7414 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:24.640839  7483 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:24.478344  7450 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.728953  7439 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:24.457617  7605 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:24.730325  7414 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:24.731376  7605 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:24.513057  6785 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:25:24.592291  5838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.007s	sys 0.025s
W20260709 12:25:24.732124  5838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.007s	sys 0.029s
I20260709 12:25:24.732246  6793 raft_consensus.cc:2368] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4.
W20260709 12:25:24.732504  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 690074us
W20260709 12:25:24.641390  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.012s	sys 0.014s
I20260709 12:25:24.635826  6018 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:24.733096  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.658484  7672 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:24.669775  7585 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:24.670090  7499 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:24.677430  7496 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.739936  7496 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:24.741729  6021 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:24.657563  7598 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:24.695279  5990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.019s	sys 0.035s
W20260709 12:25:24.751291  5990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.019s	sys 0.035s
I20260709 12:25:24.751325  6019 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:24.752336  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 769433us
I20260709 12:25:24.745043  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:24.690008  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 165093us
I20260709 12:25:24.695921  7292 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:24.698830  7511 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:24.716130  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.696192  7662 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.703645  7486 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:24.723382  5913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.010s	sys 0.020s
I20260709 12:25:24.724489  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:24.719978  6778 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:24.726473  5871 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:24.705734  7666 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.731739  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:24.733340  6024 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:24.664101  7517 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:24.766049  5874 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:24.763579  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:24.767439  5873 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:24.733099  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.012s	sys 0.014s
I20260709 12:25:24.734203  6016 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:24.704625  7664 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:24.737618  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:24.776404  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:24.777323  5871 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:24.778609  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:24.780001  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:24.712436  7597 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:24.699460  7673 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:24.743048  6775 raft_consensus.cc:2368] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5.
I20260709 12:25:24.716583  7574 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:24.762885  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:24.762257  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:24.763435  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 825691us
W20260709 12:25:24.768256  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 807090us
I20260709 12:25:24.801047  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:24.758380  7511 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:24.807298  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:24.717864  7663 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.808215  7479 raft_consensus.cc:2764] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:24.808393  7663 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:24.809085  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:24.759023  7439 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.757162  7671 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.809899  7439 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:24.810398  7479 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:24.810724  7671 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:24.812215  7439 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:24.812832  7439 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:24.756335  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:24.813912  5946 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:24.814186  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:24.813777  7608 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.813359  7439 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.774190  5990 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:24.815187  7608 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.816150  7414 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:24.816880  7439 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:24.816726  7414 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.817422  7608 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:24.783248  7664 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:24.805905  5868 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 7: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:24.818862  7430 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:24.818996  7414 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:24.819515  7430 raft_consensus.cc:697] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:24.819975  7483 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:24.820488  7483 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:24.820645  5867 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:24.815199  7674 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:24.820513  7430 consensus_queue.cc:237] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.813736  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:24.813091  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:24.822510  5948 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:24.656484  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:24.809307  7329 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:24.823733  7329 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:24.824195  7329 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.824471  5915 leader_election.cc:400] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4.
I20260709 12:25:24.825254  7589 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:24.825959  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:24.826737  5942 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:24.825258  7479 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:25:24.828882  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:24.830061  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:24.805083  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:24.830253  5942 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 6: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:24.831947  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:24.833055  5874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
mode: GRACEFUL
new_leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
 from {username='slave'} at 127.0.0.1:45236
I20260709 12:25:24.821962  7674 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:24.833384  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:24.833779  6773 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:24.833966  7674 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:24.834509  5942 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:24.835769  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:24.820986  7483 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.836483  6773 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.836776  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 451) took 1062 ms. Trace:
I20260709 12:25:24.837433  6793 rpcz_store.cc:276] 0709 12:25:23.774396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.776248 (+  1852us) service_pool.cc:224] Handling call
0709 12:25:24.836448 (+1060200us) spinlock_profiling.cc:243] Waited 954 ms on lock 0x7b540052a154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:24.836658 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2193977728}
I20260709 12:25:24.839289  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:24.840065  5870 raft_consensus.cc:2376] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3. Re-sending same reply.
I20260709 12:25:24.840413  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:24.841531  7607 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.842360  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:24.846877  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:24.845453  6793 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:24.852370  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:24.853935  5940 raft_consensus.cc:2410] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:24.854678  7483 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:24.854892  7329 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.857322  5943 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:24.858880  7483 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:24.824898  5915 leader_election.cc:403] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:24.860154  7329 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:24.860275  7607 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.873488  6793 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:25:24.874194  6793 pending_rounds.cc:85] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 3
I20260709 12:25:24.874531  6793 pending_rounds.cc:107] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:25:24.874840  6793 raft_consensus.cc:2889] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:24.878051  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:24.878916  5871 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.727790  7630 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:24.762481  7496 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.114s
W20260709 12:25:24.885103  7682 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:25:24.885531  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:24.886895  7486 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:24.887720  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:24.888489  7640 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:25:24.869732  7511 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.896965  7436 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:24.899777  5873 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:24.904428  6773 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:24.905165  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:24.882570  7292 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:24.750612  7672 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:24.765926  7278 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.112s
I20260709 12:25:24.902681  7450 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:24.919086  5869 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:24.782729  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:24.921855  5871 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:24.922180  6023 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
W20260709 12:25:24.925154  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 160657us
I20260709 12:25:24.927110  5915 leader_election.cc:400] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4.
I20260709 12:25:24.942926  5915 leader_election.cc:403] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:24.928131  7329 consensus_queue.cc:1059] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:24.942276  5873 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:25:24.943519  5873 pending_rounds.cc:85] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 1
I20260709 12:25:24.943897  5873 pending_rounds.cc:107] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:25:24.944150  5873 raft_consensus.cc:2889] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:24.944224  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:24.945858  7305 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.946151  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:24.946468  7305 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:24.775285  7666 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:24.947999  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:24.948873  7305 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:24.949357  7574 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:24.940436  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:24.950382  5940 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:24.950693  6785 raft_consensus.cc:2410] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:24.951838  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:24.954639  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 572) took 1088 ms. Trace:
I20260709 12:25:24.955122  6757 rpcz_store.cc:276] 0709 12:25:23.854631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.909521 (+ 54890us) service_pool.cc:224] Handling call
0709 12:25:24.920711 (+1011190us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b540004d8d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:24.943119 (+ 22408us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1465598208}
I20260709 12:25:24.956573  7636 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.946856  6775 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:24.956945  7438 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:24.958627  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:24.959396  6756 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:24.959612  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:24.949700  5874 raft_consensus.cc:606] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Received request to transfer leadership to TS 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:24.960315  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:24.960767  5874 raft_consensus.cc:614] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:24.964630  7662 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:24.966234  7436 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.967031  7436 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:24.969954  7436 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:24.971462  5869 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.920858  7496 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:24.976310  5990 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:24.961213  5947 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.990298  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:24.991715  6762 raft_consensus.cc:2410] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:24.994570  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:24.995595  5948 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:24.913167  5942 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:24.922465  7292 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.806177  7673 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:25.007212  5869 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:25.004083  5873 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:24.997788  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:25.008631  5943 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 5
W20260709 12:25:25.009176  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 82283.4us
I20260709 12:25:25.013913  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:25.016500  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:25.016911  7663 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:25.018069  5872 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:25.019240  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:25.019845  5872 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.986232  7496 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:25.017060  6768 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:25.022382  7496 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:25.023051  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:25.023655  7496 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:25.023957  6768 raft_consensus.cc:2410] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:25.023350  5947 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:25.024906  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:25.024899  6778 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:25.027998  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:25.029062  6778 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.003495  7436 raft_consensus.cc:3055] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:25.026245  5944 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:25:25.018134  5946 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:25.025729  6768 raft_consensus.cc:2368] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5.
I20260709 12:25:25.031438  7369 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.048240  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:25.048544  7369 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:25.049044  5946 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:24.953537  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:25.050280  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:25.050743  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:25.029974  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:25.051491  7587 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:25.051833  6793 raft_consensus.cc:2368] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5.
I20260709 12:25:25.052016  7587 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:25.052839  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:25.053105  7587 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:25.055250  5872 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:25.055835  7597 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:25.056253  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:25.057917  5872 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:25:25.058161  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:25.059399  6785 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:25.059700  6609 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:25.025408  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:25.060761  6785 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:25.061420  7369 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:25.053521  6793 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:25.063633  7597 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:25.064287  7597 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:25.064939  7436 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:25.070706  7436 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:25.066946  7596 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.076288  7630 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:25.076690  7596 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:25.078948  7596 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:25.066747  7369 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:25.082442  5947 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:25:25.084774  7625 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:25.002769  7483 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.087s
I20260709 12:25:25.070070  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:25.090039  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:25.091137  6773 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:25.070129  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:25.092408  5948 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.960292  7438 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:24.990505  7605 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.065346  7597 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:25.072611  7666 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:25.033067  7517 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.031893  7450 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:25:25.075568  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:25.076947  7436 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:25.066897  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:25.071887  7673 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:25.093577  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:25.162081  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 68605.8us
I20260709 12:25:25.046629  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:25.003096  7589 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:25.107811  7675 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.081s
I20260709 12:25:25.006888  7292 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:25.103586  7596 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:25:25.106928  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 271506us
I20260709 12:25:25.103878  7305 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:25.151600  5943 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:25.152022  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:25.166209  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 99404.5us
I20260709 12:25:25.198788  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:25:25.165710  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 119205us
I20260709 12:25:25.221525  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:25.221817  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:25.044560  7329 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:25.164942  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:25.171303  7589 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:25.225024  5940 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:25.238587  7671 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:25.223186  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:25.240109  6793 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:25:25.240576  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:25.241050  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:25.241969  5942 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:25.242358  6793 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:25.201015  7599 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:25.243059  7599 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:25.244117  7599 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:25.205978  7596 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:25.248013  7596 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:25.248690  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:25.204840  7661 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.204248  7305 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.152534  7605 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:25.192135  7496 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:25.089330  7562 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:25.236898  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.237951  7589 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:25.251686  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:25.249454  6768 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:25.167490  7479 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:25.159713  7430 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:25.258455  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 59782us
I20260709 12:25:25.259824  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:25.245491  7599 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:25.234891  7675 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:25.229350  7688 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:25.321766  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 62102us
I20260709 12:25:25.216552  6757 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
W20260709 12:25:25.305262  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.315636  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:25.312453  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:25.318140  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:25.163870  7517 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:25.335084  5874 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:25.271431  7671 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:25.335660  5867 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:25.344014  6768 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:25.317438  7675 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.267972  7661 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:25.255910  7483 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:25:25.321451  7684 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
I20260709 12:25:25.342996  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:25.398420  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 55556.3us
I20260709 12:25:25.393672  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:25.393144  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:25.358793  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:25.401207  6017 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:25.267418  7630 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.106s
I20260709 12:25:25.344841  6609 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:25.362447  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:25.255296  7414 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:25.314119  7664 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:25.386695  5867 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 12:25:25.270411  7450 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.267704  7305 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.337262  7587 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:25.322744  7479 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:25.277861  7486 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:25.397217  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:25.546068  6023 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:25.547662  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:25.266499  5948 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:25.416882  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:25.421954  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:25.550742  6015 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
W20260709 12:25:25.448237  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 103496us
W20260709 12:25:25.494846  5990 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.2.4.254:41049, user_credentials={real_user=slave}} blocked reactor thread for 96579.3us
I20260709 12:25:25.340982  7596 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:25.355934  7666 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:25.373378  7607 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4.
I20260709 12:25:25.388685  7675 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.345466  7430 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:25.567184  6016 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:25.336638  7688 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:25.570006  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:25.437888  7636 raft_consensus.cc:2764] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 12:25:25.529946  7671 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.531706  7499 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:25:25.569258  5990 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.653s (SENT) [suppressed 12 similar messages]
I20260709 12:25:25.497267  6778 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:25.591674  6024 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:25.398785  7577 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.168s
I20260709 12:25:25.539947  7450 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:25.580837  7438 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:25.587052  7511 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.659s	user 0.012s	sys 0.000s
I20260709 12:25:25.584605  7587 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:25.578424  7673 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:25.612301  5867 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.612427  5871 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:25.617990  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:25.611848  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:25.622421  5913 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:25.594897  7517 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:25.568526  7596 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:25.580277  7585 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:25:25.630805  5840 leader_election.cc:400] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5.
I20260709 12:25:25.635365  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:25.589452  7486 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:25:25.603258  7607 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:25.609491  7669 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:25.623971  7694 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.642306  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:25.660988  6024 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:25.611172  7329 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4.
I20260709 12:25:25.652496  5871 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:25.613119  7587 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.599805  7671 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:25.617908  7690 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:25.599304  7499 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:25.664543  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 274751us
W20260709 12:25:25.673462  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 44358us
I20260709 12:25:25.693066  5913 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:25.697082  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 127867us
I20260709 12:25:25.783281  6023 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:25.619942  7511 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:25.687801  6049 heartbeater.cc:646] Failed to heartbeat to 127.2.4.254:41049 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.2.4.254:41049 timed out after 15.000s (SENT)
I20260709 12:25:25.696722  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:25.679755  7694 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:25.701257  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.706561  7696 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
W20260709 12:25:25.710294  7697 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:25:25.632117  7479 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:25.711320  7695 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:25:25.711639  7596 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:25.638039  7278 consensus_queue.cc:1059] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
W20260709 12:25:25.713305  7674 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.879s	user 0.010s	sys 0.002s
I20260709 12:25:25.713901  7329 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:25.713992  7262 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:25.681483  7587 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:25.785799  7630 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:25.715703  7704 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.688726  7666 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:25.831569  7698 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:25.673969  7517 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:25.832705  7517 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:25.833608  7517 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:25.834791  7517 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.835490  7694 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:25.724149  7496 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:25.838063  7496 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:25.689287  7605 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:25.731370  7430 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:25.733340  7688 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:25.739226  7661 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:25.659101  7607 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:25.656287  7669 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:25.658540  7699 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:25.794565  7577 consensus_queue.cc:1059] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:25:25.747087  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:25.748162  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:25.851025  5873 raft_consensus.cc:2410] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:25.747041  7675 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:25.751331  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:25.753188  7598 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:25.754433  7436 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.596s	user 0.013s	sys 0.000s
I20260709 12:25:25.627190  7673 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:25.756791  5944 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:25.760066  7305 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:25.695302  7690 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:25.762917  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:25.764035  7671 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:25.764822  7672 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:25.785574  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:25.842401  7704 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:25.875663  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:25.876598  6023 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:25.877076  7704 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:25.774838  7691 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.536s	user 0.005s	sys 0.002s
I20260709 12:25:25.793246  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:25.789121  6017 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
W20260709 12:25:25.790603  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 237131us
I20260709 12:25:25.811100  6022 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:25.714761  6024 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:25.877578  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:25.773697  7414 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:25.811544  7329 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.890393  7661 raft_consensus.cc:3055] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:25.889372  7673 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:25.940577  7414 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:25.881893  7669 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:25.942824  7669 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:25.889487  7436 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:25.946693  7436 raft_consensus.cc:697] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:25.948055  7436 consensus_queue.cc:237] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:25.879433  7511 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:25.880739  7496 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:25.892061  7690 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:25.878973  7704 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.952549  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:25.952831  7704 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:25.953863  7577 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:25.954423  7577 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:25.955324  7577 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:25.956267  7704 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:25.829038  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 636) took 1069 ms. Trace:
I20260709 12:25:25.806715  7636 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.957798  7636 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:25.958741  7663 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:25.959292  7663 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:25.743116  7625 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:25:25.960305  7686 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.861s	user 0.009s	sys 0.005s
I20260709 12:25:25.960562  7625 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:25.940654  7605 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:25.962387  7605 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:25.965102  7605 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:25.853971  5867 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:25.839222  7709 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.869226  7607 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:25.869319  7305 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:25.870760  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:25.819240  7710 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:25:25.874119  6021 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:25.845736  7699 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.840378  7630 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:25.787930  6611 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:25.873227  7675 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:25.873229  5946 raft_consensus.cc:2368] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5.
I20260709 12:25:25.878509  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:26.100407  5873 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:25.824437  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:25.894524  6023 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:25.886705  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:26.101722  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:26.102066  6017 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
W20260709 12:25:25.882727  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 89542.8us
I20260709 12:25:25.894284  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:26.103855  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:26.104450  6024 raft_consensus.cc:2410] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:25.940022  7329 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.832322  7666 raft_consensus.cc:697] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:25.961506  7625 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:25.957566  5872 rpcz_store.cc:276] 0709 12:25:24.759786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.059404 (+299618us) service_pool.cc:224] Handling call
0709 12:25:25.059707 (+   303us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:25.060071 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:25.060099 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:25.060699 (+   600us) log.cc:844] Serialized 52 byte log entry
0709 12:25:25.250476 (+189777us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:25.606911 (+356435us) log.cc:844] Serialized 14 byte log entry
0709 12:25:25.626782 (+ 19871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.626903 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.828761 (+201858us) raft_consensus.cc:1685] finished
0709 12:25:25.828790 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.828971 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25060,"prepare.queue_time_us":487131,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":73,"raft.queue_time_us":42145,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":5639,"thread_start_us":534159,"threads_started":2,"wal-append.queue_time_us":522803}
I20260709 12:25:25.960363  7636 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:25.960391  7663 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:25.966775  7672 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:26.105412  7710 consensus_queue.cc:1243] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:25:25.956959  7587 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:26.069912  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:26.104067  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:25.819696  7674 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:26.147285  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:26.162204  7589 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.034s	sys 0.027s
I20260709 12:25:26.150130  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 198) took 1110 ms. Trace:
W20260709 12:25:26.191465  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 87507.5us
I20260709 12:25:26.146793  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:26.070091  7483 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:26.192354  5944 rpcz_store.cc:276] 0709 12:25:25.040143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.047153 (+  7010us) service_pool.cc:224] Handling call
0709 12:25:26.149864 (+1102711us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b5400675f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:26.150053 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1470978944}
I20260709 12:25:26.162132  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 754) took 1255 ms. Trace:
W20260709 12:25:26.162854  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 388411us
W20260709 12:25:26.110520  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 322653us
W20260709 12:25:26.159914  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 466913us
I20260709 12:25:26.170084  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:25.940618  7661 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:26.215378  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:26.216177  5943 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:26.088023  7699 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:26.184883  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:26.109907  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:26.151326  7707 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.191375  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 762) took 1102 ms. Trace:
I20260709 12:25:26.193109  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:26.170331  7708 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:26.152503  7675 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:26.183019  7625 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:26.178294  7681 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.149s	user 0.000s	sys 0.037s
I20260709 12:25:26.146047  7718 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:26.181787  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:26.201660  6611 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:26.192152  7292 consensus_meta.cc:287] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.986s	user 0.008s	sys 0.008s
I20260709 12:25:26.171087  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:26.220191  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:26.192256  7589 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.035s	sys 0.028s
I20260709 12:25:26.221395  6024 raft_consensus.cc:2393] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 3: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:26.271054  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:26.263528  7675 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:26.271791  6024 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:26.272740  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:26.207675  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:26.273260  7675 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:26.224949  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:26.227767  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.215378  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:26.219964  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:26.261824  5915 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:26.225544  5871 rpcz_store.cc:276] 0709 12:25:25.088615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.251382 (+162767us) service_pool.cc:224] Handling call
0709 12:25:26.191084 (+939702us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b540069ee54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:26.191270 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":786461568}
I20260709 12:25:26.263967  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:26.266670  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 73673.2us
I20260709 12:25:26.409430  5987 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:26.269340  7714 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:26.273813  5944 raft_consensus.cc:2410] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:26.220983  5872 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:26.412406  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:26.273273  6024 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:26.230408  7708 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.221823  7710 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:25:26.254200  7707 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:26.216236  7715 raft_consensus.cc:2764] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 12:25:26.226044  7608 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.222597  7719 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.281812  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:26.283932  7587 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:26.289255  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.267340  7305 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:26.287513  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:26.294421  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:26.285858  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 755) took 1372 ms. Trace:
I20260709 12:25:26.216621  7672 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:26.414706  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:26.475797  6024 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
W20260709 12:25:26.414976  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 153280us
I20260709 12:25:26.478618  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:26.413316  5872 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:26.416401  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 245073us
I20260709 12:25:26.423260  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:26.268527  7681 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.228s	user 0.000s	sys 0.040s
I20260709 12:25:26.230690  7663 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.454470  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:26.268330  7718 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:26.055083  7709 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:26.480276  6023 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:25:26.412948  7711 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:26.460124  5941 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:26.287088  7292 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:26.484732  7305 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:26.488289  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.484527  5871 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:26.489810  6024 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:26.485394  7707 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:26.439824  5870 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:26.488476  6018 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:26.396304  5945 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:26.481158  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:26.447343  7675 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:26.494939  5944 raft_consensus.cc:2410] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:25:26.497263  6023 pending_rounds.cc:85] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 1
I20260709 12:25:26.446378  7708 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:26.526909  6023 pending_rounds.cc:107] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:25:26.529955  6016 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:26.529829  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:26.528458  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:26.531332  6023 raft_consensus.cc:2889] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:26.531785  6762 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:26.494275  7605 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:26.484411  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:26.511266  5915 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:26.542869  5874 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:26.448763  7666 consensus_queue.cc:237] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:26.544239  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:26.497485  7718 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:26.511587  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 309996us
I20260709 12:25:26.476312  7577 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:26.518077  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 761) took 1429 ms. Trace:
I20260709 12:25:26.466321  7724 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:26.523712  5872 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:26.538354  7605 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:26.570021  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:26.570272  7605 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.571376  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:26.572078  7672 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:26.572868  5872 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:26.573472  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 643) took 1172 ms. Trace:
I20260709 12:25:26.573792  5872 rpcz_store.cc:276] 0709 12:25:25.400839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.572363 (+1171524us) service_pool.cc:224] Handling call
0709 12:25:26.572655 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:26.573239 (+   584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.573421 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:26.575273  5872 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:26.526607  7728 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:26.575860  7728 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.575932  7709 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:26.580966  7517 consensus_meta.cc:287] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.746s	user 0.010s	sys 0.001s
I20260709 12:25:26.475787  7576 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:25:26.482430  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 59300.4us
I20260709 12:25:26.530135  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:26.540246  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:26.592269  6762 raft_consensus.cc:2410] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:25:26.550647  6611 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:26.465451  7608 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:26.552793  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:26.591847  6756 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:26.604014  5943 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 3: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:26.554000  5871 rpcz_store.cc:276] 0709 12:25:25.088275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.483675 (+1395400us) service_pool.cc:224] Handling call
0709 12:25:26.484103 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.517731 (+ 33628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.517994 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:26.548833  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:26.605337  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56629us
I20260709 12:25:26.559710  7662 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:26.612255  7662 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:26.562521  5870 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:26.616014  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 638) took 1216 ms. Trace:
I20260709 12:25:26.616459  5870 rpcz_store.cc:276] 0709 12:25:25.399609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.561407 (+1161798us) service_pool.cc:224] Handling call
0709 12:25:26.561945 (+   538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.615632 (+ 53687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.615938 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:26.514952  7711 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:26.467096  5865 rpcz_store.cc:276] 0709 12:25:24.913583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.223999 (+1310416us) service_pool.cc:224] Handling call
0709 12:25:26.284892 (+ 60893us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.285255 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:26.285299 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:26.285376 (+    77us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:26.285427 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.285519 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.285746 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:26.555121  7710 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:26.564805  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:26.618482  5870 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:26.565867  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:26.632656  7305 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:26.566737  7732 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:26.633690  7587 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:26.634168  7732 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:26.461179  7661 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:26.465368  7719 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:26.608897  6762 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:26.610067  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:26.620167  7710 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:26.496716  7483 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:26.621901  7479 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:26.623027  7292 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:26.623307  7724 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:26.625553  7669 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.682s	user 0.011s	sys 0.006s
I20260709 12:25:26.627912  7735 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:25:26.565689  7729 log.cc:927] Time spent T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:25:26.629763  7674 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:26.630400  7630 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:26.629844  7576 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:26.816208  7576 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:26.629763  7725 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.406s	user 0.002s	sys 0.000s
I20260709 12:25:26.634331  7587 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:26.597582  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 47086.9us
I20260709 12:25:26.637158  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:26.818641  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 181576us
I20260709 12:25:26.819706  7587 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:26.813513  7674 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.577301  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:26.822237  6775 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 7: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:26.822322  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:26.656086  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.823187  6024 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:26.639966  6766 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:26.659652  5871 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:26.663514  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:26.673658  7329 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:26.657160  5870 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:26.798777  7369 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.716076  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:26.620489  7711 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:26.753279  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 559) took 1208 ms. Trace:
I20260709 12:25:26.742340  5945 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:26.719877  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:26.747932  7450 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:26.615177  7662 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:26.675916  7720 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:26.723557  5867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 645) took 1161 ms (client timeout 1513 ms). Trace:
W20260709 12:25:26.752518  7683 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 1.848s	user 0.028s	sys 0.000s
I20260709 12:25:26.589665  7728 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:26.672375  7710 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.739694  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:26.633363  7608 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:26.757493  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 952) took 2014 ms. Trace:
I20260709 12:25:26.894157  5949 rpcz_store.cc:276] 0709 12:25:24.743684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.743866 (+   182us) service_pool.cc:224] Handling call
0709 12:25:24.758496 (+ 14630us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:25.070505 (+312009us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:25.070848 (+   343us) log.cc:844] Serialized 14 byte log entry
0709 12:25:25.542179 (+471331us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:25.542241 (+    62us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:25.542285 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.214759 (+672474us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b18003008e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb96522 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:25:26.757131 (+542372us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b54006761d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:26.757205 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.757402 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4292,"prepare.queue_time_us":694,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":132572,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":33894,"spinlock_wait_cycles":739940864,"thread_start_us":470021,"threads_started":2,"wal-append.queue_time_us":301673,"wal-append.run_cpu_time_us":2799,"wal-append.run_wall_time_us":1439737}
I20260709 12:25:26.562628  7723 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:25:26.802641  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.897418  7723 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:26.792733  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:26.781685  7661 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:26.745118  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 567) took 1042 ms. Trace:
I20260709 12:25:26.825615  7630 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:26.826610  5990 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:26.821453  7674 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:26.916225  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:26.917096  5870 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:26.918185  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:26.918834  5870 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:26.920501  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:26.824597  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:26.676299  7675 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.122s
I20260709 12:25:26.673485  7479 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.850603  6762 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:26.846570  7672 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.851332  6015 rpcz_store.cc:276] 0709 12:25:25.544704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.565279 (+ 20575us) service_pool.cc:224] Handling call
0709 12:25:25.586966 (+ 21687us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:25.773118 (+186152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:25.773166 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:25.809035 (+ 35869us) log.cc:844] Serialized 26 byte log entry
0709 12:25:26.223276 (+414241us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:26.223319 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.223398 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:26.751957 (+528559us) raft_consensus.cc:1685] finished
0709 12:25:26.752108 (+   151us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.753166 (+  1058us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":169053,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":157,"thread_start_us":575177,"threads_started":2,"wal-append.queue_time_us":925520}
I20260709 12:25:26.865486  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:26.870090  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 263552us
I20260709 12:25:26.921597  6024 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:26.928035  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:26.828925  7587 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:26.878014  5948 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:25:26.665735  7709 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.185s
W20260709 12:25:26.887859  5867 rpcz_store.cc:269] 0709 12:25:25.554810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.569686 (+ 14876us) service_pool.cc:224] Handling call
0709 12:25:26.674346 (+1104660us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b5400027454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:26.716008 (+ 41662us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54640512}
I20260709 12:25:26.898728  7723 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:26.939095  6024 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:26.909662  7605 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:26.910302  7734 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:25:26.847723  7735 raft_consensus.cc:697] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:26.869215  7369 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:26.911022  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 86468.3us
I20260709 12:25:26.950088  5990 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:26.914002  7329 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:26.914683  7732 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:26.915475  7414 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:26.797888  7577 raft_consensus.cc:697] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:26.929560  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:26.924664  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:26.925009  5871 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:26.887019  7450 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.899802  6019 rpcz_store.cc:276] 0709 12:25:25.703100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.762360 (+ 59260us) service_pool.cc:224] Handling call
0709 12:25:26.744824 (+982464us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54001612d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:26.745026 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1512214272}
I20260709 12:25:26.945391  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:26.889747  7707 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
W20260709 12:25:26.923070  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.936631  7608 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:25:26.952426  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:26.922230  7740 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.922987  7479 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:26.953251  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 290460us
I20260709 12:25:26.971689  7735 consensus_queue.cc:237] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:26.972708  7689 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 1.734s	user 0.014s	sys 0.020s
I20260709 12:25:26.973677  7690 raft_consensus.cc:3055] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:26.974200  7690 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:26.975139  7690 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:26.976203  7690 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.981818  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:26.983340  5870 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:45236
I20260709 12:25:26.983927  5870 raft_consensus.cc:606] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:26.984283  5870 raft_consensus.cc:614] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:26.966238  7450 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:26.986541  7743 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:26.986986  7743 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.961457  7723 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:26.987601  7450 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:26.991904  5946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 200) took 1932 ms (client timeout 1807 ms). Trace:
W20260709 12:25:26.992542  5946 rpcz_store.cc:269] 0709 12:25:25.059051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.075189 (+ 16138us) service_pool.cc:224] Handling call
0709 12:25:26.991636 (+1916447us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b54008b06d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:26.991804 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1651543424}
I20260709 12:25:27.000501  5867 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:26.923723  7672 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:26.939857  7587 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:27.010587  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:27.011477  5949 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.965441  7709 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:27.014479  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:27.015614  5946 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:26.987290  7698 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:26.987310  7743 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:27.011040  7597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.301s	user 0.020s	sys 0.009s
W20260709 12:25:27.030004  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 223108us
I20260709 12:25:27.030380  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:27.009480  7486 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.039551  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:27.004060  7592 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:27.009048  7674 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:27.042642  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:27.019778  7709 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.046180  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:27.046986  6762 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:25:27.037760  7699 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.796s	user 0.007s	sys 0.007s
I20260709 12:25:27.045500  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:27.056151  5867 raft_consensus.cc:2376] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5. Re-sending same reply.
I20260709 12:25:27.058327  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:27.059099  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:27.059417  5949 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:27.059792  6793 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:27.060981  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:27.063025  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:26.995554  7740 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:27.064728  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:27.043651  7597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.328s	user 0.021s	sys 0.009s
I20260709 12:25:27.075059  6766 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:27.076320  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:27.076994  5867 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:27.049374  7709 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:27.106474  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 177058us
I20260709 12:25:27.045118  7719 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:27.057715  7738 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.109714  5915 leader_election.cc:400] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5.
I20260709 12:25:27.063982  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:27.113417  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 163410us
I20260709 12:25:27.111994  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:27.112938  5915 leader_election.cc:403] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:27.073572  7675 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:27.117916  5915 leader_election.cc:400] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5.
I20260709 12:25:27.049500  7486 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:27.123819  5915 leader_election.cc:403] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:25:27.064003  7728 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:27.071306  7589 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:27.050032  7592 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:27.078603  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:27.119262  5840 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:27.149559  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:27.141104  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:27.142571  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:27.151880  5867 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:27.152070  5949 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:27.114871  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:27.041491  7577 consensus_queue.cc:237] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:27.058566  7699 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:27.171077  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:27.170045  7576 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.175913  7576 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:27.176817  7701 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 1.464s	user 0.010s	sys 0.006s
I20260709 12:25:27.177892  7329 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:27.178151  7576 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:27.127249  7698 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5.
W20260709 12:25:27.184245  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 121551us
I20260709 12:25:27.185576  6609 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:27.185681  6762 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:27.078401  7718 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:27.162092  7754 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.189093  5941 raft_consensus.cc:2410] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:27.145038  7675 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:27.201719  5942 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:27.189527  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 466) took 2231 ms. Trace:
W20260709 12:25:27.202996  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 83215.5us
I20260709 12:25:27.202540  6773 rpcz_store.cc:276] 0709 12:25:24.954694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.102593 (+147899us) service_pool.cc:224] Handling call
0709 12:25:25.160958 (+ 58365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:25.166467 (+  5509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:25.166500 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:25.167624 (+  1124us) log.cc:844] Serialized 26 byte log entry
0709 12:25:25.246742 (+ 79118us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b480052c158. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d7bacaa1 00007f39d7ba3b37 00007f39d7ba39ca 00007f39d7c9b26f 00007f39d7c9af37 00007f39d7caf55c 00007f39d7caf508 00007f39d7caf488 00007f39d7caf443 00007f39d7cae6ec 00007f39d6bd8a96
0709 12:25:25.585356 (+338614us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:25.585427 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.585543 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:27.176087 (+1590544us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:27.185858 (+  9771us) raft_consensus.cc:1685] finished
0709 12:25:27.185880 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.186037 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36510,"raft.queue_time_us":5815,"raft.run_cpu_time_us":2108,"raft.run_wall_time_us":79380,"spinlock_wait_cycles":87986688,"thread_start_us":370723,"threads_started":1,"wal-append.queue_time_us":480936}
I20260709 12:25:27.192973  5867 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:27.156762  7597 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:27.184681  7605 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:27.170228  7663 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:27.207126  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:27.204633  6611 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:27.157049  7669 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:27.221589  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 67043.2us
I20260709 12:25:27.184432  7758 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:25:27.206552  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:27.232507  6773 raft_consensus.cc:2393] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:27.199098  6766 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:27.226858  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:27.146694  7750 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5.
I20260709 12:25:27.236855  5867 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 4: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:27.190857  6775 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:27.178891  7666 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.159412  7589 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:27.234557  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:27.179064  7671 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:25:27.119859  7738 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:27.251479  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:27.183903  7752 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:27.219837  7663 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.258268  7671 raft_consensus.cc:3055] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:27.258461  7663 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:27.258715  7671 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:27.259683  7671 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:27.260927  7671 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:27.243422  7436 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:27.244050  7329 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:25:27.262285  7748 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:25:27.208359  7754 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:27.263200  7329 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:27.231323  7625 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.856s	user 0.011s	sys 0.004s
I20260709 12:25:27.231745  6757 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:25:27.220580  7757 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:27.276593  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 102502us
I20260709 12:25:27.277724  6017 raft_consensus.cc:2410] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:25:27.273315  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 158550us
I20260709 12:25:27.274049  6766 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:27.283299  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:27.284330  6775 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:27.284272  5990 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:27.254453  7450 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:27.290781  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 84887.9us
I20260709 12:25:27.294984  7745 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:27.295481  7762 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.295746  7745 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:27.296000  7762 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:27.285722  5990 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:27.298400  7745 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:27.298230  7762 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:27.299257  7762 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:27.300267  7762 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:27.265858  7752 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.223501  7751 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 12:25:27.230264  7414 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:25:27.276340  7669 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:27.308635  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:27.314002  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 89267.3us
I20260709 12:25:27.311152  7669 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:27.316780  7669 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:27.317076  7592 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:27.311517  7708 consensus_meta.cc:287] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.764s	user 0.014s	sys 0.001s
I20260709 12:25:27.318293  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:27.320025  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:27.320281  7698 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.320528  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:27.318122  7708 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:27.317423  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:27.280220  7757 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:27.290007  7450 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.255679  7666 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:27.331444  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.323271  5949 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
W20260709 12:25:27.353312  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.320427  7749 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:25:27.326709  7587 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:27.358733  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 154254us
I20260709 12:25:27.317126  7718 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:27.320788  7698 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:27.307112  7752 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:27.351943  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:27.390673  6611 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:27.358295  7765 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:25:27.361394  7450 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:27.401615  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 201) took 1207 ms. Trace:
I20260709 12:25:27.419957  5872 rpcz_store.cc:276] 0709 12:25:26.194150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.577196 (+383046us) service_pool.cc:224] Handling call
0709 12:25:26.577442 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.737775 (+160333us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:25:26.737827 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:26.737895 (+    68us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:26.737940 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.401339 (+663399us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54002f9ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:27.401397 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.401543 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":541,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":21314,"spinlock_wait_cycles":367672704}
W20260709 12:25:27.409672  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 57909.7us
I20260709 12:25:27.371124  5868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60934281a651439a87cd8c11702f67d2"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
mode: GRACEFUL
new_leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
 from {username='slave'} at 127.0.0.1:45236
I20260709 12:25:27.424688  5868 raft_consensus.cc:606] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Received request to transfer leadership to TS 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:27.423552  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:27.437588  5868 raft_consensus.cc:614] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:27.439581  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:27.357267  7740 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:27.401388  7589 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:27.431695  7772 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:27.402078  7766 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:27.431854  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:27.366564  7759 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:27.435487  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 149898us
I20260709 12:25:27.326002  7675 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:27.358304  7757 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:27.403257  7666 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:27.431546  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:27.419641  7483 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:27.507553  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 66049.5us
I20260709 12:25:27.508230  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 722) took 2669 ms. Trace:
I20260709 12:25:27.523836  6022 rpcz_store.cc:276] 0709 12:25:24.772800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.054544 (+281744us) service_pool.cc:224] Handling call
0709 12:25:25.054852 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:25.893186 (+838334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.442433 (+1549247us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b54001072d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:27.442625 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1738461056}
I20260709 12:25:27.446918  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:27.517762  5990 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:27.520015  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:27.522637  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:27.526119  6024 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:27.526352  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:27.511592  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.526963  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:27.403875  7605 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.105s
I20260709 12:25:27.525539  5868 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
W20260709 12:25:27.528074  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 81252.8us
I20260709 12:25:27.528779  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:27.527561  6024 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:27.527544  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:27.530244  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 756) took 2610 ms. Trace:
I20260709 12:25:27.530368  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:27.530678  5866 rpcz_store.cc:276] 0709 12:25:24.919570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.433158 (+2513588us) service_pool.cc:224] Handling call
0709 12:25:27.434901 (+  1743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.507717 (+ 72816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.511055 (+  3338us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:27.514913 (+  3858us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:27.523698 (+  8785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.529995 (+  6297us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.530137 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:27.531338  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:27.531632  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:27.532060  6022 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:27.532392  6775 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:27.532938  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:27.532092  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:27.534173  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:27.534502  5866 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:27.534816  7747 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 12:25:27.537261  6022 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:27.507522  7588 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:27.507392  7608 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.513221  7414 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:27.544505  7738 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:27.554634  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:27.555495  5868 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:27.556322  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:27.556833  5868 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:27.557185  6793 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:27.541958  7499 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:27.545724  7769 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:27.550107  7775 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:26.198395  5869 rpcz_store.cc:276] 0709 12:25:24.906996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.501992 (+594996us) service_pool.cc:224] Handling call
0709 12:25:25.502490 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:25.502799 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:25.789737 (+286938us) log.cc:844] Serialized 14 byte log entry
0709 12:25:26.161663 (+371926us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:26.161734 (+    71us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:26.161785 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.161872 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.162044 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7567,"prepare.queue_time_us":235225,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":25796,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":7967,"thread_start_us":590953,"threads_started":2,"wal-append.queue_time_us":424620}
I20260709 12:25:27.547974  7414 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:27.559197  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:27.523608  7483 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:27.546828  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:27.519488  7759 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.568639  7698 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:27.546058  7766 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:27.563024  7752 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:27.565732  7479 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.548s	user 0.013s	sys 0.000s
I20260709 12:25:27.567361  7588 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:27.575297  7414 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:27.579522  7605 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:27.577658  7759 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:27.580319  7708 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:27.604784  6773 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 4: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:27.545957  7772 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:27.581053  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:27.582334  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:27.575686  6024 raft_consensus.cc:2393] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:25:27.594696  7723 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.607s	user 0.008s	sys 0.004s
I20260709 12:25:27.587296  7479 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:27.572947  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:27.576934  7483 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:27.595976  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 67259.7us
I20260709 12:25:27.596274  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:27.597465  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.615998  6766 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:27.564913  7608 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:27.599027  7690 consensus_meta.cc:287] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.622s	user 0.012s	sys 0.002s
I20260709 12:25:27.600404  7588 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:27.605800  6775 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:27.610397  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:27.580868  7710 consensus_meta.cc:287] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.003s
I20260709 12:25:27.621981  6024 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:27.611145  5869 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:27.574671  7663 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:27.587623  7762 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:25:27.614269  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 84158.6us
I20260709 12:25:27.638857  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 591) took 1116 ms. Trace:
I20260709 12:25:27.588075  7775 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:27.617303  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:27.619699  7709 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:27.634891  7720 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:27.601560  7766 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:27.637238  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:27.618178  7671 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:27.614953  5868 raft_consensus.cc:2410] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:27.646989  5990 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:27.627308  7690 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.650532  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:27.625007  7710 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:27.635042  6757 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:27.656436  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:27.621554  7499 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.026s
W20260709 12:25:27.649039  7744 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.644s	user 0.002s	sys 0.000s
I20260709 12:25:27.663928  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:27.652822  7762 raft_consensus.cc:697] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:27.666203  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:27.658075  7720 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:27.672067  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:27.649049  7774 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:25:27.671329  5990 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:27.670014  5868 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:27.663184  5870 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:27.678860  5947 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 4: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:27.681173  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:27.665024  7597 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.689157  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:27.690917  6024 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:27.692971  6024 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:27.681967  5868 raft_consensus.cc:2376] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3. Re-sending same reply.
I20260709 12:25:27.649941  6785 rpcz_store.cc:276] 0709 12:25:26.522711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.538699 (+ 15988us) service_pool.cc:224] Handling call
0709 12:25:26.588846 (+ 50147us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.631403 (+ 42557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:26.631445 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:26.631505 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:26.631547 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.638564 (+1007017us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b5400005d54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:27.638625 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.638790 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97227648}
I20260709 12:25:27.674595  7709 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:27.685489  5913 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:27.697098  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:27.703006  5869 raft_consensus.cc:2393] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 5: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:27.703644  5913 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:27.675590  7608 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:27.673121  7479 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:27.705574  5913 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:27.662771  7690 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:27.694455  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:27.661450  7671 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:27.695390  7597 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:27.711021  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:27.693828  7758 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:27.712889  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 206) took 1145 ms. Trace:
I20260709 12:25:27.708168  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:27.661592  7772 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:27.716199  7723 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:27.686163  7775 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:27.707204  7499 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:27.705346  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:27.687253  7669 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:27.780136  7777 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:25:27.718961  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:27.797430  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:27.727341  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:27.734663  7778 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:27.800097  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:27.800805  5869 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:27.802886  5868 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:45236
I20260709 12:25:27.803848  5868 raft_consensus.cc:606] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:27.719614  5870 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:27.811833  7769 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:27.813043  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:27.776765  7781 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:27.781100  7723 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:27.774971  7486 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:27.745553  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.673612  7773 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.110s
I20260709 12:25:27.790153  5866 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:27.713517  7709 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:27.811343  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:27.724440  5865 rpcz_store.cc:276] 0709 12:25:26.561507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.630054 (+ 68547us) service_pool.cc:224] Handling call
0709 12:25:26.630607 (+   553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:26.631119 (+   512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:26.631169 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:26.631843 (+   674us) log.cc:844] Serialized 26 byte log entry
0709 12:25:26.847573 (+215730us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:26.867212 (+ 19639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.878726 (+ 11514us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:27.562979 (+684253us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:27.678717 (+115738us) raft_consensus.cc:1685] finished
0709 12:25:27.678751 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.705462 (+ 26711us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1510,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":131,"spinlock_wait_cycles":28393856,"thread_start_us":198055,"threads_started":1,"wal-append.queue_time_us":287477}
I20260709 12:25:27.817479  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:27.794798  7499 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:27.825843  7781 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.826272  7671 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:27.833680  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 577) took 1605 ms. Trace:
I20260709 12:25:27.824283  5913 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:27.726050  7762 consensus_queue.cc:237] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:27.814378  7666 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.818712  7710 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:27.841213  5866 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:27.836755  7499 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:27.843689  5865 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:27.844970  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 285) took 1015 ms. Trace:
I20260709 12:25:27.866398  6022 rpcz_store.cc:276] 0709 12:25:26.823176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.538151 (+714975us) service_pool.cc:224] Handling call
0709 12:25:27.538418 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.837356 (+298938us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.837963 (+   607us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:27.838041 (+    78us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:27.838078 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.838146 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.838287 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6548,"raft.queue_time_us":50662,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":5318,"thread_start_us":288256,"threads_started":1}
I20260709 12:25:27.820583  7663 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:27.868021  7438 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.876062  7438 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:27.838240  7781 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:27.883311  5866 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:27.886662  7669 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:27.897843  7710 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:25:27.836587  7743 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.794s	user 0.004s	sys 0.011s
I20260709 12:25:27.871426  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:27.838011  6016 rpcz_store.cc:276] 0709 12:25:26.222326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.281478 (+ 59152us) service_pool.cc:224] Handling call
0709 12:25:27.827382 (+1545904us) spinlock_profiling.cc:243] Waited 73.7 ms on lock 0x7b5400171bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:27.827476 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169485184}
I20260709 12:25:27.868906  7758 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:27.899359  7773 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:27.899695  7723 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:27.880851  7414 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:27.896840  7597 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:27.878999  7562 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:27.871860  5870 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:45236
I20260709 12:25:27.905495  7479 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:27.914916  7562 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:27.915304  7479 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:27.917596  7479 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:27.918133  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 748) took 3117 ms. Trace:
I20260709 12:25:27.922581  5869 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:27.910177  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:27.890270  7690 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:27.865984  7666 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:27.906941  7743 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:27.920552  7414 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:27.900306  7438 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:27.936419  5870 raft_consensus.cc:606] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:27.937112  5870 raft_consensus.cc:614] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:27.938912  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:27.906762  7758 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.932425  5869 pending_rounds.cc:85] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 1
I20260709 12:25:27.921564  5874 rpcz_store.cc:276] 0709 12:25:24.800814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.339696 (+538882us) service_pool.cc:224] Handling call
0709 12:25:25.388962 (+ 49266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.543504 (+1154542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.917856 (+1374352us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400186d54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:27.918041 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2653081344}
I20260709 12:25:27.901227  7752 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:27.939042  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:27.824365  5840 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:27.945031  7671 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:27.946772  7671 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.946997  5874 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:27.947154  7671 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:27.947736  7672 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:27.944480  5869 pending_rounds.cc:107] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:25:27.950142  5944 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:27.951094  5869 raft_consensus.cc:2889] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:27.925179  7562 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:27.943735  7758 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:27.944612  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 120350us
I20260709 12:25:27.889989  7757 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:27.919034  7450 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:27.949913  7743 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:27.953192  5866 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: [2.2-3.3]
I20260709 12:25:27.963236  7438 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.918819  7759 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:27.960485  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:27.950004  7720 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:27.969762  7438 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:27.950655  7483 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:27.954208  7774 consensus_queue.cc:1059] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:25:27.981835  7607 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:27.983227  7414 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.009758  6024 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:28.010517  6021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 274) took 2636 ms (client timeout 1803 ms). Trace:
I20260709 12:25:28.010852  6022 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:25:28.012946  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.011040  6021 rpcz_store.cc:269] 0709 12:25:25.357221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.361827 (+  4606us) service_pool.cc:224] Handling call
0709 12:25:27.993405 (+2631578us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b5400160b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:27.993671 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":978698112}
I20260709 12:25:28.024359  6021 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:27.989079  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:28.026239  6021 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:28.050391  6021 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:27.992480  7607 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:28.044438  7752 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.025681  7588 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:28.014189  7414 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:28.077441  7588 raft_consensus.cc:697] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:28.083985  7588 consensus_queue.cc:237] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:28.090611  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:28.091384  6021 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:28.094306  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 113351us
W20260709 12:25:28.101132  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.071911  7499 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:28.102638  7798 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.074476  7752 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:28.102988  6021 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
W20260709 12:25:28.106990  7742 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 1.195s	user 0.003s	sys 0.000s
I20260709 12:25:28.114406  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:28.115778  6793 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:28.117177  6021 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:28.113512  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:28.127943  6021 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:28.107033  7759 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:28.132251  7720 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:28.118711  7450 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:28.141762  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:28.119014  7757 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:28.088692  7607 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:28.137732  7592 consensus_queue.cc:1059] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:25:28.160069  7675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.006s	sys 0.006s
I20260709 12:25:28.155617  7483 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:28.162357  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:28.168111  7630 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:25:28.168545  7789 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:25:28.169368  7784 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:25:28.160866  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:28.170172  7786 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.082s	user 0.001s	sys 0.001s
I20260709 12:25:28.156258  7438 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:28.115237  7798 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:28.155124  7666 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:28.182248  7675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.007s	sys 0.006s
W20260709 12:25:28.193743  7794 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:25:28.194259  7414 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:28.195763  7499 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:28.200161  7671 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:28.183827  7630 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:28.201988  7630 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:28.203711  7630 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:28.211974  7671 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:28.185848  7607 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:28.218044  7798 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:28.218761  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.222647  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:28.223315  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:28.224334  6016 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 7: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:28.224179  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:28.227077  6017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 570) took 2523 ms (client timeout 2060 ms). Trace:
I20260709 12:25:28.214360  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:28.230307  6768 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 1.
I20260709 12:25:28.230932  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 495) took 1160 ms. Trace:
I20260709 12:25:28.233561  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:28.227658  6017 rpcz_store.cc:269] 0709 12:25:25.704094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.147016 (+442922us) service_pool.cc:224] Handling call
0709 12:25:28.226796 (+2079780us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54005659d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:28.226984 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2507825024}
I20260709 12:25:28.231298  6768 rpcz_store.cc:276] 0709 12:25:27.070517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.070660 (+   143us) service_pool.cc:224] Handling call
0709 12:25:28.214152 (+1143492us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b580001f898. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:28.230858 (+ 16706us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3685632}
I20260709 12:25:28.245699  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:28.246076  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:28.246727  6793 raft_consensus.cc:2410] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:28.249114  6017 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:28.260933  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:28.261839  6793 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:28.242823  7414 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:25:28.255117  7792 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:25:28.263535  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:28.267766  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:28.268368  5872 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:25:28.209926  7781 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:28.264853  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:28.276618  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:28.278683  6768 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:28.282887  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 493) took 1226 ms. Trace:
I20260709 12:25:28.283473  6762 rpcz_store.cc:276] 0709 12:25:27.056475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.061691 (+  5216us) service_pool.cc:224] Handling call
0709 12:25:27.061999 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.186254 (+124255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.282568 (+1096314us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b540015efd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:28.282809 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232308096}
I20260709 12:25:28.285713  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:28.286005  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:28.286720  6016 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:28.286954  6021 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:28.288053  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:28.288727  6021 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
W20260709 12:25:28.289155  7746 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 1.183s	user 0.004s	sys 0.000s
I20260709 12:25:28.291222  6021 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:28.293712  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:28.299098  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 592) took 1776 ms. Trace:
I20260709 12:25:28.299649  6778 rpcz_store.cc:276] 0709 12:25:26.523135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.526335 (+  3200us) service_pool.cc:224] Handling call
0709 12:25:26.602895 (+ 76560us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:26.628760 (+ 25865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:26.628801 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:26.629405 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 12:25:26.959048 (+329643us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:26.961961 (+  2913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.967915 (+  5954us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:28.288527 (+1320612us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:28.291116 (+  2589us) raft_consensus.cc:1685] finished
0709 12:25:28.295373 (+  4257us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:28.299018 (+  3645us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28199,"raft.queue_time_us":95416,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":12565,"spinlock_wait_cycles":5782400,"thread_start_us":279568,"threads_started":1,"wal-append.queue_time_us":414852}
I20260709 12:25:28.283492  6785 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:25:28.304201  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 289) took 1166 ms. Trace:
I20260709 12:25:28.304718  6024 rpcz_store.cc:276] 0709 12:25:27.138251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.694262 (+556011us) service_pool.cc:224] Handling call
0709 12:25:27.694549 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.010510 (+315961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.303946 (+293436us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b540065b8d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:28.304137 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724038400}
I20260709 12:25:28.309103  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:28.308336  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:28.318502  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:28.318758  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:28.326295  5915 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:28.326627  6016 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:28.274972  7414 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:28.328629  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:28.328954  5870 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:28.351400  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:28.352344  5870 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:28.341162  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:28.324469  7743 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:28.353991  6785 raft_consensus.cc:2393] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:28.354102  7743 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:28.356747  7743 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:28.318243  7669 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:28.363298  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:28.364902  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 652) took 1534 ms. Trace:
W20260709 12:25:28.376428  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.307471  7562 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:25:28.379467  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:28.328459  7708 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:28.322949  7754 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:28.365374  5871 rpcz_store.cc:276] 0709 12:25:26.830012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.958088 (+128076us) service_pool.cc:224] Handling call
0709 12:25:28.363006 (+1404918us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b58002e2f98. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:28.364799 (+  1793us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66346880}
I20260709 12:25:28.357482  7743 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.378124  7329 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:28.384111  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:28.384508  7708 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:28.349355  7758 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:28.401432  5945 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:28.389166  7438 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:28.386338  7723 consensus_queue.cc:1059] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:25:28.407750  7781 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:28.411813  7499 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:28.422569  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 697) took 1411 ms. Trace:
I20260709 12:25:28.422873  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:28.397679  7743 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:28.423086  5947 rpcz_store.cc:276] 0709 12:25:27.010656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.010783 (+   127us) service_pool.cc:224] Handling call
0709 12:25:28.422294 (+1411511us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b54008b01d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:28.422455 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1471149312}
I20260709 12:25:28.417264  7708 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:28.430034  6018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 727) took 2639 ms (client timeout 1861 ms). Trace:
I20260709 12:25:28.433941  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:28.433727  6018 rpcz_store.cc:269] 0709 12:25:25.790859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.069561 (+278702us) service_pool.cc:224] Handling call
0709 12:25:28.429809 (+2360248us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b54003647d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:28.429961 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":695439616}
I20260709 12:25:28.458520  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 202) took 2395 ms. Trace:
I20260709 12:25:28.459107  5942 rpcz_store.cc:276] 0709 12:25:26.035524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.035720 (+   196us) service_pool.cc:224] Handling call
0709 12:25:26.036200 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.202447 (+1166247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.422932 (+1220485us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400674654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:28.430874 (+  7942us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2679700352}
I20260709 12:25:28.463045  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:28.467375  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:28.468823  5943 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:28.422508  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 696) took 1404 ms. Trace:
W20260709 12:25:28.480181  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.481469  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 102120us
I20260709 12:25:28.417578  7438 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:28.485540  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:28.466876  5940 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:28.466797  7577 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:28.434659  7486 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:25:28.492956  5942 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:28.503861  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:28.505282  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 46994.5us
I20260709 12:25:28.517302  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:28.517957  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:28.518344  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:28.516839  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:28.519672  6018 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:28.515707  5943 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:28.488193  7414 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:28.500823  7577 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.529997  7577 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:28.518787  5946 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
W20260709 12:25:28.526037  7809 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:25:28.514595  7772 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.538113  7811 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:28.538499  7758 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:28.539901  7743 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:28.479624  5945 rpcz_store.cc:276] 0709 12:25:27.010305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.022291 (+ 11986us) service_pool.cc:224] Handling call
0709 12:25:28.379630 (+1357339us) spinlock_profiling.cc:243] Waited 6.48 ms on lock 0x7b5800030398. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:28.414248 (+ 34618us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14895488}
I20260709 12:25:28.543692  7772 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:28.550843  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:28.551717  7811 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:28.563107  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.498489  7669 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:28.648737  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:28.655725  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:28.658839  5943 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:28.663163  5873 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:28.663954  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 758) took 3723 ms. Trace:
I20260709 12:25:28.664264  5873 rpcz_store.cc:276] 0709 12:25:24.940718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.662581 (+3721863us) service_pool.cc:224] Handling call
0709 12:25:28.662966 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.663682 (+   716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.663899 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:28.667368  7811 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:28.668357  7305 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:28.669085  7305 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:28.669086  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 690) took 2464 ms. Trace:
I20260709 12:25:28.669610  7772 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:28.670382  7305 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:28.672021  7305 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:28.673894  7825 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:28.674500  7825 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:28.674901  7825 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:28.675238  7830 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:28.678087  7669 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:28.669765  5948 rpcz_store.cc:276] 0709 12:25:26.204390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.467910 (+263520us) service_pool.cc:224] Handling call
0709 12:25:26.468563 (+   653us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:26.934304 (+465741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.668800 (+1734496us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b5400633154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:28.669021 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":940431616}
I20260709 12:25:28.679746  6611 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:28.684595  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:28.675315  7793 raft_consensus.cc:3055] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:28.689977  7793 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:28.691035  7793 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:28.692281  7793 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:28.691370  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 290) took 1541 ms. Trace:
W20260709 12:25:28.693984  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 199190us
I20260709 12:25:28.694360  6022 rpcz_store.cc:276] 0709 12:25:27.149621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.906806 (+757185us) service_pool.cc:224] Handling call
0709 12:25:27.993616 (+ 86810us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.011243 (+ 17627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.691041 (+679798us) spinlock_profiling.cc:243] Waited 81.4 ms on lock 0x7b5400160b50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:28.691276 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":187212672}
W20260709 12:25:28.697736  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 52583.3us
I20260709 12:25:28.696524  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:28.692688  7820 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:28.698877  6756 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:28.700347  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.697901  6611 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:28.701718  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:28.703769  7479 raft_consensus.cc:3055] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:28.705063  7728 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:28.707851  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:28.706394  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:28.712750  5948 raft_consensus.cc:2393] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:28.714386  6611 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:28.686518  7743 consensus_queue.cc:1059] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:25:28.719333  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:28.719673  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:28.720331  5948 raft_consensus.cc:2410] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:28.718381  7818 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:28.720436  5870 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:28.711651  7479 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:28.714290  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:28.716104  7608 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:28.727344  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:28.729549  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:28.717250  7820 raft_consensus.cc:697] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:28.731607  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:28.724191  7486 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:28.736589  7562 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:28.736922  7818 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:28.739552  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:28.726861  7438 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:28.731084  7750 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:25:28.782174  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.782325  5915 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:28.788529  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:28.803988  5915 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:28.735597  6785 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:28.847720  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:28.850667  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 126675us
I20260709 12:25:28.855301  5838 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:28.790369  7821 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.791335  7728 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:28.860584  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.852833  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:28.864351  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 140708us
I20260709 12:25:28.865190  6762 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:25:28.799012  7816 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:28.870519  6611 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:28.871534  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:28.856897  7819 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:28.872370  7577 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:28.886991  5838 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:28.889057  5870 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 12:25:28.890275  5871 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:28.871618  7810 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:28.857254  7750 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:28.859426  7329 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:28.799353  7608 consensus_queue.cc:1059] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 12:25:28.901872  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:28.866475  7766 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.912693  6611 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:28.919013  5871 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:25:28.871971  7728 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:28.927834  7292 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:28.932085  7605 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.938257  7605 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:28.938720  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 779) took 1254 ms. Trace:
I20260709 12:25:28.939179  5866 rpcz_store.cc:276] 0709 12:25:27.684541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.884237 (+199696us) service_pool.cc:224] Handling call
0709 12:25:27.925004 (+ 40767us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:27.973987 (+ 48983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.938454 (+964467us) spinlock_profiling.cc:243] Waited 38.6 ms on lock 0x7b54004f60d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:28.938649 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88791424}
I20260709 12:25:28.940641  7605 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:28.941918  7833 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:25:28.941747  7766 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:28.943425  7831 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 12:25:28.893849  7562 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:28.949755  5871 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:28.955300  5872 pending_rounds.cc:85] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 1
I20260709 12:25:28.959628  5872 pending_rounds.cc:107] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:25:28.960116  5872 raft_consensus.cc:2889] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:28.899003  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:28.968058  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:28.908864  7810 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.869834  7811 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 12:25:28.967006  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:28.967566  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.850062  7479 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:28.913547  7816 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:28.869491  7821 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:28.913988  7819 raft_consensus.cc:697] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:25:29.008153  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 60378.1us
I20260709 12:25:28.902479  7772 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 12:25:29.005792  5942 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
W20260709 12:25:29.020407  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 107804us
I20260709 12:25:28.926517  7329 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:28.903174  7708 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:28.951372  7728 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.028505  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:29.030771  5947 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:25:29.035776  5947 pending_rounds.cc:85] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 2
I20260709 12:25:29.036191  5947 pending_rounds.cc:107] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:25:29.036424  5947 raft_consensus.cc:2889] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:28.957942  7562 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.934190  7835 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:25:29.101481  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.024331  7486 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:29.040916  5941 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:25:29.041877  6756 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:29.043780  7834 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:29.047468  7369 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:29.054593  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 966) took 2485 ms. Trace:
I20260709 12:25:28.870277  7820 consensus_queue.cc:237] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:29.058225  7832 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.201s	user 0.000s	sys 0.000s
I20260709 12:25:29.031908  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:29.060600  7821 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:29.138482  5866 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.142009  7499 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:29.108088  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:29.143927  5945 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 7: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:29.005555  7758 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.146323  7758 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:29.062002  7711 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 12:25:29.148543  7711 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:29.149055  7711 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:29.151679  7708 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:29.010102  7816 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.155099  7816 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.039165  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:29.157920  5942 raft_consensus.cc:2410] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:29.159137  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:29.160125  5942 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 7: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:29.102147  5870 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:28.937093  7836 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.131s
I20260709 12:25:29.005430  7810 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:29.162109  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:29.162956  5870 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:29.164323  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:29.117208  7818 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 12:25:29.171114  6609 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:29.027076  7479 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.028405  7766 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:29.110327  5941 pending_rounds.cc:85] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 2
I20260709 12:25:29.024856  7329 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.177942  5941 pending_rounds.cc:107] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:25:29.178350  5941 raft_consensus.cc:2889] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:29.180826  7834 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:29.181450  7834 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.181799  7699 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:29.181924  7834 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:29.188158  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:29.189397  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 965) took 2642 ms. Trace:
I20260709 12:25:29.187664  7329 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:29.190536  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:29.191048  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:29.191990  6793 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
W20260709 12:25:29.193454  6020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 276) took 3835 ms (client timeout 2096 ms). Trace:
W20260709 12:25:29.194123  6020 rpcz_store.cc:269] 0709 12:25:25.358037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.358159 (+   122us) service_pool.cc:224] Handling call
0709 12:25:29.193172 (+3835013us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b5400661cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:29.193358 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3679350528}
I20260709 12:25:29.194597  7711 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:29.197184  7711 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.198410  7329 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:29.200435  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.113794  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:29.115243  7835 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:29.203539  7835 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:29.204130  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:29.205086  5865 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:29.062675  6757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 470) took 1985 ms (client timeout 2512 ms). Trace:
W20260709 12:25:29.206102  6757 rpcz_store.cc:269] 0709 12:25:27.051749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.051897 (+   148us) service_pool.cc:224] Handling call
0709 12:25:29.019180 (+1967283us) spinlock_profiling.cc:243] Waited 72.6 ms on lock 0x7b540081b5d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:29.034273 (+ 15093us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166881664}
I20260709 12:25:29.207515  7438 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:29.208242  7438 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:29.191182  6756 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.106920  7723 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:25:29.218079  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:29.218287  7329 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:29.047473  7819 consensus_queue.cc:237] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:29.125001  7486 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:29.218663  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:29.219822  7486 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:29.221061  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:29.125839  7781 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.096s
I20260709 12:25:29.222820  7781 raft_consensus.cc:3055] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:29.223279  7781 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:29.224350  7781 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:29.171238  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 783) took 1279 ms. Trace:
I20260709 12:25:29.229879  5869 rpcz_store.cc:276] 0709 12:25:27.891863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.173789 (+281926us) service_pool.cc:224] Handling call
0709 12:25:28.174187 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:28.306542 (+132355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:28.306578 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:28.319705 (+ 13127us) log.cc:844] Serialized 26 byte log entry
0709 12:25:28.696111 (+376406us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:28.696153 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.696230 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:29.170939 (+474709us) raft_consensus.cc:1685] finished
0709 12:25:29.170970 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:29.171155 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6492,"raft.queue_time_us":9523,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":4083,"thread_start_us":439169,"threads_started":2,"wal-append.queue_time_us":385362}
W20260709 12:25:29.234743  7793 consensus_meta.cc:287] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.542s	user 0.013s	sys 0.000s
I20260709 12:25:29.202185  7752 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:29.109588  7562 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.138616  7577 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:29.121137  5940 rpcz_store.cc:276] 0709 12:25:26.568859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.560447 (+991588us) service_pool.cc:224] Handling call
0709 12:25:28.425025 (+864578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.502695 (+ 77670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.054228 (+551533us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b54008b01d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:29.054489 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1982924544}
I20260709 12:25:29.239235  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:29.240216  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:29.241148  5945 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
W20260709 12:25:29.246482  7825 consensus_meta.cc:287] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.003s
I20260709 12:25:29.247164  7825 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:29.218837  7329 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:29.223294  7438 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:29.210407  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:29.280225  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.282155  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:29.283350  6015 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:29.190042  5944 rpcz_store.cc:276] 0709 12:25:26.546753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.624875 (+1078122us) service_pool.cc:224] Handling call
0709 12:25:27.638026 (+ 13151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.950642 (+312616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.189063 (+1238421us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b54003093d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:29.189309 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":717258112}
I20260709 12:25:29.248580  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:29.284878  5870 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:29.258479  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:29.211441  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:29.267731  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 396276us
I20260709 12:25:29.286975  5869 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 2.
I20260709 12:25:29.248405  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:29.271656  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:29.287834  6016 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:29.288028  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:29.149290  7758 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:29.275909  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:29.269847  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:29.289775  6016 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:29.292523  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:29.293605  6609 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:29.271162  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:29.290899  6020 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:29.301113  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:29.306394  6016 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:29.302273  5942 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:29.307777  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:29.308434  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:29.310601  5942 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 6: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:29.316442  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:29.338497  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:29.339318  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.343070  5946 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:29.343833  5915 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:29.278342  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:29.347375  6611 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:29.340516  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:29.285943  7781 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.350093  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:29.356863  5946 raft_consensus.cc:2393] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 4: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:29.351954  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:29.359921  5913 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:29.300839  7811 raft_consensus.cc:2764] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:25:29.306628  7836 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:29.362512  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:29.306488  7577 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:29.367304  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:29.310405  7752 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:29.305879  7292 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:29.334590  7438 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.370492  6768 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:25:29.384086  5870 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:29.385545  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:29.283144  7700 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 12:25:29.386449  5942 raft_consensus.cc:2410] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:25:29.426388  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:29.425951  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 82215.7us
I20260709 12:25:29.466888  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:25:29.435427  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 76105.4us
W20260709 12:25:29.436086  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 76642.5us
I20260709 12:25:29.468590  6609 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:25:29.426179  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 121991us
I20260709 12:25:29.469842  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:29.470489  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:29.437268  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:29.438470  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 91805.9us
I20260709 12:25:29.358883  7305 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.484103  5946 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 6: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:29.314500  7636 consensus_queue.cc:1059] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 12:25:29.366734  7818 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:29.337790  7588 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.114s
W20260709 12:25:29.452795  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.371974  7836 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.387246  7577 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.368755  5945 raft_consensus.cc:2376] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4. Re-sending same reply.
I20260709 12:25:29.388480  7845 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.142s
I20260709 12:25:29.449357  6778 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:29.372993  7329 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:29.513926  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.487924  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:29.453816  7766 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:29.454109  7840 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:29.544346  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:29.466255  6015 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:29.449577  7669 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.439910  6756 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:29.535979  5870 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:29.457579  7752 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:29.542837  5946 raft_consensus.cc:2410] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:29.526355  7836 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:29.538447  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 68011.2us
I20260709 12:25:29.520391  7818 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.552148  5840 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:29.618157  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 66108.3us
I20260709 12:25:29.553285  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:29.554314  7844 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:25:29.554682  7840 raft_consensus.cc:697] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:29.555105  7766 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:29.516793  7305 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:29.558136  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 213) took 2516 ms. Trace:
I20260709 12:25:29.632266  5868 rpcz_store.cc:276] 0709 12:25:27.041468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.823759 (+782291us) service_pool.cc:224] Handling call
0709 12:25:27.825317 (+  1558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.946286 (+120969us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.946330 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:27.946375 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:27.950678 (+  4303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.557855 (+1607177us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400193054. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:29.557916 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:29.558072 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":2492,"spinlock_wait_cycles":277085440}
I20260709 12:25:29.592787  7825 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:29.559221  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:29.636832  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 77721.6us
I20260709 12:25:29.638478  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:29.559898  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:29.562249  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:29.581122  6757 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:29.583485  7599 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:29.585245  7700 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:29.621552  7636 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:29.591679  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:29.549710  7720 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:29.596786  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:29.606609  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:29.652184  6021 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.519814  7625 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:29.604498  7816 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:29.620640  6609 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:29.525885  7852 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:29.622893  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:29.546046  7754 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:29.629432  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.557557  7669 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:29.639873  5947 raft_consensus.cc:2368] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5.
I20260709 12:25:29.619675  7818 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:29.643985  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:29.645115  7714 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:29.589475  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:29.664300  5946 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:25:29.648386  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 56902.1us
I20260709 12:25:29.526679  7577 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.658907  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:29.675225  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:29.653977  7758 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:29.677279  6756 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:25:29.654546  7562 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:29.658452  7852 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:25:29.671574  7843 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
I20260709 12:25:29.649133  7810 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:29.690493  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.659572  7754 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:29.678287  5940 raft_consensus.cc:2410] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:29.661178  7636 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:29.667439  7599 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:29.690094  6021 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:29.668428  7714 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.714497  6021 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:29.684641  7741 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.658296  7625 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:29.651633  7700 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:29.680668  7838 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.719638  6762 raft_consensus.cc:2393] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:29.706930  7369 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:29.713292  7599 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:29.710420  7754 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:29.720269  7714 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.764300  6021 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:29.766808  5871 raft_consensus.cc:2368] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5.
I20260709 12:25:29.696651  7852 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:29.770696  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 126756us
W20260709 12:25:29.774060  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.766119  7816 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:29.695405  7766 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:29.797057  7700 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:29.784266  7599 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.811486  6020 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
W20260709 12:25:29.822944  7855 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
W20260709 12:25:29.825606  7847 log.cc:927] Time spent T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 12:25:29.767373  7636 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:29.827381  7851 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:25:29.769692  7858 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:29.805711  7850 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.184s	user 0.001s	sys 0.001s
I20260709 12:25:29.801805  7752 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:29.706804  7840 consensus_queue.cc:237] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:29.853322  6775 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 12:25:29.859333  6773 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 498) took 2421 ms (client timeout 2838 ms). Trace:
I20260709 12:25:29.778481  7838 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:29.859894  6773 rpcz_store.cc:269] 0709 12:25:27.431131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.431282 (+   151us) service_pool.cc:224] Handling call
0709 12:25:29.852588 (+2421306us) spinlock_profiling.cc:243] Waited 83.1 ms on lock 0x7b5400470c54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:29.852827 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191075200}
I20260709 12:25:29.840020  7607 raft_consensus.cc:3055] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:25:29.850471  7834 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.666s	user 0.011s	sys 0.003s
W20260709 12:25:29.854044  7854 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:25:29.865617  6766 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:29.856880  7305 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:29.861092  7625 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:29.875170  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 82652.3us
I20260709 12:25:29.861699  7858 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:29.863611  7607 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:29.885124  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:29.888207  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:29.890059  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:29.889752  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:29.891265  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:29.899959  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:29.903003  5870 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:29.900552  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:29.895288  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:29.873787  7669 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:29.874068  7819 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:29.906859  5868 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:29.920161  7577 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:29.890931  7625 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:25:29.922466  7625 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:29.922894  7625 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:29.923254  7625 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.889565  7838 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:29.926129  5865 consensus_meta.cc:287] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.680s	user 0.012s	sys 0.001s
I20260709 12:25:29.925964  7599 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:29.928848  7599 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:29.929941  7864 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:25:29.935628  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:29.939189  7856 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:25:29.940243  7867 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:29.960871  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 65655.9us
I20260709 12:25:30.010926  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:29.926637  5865 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
W20260709 12:25:30.005316  5866 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.866s	user 0.014s	sys 0.000s
I20260709 12:25:29.867357  7834 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:30.007009  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:30.012408  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:29.909282  7562 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:29.883344  7858 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:25:30.095956  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.998147  7838 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.004874  7873 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:30.140388  5946 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 6: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:25:30.102023  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 91202.4us
I20260709 12:25:30.155798  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:30.144863  5870 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
W20260709 12:25:30.101301  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.146615  7874 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:25:30.147481  7861 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 12:25:30.161787  7597 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:30.100751  5866 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:29.990872  7819 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:30.043545  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 799) took 1021 ms. Trace:
I20260709 12:25:30.086823  7871 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:30.034552  7588 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:30.166225  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:30.167147  5946 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:30.162189  7711 consensus_meta.cc:287] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.964s	user 0.005s	sys 0.010s
I20260709 12:25:30.168286  5943 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:30.204325  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:30.148650  7858 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:30.177464  7759 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:30.186131  5868 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:30.186169  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.214335  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 48229us
I20260709 12:25:30.213526  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:30.172410  7808 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.869s	user 0.014s	sys 0.000s
I20260709 12:25:30.210568  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:30.043681  7867 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:30.199352  7728 consensus_meta.cc:287] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.170s	user 0.016s	sys 0.000s
I20260709 12:25:30.044225  7607 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:30.216845  5943 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:30.205986  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 797) took 1224 ms. Trace:
I20260709 12:25:30.167200  5870 raft_consensus.cc:3055] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:30.196554  7562 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:30.222751  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:30.200294  7871 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:30.220153  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:30.246197  5946 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:30.153070  7838 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:30.213485  7759 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:30.245347  5870 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:25:30.246999  5990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.552s	user 0.018s	sys 0.032s
I20260709 12:25:30.249068  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:30.195263  5865 rpcz_store.cc:276] 0709 12:25:29.022188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.203766 (+181578us) service_pool.cc:224] Handling call
0709 12:25:30.043391 (+839625us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:30.217571  7874 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:30.219915  7810 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:30.217379  7875 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:30.255112  7766 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.259380  5870 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:30.261464  7819 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:30.221657  7836 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.595s	user 0.015s	sys 0.000s
I20260709 12:25:30.212587  7588 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:30.267048  5870 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:30.246296  7607 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:30.277082  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 665) took 2468 ms. Trace:
I20260709 12:25:30.250006  7577 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:30.248212  7862 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:30.221206  7728 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:30.267170  7863 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.395s	user 0.001s	sys 0.001s
I20260709 12:25:30.286623  5874 rpcz_store.cc:276] 0709 12:25:27.808163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.948144 (+139981us) service_pool.cc:224] Handling call
0709 12:25:27.948512 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:27.948898 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.948930 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:27.949497 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 12:25:28.248262 (+298765us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:28.248307 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.248376 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.171585 (+1923209us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:30.276615 (+105030us) raft_consensus.cc:1685] finished
0709 12:25:30.276651 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.276976 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6160,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":5391,"spinlock_wait_cycles":3736960,"thread_start_us":298300,"threads_started":1,"wal-append.queue_time_us":353462}
I20260709 12:25:30.298357  5943 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:30.226120  5866 rpcz_store.cc:276] 0709 12:25:28.977338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.999042 (+ 21704us) service_pool.cc:224] Handling call
0709 12:25:30.201013 (+1201971us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:30.296061  7866 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.412s	user 0.000s	sys 0.002s
I20260709 12:25:30.310496  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:30.263949  7605 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:30.312736  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 103373us
I20260709 12:25:30.251572  7871 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:30.311623  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 243) took 1208 ms. Trace:
I20260709 12:25:30.249890  7562 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:30.316583  5869 rpcz_store.cc:276] 0709 12:25:29.103573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.369224 (+265651us) service_pool.cc:224] Handling call
0709 12:25:29.369578 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.523524 (+153946us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:29.628797 (+105273us) log.cc:844] Serialized 14 byte log entry
0709 12:25:29.629960 (+  1163us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:29.630019 (+    59us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:29.630298 (+   279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.311390 (+681092us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400027954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:30.311440 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.311576 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":118704,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":205,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":4324,"spinlock_wait_cycles":353367808,"thread_start_us":97450,"threads_started":1}
I20260709 12:25:30.324357  5870 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:25:30.316715  6609 leader_election.cc:336] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Timed out: RequestConsensusVote RPC to 127.2.4.195:43329 timed out after 1.807s (SENT)
I20260709 12:25:30.318646  5948 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:30.336265  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.300895  7875 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:30.279685  7766 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:30.312232  7774 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:30.292490  7836 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:30.352113  5990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.657s	user 0.019s	sys 0.032s
W20260709 12:25:30.352934  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 2.68138e+06us
I20260709 12:25:30.354382  5990 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:30.354732  6793 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 7: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:30.355114  7862 raft_consensus.cc:2764] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 12:25:30.356665  7852 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.561s	user 0.009s	sys 0.000s
I20260709 12:25:30.350731  7834 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:30.308606  7759 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:30.373492  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 237) took 1016 ms. Trace:
I20260709 12:25:30.359058  7436 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:30.375407  5947 rpcz_store.cc:276] 0709 12:25:29.356896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.695516 (+338620us) service_pool.cc:224] Handling call
0709 12:25:29.717834 (+ 22318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.722068 (+  4234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.722127 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.722851 (+   724us) log.cc:844] Serialized 26 byte log entry
0709 12:25:29.782092 (+ 59241us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:29.783805 (+  1713us) log.cc:844] Serialized 14 byte log entry
0709 12:25:29.787917 (+  4112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.788037 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.373075 (+585038us) raft_consensus.cc:1685] finished
0709 12:25:30.373113 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.373373 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8281,"prepare.queue_time_us":166,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":8954,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":47052,"thread_start_us":58810,"threads_started":1,"wal-append.queue_time_us":132777}
I20260709 12:25:30.364254  7852 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:30.438324  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:30.439337  5947 raft_consensus.cc:2410] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:30.445293  5946 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:30.447576  5946 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:30.425034  7838 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:30.429301  7728 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:30.417924  7875 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:30.436762  7867 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:30.382078  7436 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:30.447098  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.449195  5946 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:30.465520  6022 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:30.523698  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:30.524652  6022 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:30.466046  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:30.472237  6024 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:30.528609  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:30.472586  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:30.477767  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.475736  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:30.482211  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.533844  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:30.535447  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.482884  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 575) took 4276 ms. Trace:
I20260709 12:25:30.472898  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:30.539207  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 481) took 2281 ms. Trace:
I20260709 12:25:30.497129  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:30.540654  6756 raft_consensus.cc:2368] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4.
I20260709 12:25:30.507601  6773 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:25:30.541855  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:30.535544  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.507081  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 497) took 1212 ms. Trace:
I20260709 12:25:30.521066  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:30.498340  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:30.508140  5871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 796) took 1542 ms (client timeout 1624 ms). Trace:
I20260709 12:25:30.536139  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:30.490475  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 591) took 2206 ms. Trace:
I20260709 12:25:30.543277  6793 rpcz_store.cc:276] 0709 12:25:29.294755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.340225 (+ 45470us) service_pool.cc:224] Handling call
0709 12:25:30.506762 (+1166537us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b540015efd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:30.506982 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2311241088}
I20260709 12:25:30.465946  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:30.545980  6017 rpcz_store.cc:276] 0709 12:25:28.284031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.284186 (+   155us) service_pool.cc:224] Handling call
0709 12:25:28.284546 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.385492 (+100946us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:28.397020 (+ 11528us) log.cc:844] Serialized 14 byte log entry
0709 12:25:28.495342 (+ 98322us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:28.495418 (+    76us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:28.495482 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.706574 (+211092us) spinlock_profiling.cc:243] Waited 3.3 ms on lock 0x7b180026b480. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb642a6 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419
0709 12:25:30.490180 (+1783606us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400363dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:30.490234 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.490402 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9866,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":295,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":5585,"spinlock_wait_cycles":239053952,"thread_start_us":97630,"threads_started":1,"wal-append.queue_time_us":142911,"wal-append.run_cpu_time_us":1093,"wal-append.run_wall_time_us":1001109}
I20260709 12:25:30.547463  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:30.491983  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 515) took 1305 ms. Trace:
W20260709 12:25:30.473357  7818 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.801s	user 0.013s	sys 0.000s
I20260709 12:25:30.550076  7818 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:30.549177  7699 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:30.494330  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:30.542464  5913 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:30.545574  5869 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:30.525151  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:30.553416  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 519) took 1365 ms. Trace:
I20260709 12:25:30.553617  6766 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:30.544229  5871 rpcz_store.cc:269] 0709 12:25:28.966230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.966664 (+   434us) service_pool.cc:224] Handling call
0709 12:25:30.507916 (+1541252us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b54002f62d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:30.508072 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1757477760}
I20260709 12:25:30.543416  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:30.540140  6775 rpcz_store.cc:276] 0709 12:25:28.237000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.237181 (+   181us) service_pool.cc:224] Handling call
0709 12:25:29.852957 (+1615776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.854992 (+  2035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.517501 (+662509us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b5400470c50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:30.518396 (+   895us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3714902912}
I20260709 12:25:30.536338  6023 rpcz_store.cc:276] 0709 12:25:26.206469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.609123 (+402654us) service_pool.cc:224] Handling call
0709 12:25:28.429950 (+1820827us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:28.430365 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:28.430395 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:28.458151 (+ 27756us) log.cc:844] Serialized 26 byte log entry
0709 12:25:28.523125 (+ 64974us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:28.523156 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.523208 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:28.541707 (+ 18499us) raft_consensus.cc:1685] finished
0709 12:25:28.541730 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.482478 (+1940748us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b54003647d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:30.482797 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28018,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":13130,"spinlock_wait_cycles":4186435968,"thread_start_us":38528,"threads_started":1,"wal-append.queue_time_us":59455,"wal-append.run_cpu_time_us":1864,"wal-append.run_wall_time_us":1003449}
W20260709 12:25:30.595628  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 53258.7us
W20260709 12:25:30.596984  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 102764us
I20260709 12:25:30.599741  6768 rpcz_store.cc:276] 0709 12:25:29.188285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.188416 (+   131us) service_pool.cc:224] Handling call
0709 12:25:29.188834 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.453967 (+265133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.552945 (+1098978us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b5400200154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:30.553330 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416735488}
W20260709 12:25:30.472563  7714 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.667s	user 0.010s	sys 0.000s
I20260709 12:25:30.596236  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:30.471412  7878 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:25:30.601871  6756 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 7: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:30.556941  7845 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:30.610744  7773 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:30.490633  7858 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:30.467938  7781 consensus_meta.cc:287] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.106s	user 0.010s	sys 0.004s
I20260709 12:25:30.548724  6793 raft_consensus.cc:2368] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6.
I20260709 12:25:30.669194  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:30.670316  7607 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.670436  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:30.671092  7607 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:30.673350  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 708) took 3002 ms. Trace:
I20260709 12:25:30.450765  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:30.674247  5945 raft_consensus.cc:2410] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:30.488471  6609 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:30.475354  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 494) took 1186 ms. Trace:
I20260709 12:25:30.497565  7810 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.115s
I20260709 12:25:30.474727  7588 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:30.462373  7754 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.663s	user 0.010s	sys 0.006s
I20260709 12:25:30.583976  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 517) took 1396 ms. Trace:
I20260709 12:25:30.494656  7867 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.181s
W20260709 12:25:30.619763  7823 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 1.967s	user 0.002s	sys 0.013s
I20260709 12:25:30.620585  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:30.557128  7517 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.120s
I20260709 12:25:30.477348  7819 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.627434  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:30.483063  7852 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:30.633884  7837 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.334s	user 0.003s	sys 0.011s
W20260709 12:25:30.634393  7625 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.711s	user 0.018s	sys 0.011s
I20260709 12:25:30.611292  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:30.639921  7707 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:30.640626  7599 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.651451  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:30.485363  7741 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.751s	user 0.004s	sys 0.009s
I20260709 12:25:30.486881  7329 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:30.522622  7414 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.512192  7799 raft_consensus.cc:3055] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:30.496666  7745 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.655244  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:30.552760  7818 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:30.828886  7745 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:30.666210  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.666354  7430 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:30.666774  7845 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:30.502028  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:30.495580  7877 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.190s	user 0.001s	sys 0.002s
I20260709 12:25:30.498310  7766 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:30.592742  7838 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:30.665975  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 57701.3us
I20260709 12:25:30.673861  5949 rpcz_store.cc:276] 0709 12:25:27.671279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.671472 (+   193us) service_pool.cc:224] Handling call
0709 12:25:27.671810 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:27.673723 (+  1913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.673760 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:30.089317 (+2415557us) spinlock_profiling.cc:243] Waited 2.84 ms on lock 0x7b2c001b1e88. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:25:30.089918 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.308770 (+218852us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:30.308808 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.308904 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.470764 (+161860us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:30.673069 (+202305us) raft_consensus.cc:1685] finished
0709 12:25:30.673092 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.673274 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1486,"raft.queue_time_us":2152,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":51,"spinlock_wait_cycles":13703168,"thread_start_us":142471,"threads_started":1,"wal-append.queue_time_us":203442}
W20260709 12:25:30.675295  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 186945us
I20260709 12:25:30.677945  6022 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:30.595153  7699 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:30.823972  7599 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:30.876925  7818 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:30.683657  6762 rpcz_store.cc:276] 0709 12:25:29.187630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.187825 (+   195us) service_pool.cc:224] Handling call
0709 12:25:30.583598 (+1395773us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54002c1ad4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:30.583880 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1219590016}
I20260709 12:25:30.609952  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:30.706686  6793 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:30.879802  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:30.622499  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:30.707094  6023 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:25:30.799880  5840 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.196:34835 timed out after 1.562s (SENT) [suppressed 47 similar messages]
I20260709 12:25:30.881212  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:30.882839  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 82985.6us
I20260709 12:25:30.883674  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:30.680964  7773 raft_consensus.cc:697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:30.819943  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:30.890199  6024 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:30.825337  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:30.891654  6020 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:30.654294  7714 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:30.827370  6756 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:30.471637  7436 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:30.554216  6778 rpcz_store.cc:276] 0709 12:25:29.186936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.201825 (+ 14889us) service_pool.cc:224] Handling call
0709 12:25:29.202336 (+   511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.540782 (+338446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.491687 (+950905us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54002a0dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:30.491912 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":515458944}
I20260709 12:25:30.689744  7754 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:30.813898  7834 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:30.754209  7562 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:30.846817  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:30.872067  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:30.872445  5949 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:30.923657  6022 raft_consensus.cc:2368] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4.
I20260709 12:25:30.924547  5949 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:25:30.876750  7838 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:30.926290  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:30.927138  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:30.927578  7838 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:30.928126  6015 raft_consensus.cc:2410] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:30.928179  7838 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:30.928706  6778 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:30.886106  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:30.929152  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:30.822757  7799 raft_consensus.cc:740] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:30.892896  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:30.669477  7858 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:30.802044  7625 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:30.898734  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:30.708870  7819 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:30.896593  5990 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:30.946327  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 49868.7us
I20260709 12:25:30.948024  5990 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:30.948342  6766 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:25:30.948967  6766 pending_rounds.cc:85] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 3
I20260709 12:25:30.948946  5990 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:30.823436  7741 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:30.950230  6021 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:30.831874  7836 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:30.907986  6609 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:30.676891  6757 rpcz_store.cc:276] 0709 12:25:29.288752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.289585 (+   833us) service_pool.cc:224] Handling call
0709 12:25:29.580855 (+291270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.645933 (+ 65078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.475117 (+829184us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b5400574ed0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:30.475280 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":539572992}
I20260709 12:25:30.929874  6762 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:30.951997  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:30.952503  7836 consensus_queue.cc:1059] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:30.930015  5949 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:30.954161  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:30.955338  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:30.955585  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.956013  6757 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 12:25:30.956393  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.924602  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:30.957234  6022 raft_consensus.cc:2368] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4.
I20260709 12:25:30.958863  6768 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
W20260709 12:25:30.959249  5942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.001s	sys 0.018s
I20260709 12:25:30.874616  6775 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:30.830390  7414 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:30.923446  7672 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.964986  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:30.965885  6024 raft_consensus.cc:2393] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:30.934537  7588 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.936527  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 237) took 2220 ms. Trace:
I20260709 12:25:30.973425  5873 rpcz_store.cc:276] 0709 12:25:28.716649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.716778 (+   129us) service_pool.cc:224] Handling call
0709 12:25:28.724934 (+  8156us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:28.727430 (+  2496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:28.727468 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:28.727997 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 12:25:28.922744 (+194747us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:28.922806 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.922910 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.630189 (+1707279us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:30.912660 (+282471us) raft_consensus.cc:1685] finished
0709 12:25:30.912688 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.936439 (+ 23751us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2189,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":3455,"spinlock_wait_cycles":77477248,"thread_start_us":178747,"threads_started":1,"wal-append.queue_time_us":525731}
I20260709 12:25:30.876784  7699 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:30.940533  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 982) took 2157 ms. Trace:
I20260709 12:25:30.978665  5948 rpcz_store.cc:276] 0709 12:25:28.783461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.783606 (+   145us) service_pool.cc:224] Handling call
0709 12:25:28.797889 (+ 14283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.335352 (+1537463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.925426 (+590074us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54002e7054. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:30.940425 (+ 14999us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3496729344}
I20260709 12:25:30.885517  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:30.941252  6015 raft_consensus.cc:2410] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:30.942688  6023 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:30.949456  6766 pending_rounds.cc:107] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 5.4
W20260709 12:25:30.959645  5942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.001s	sys 0.018s
I20260709 12:25:30.912058  7880 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:30.961928  6775 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:30.963222  6756 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:30.963524  5840 leader_election.cc:400] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4.
I20260709 12:25:30.958479  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:30.988626  5840 leader_election.cc:403] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:25:30.965977  5990 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:30.894767  7766 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 12:25:30.982292  6766 raft_consensus.cc:2889] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:30.960826  7840 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.009s	sys 0.022s
I20260709 12:25:31.000442  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:31.000682  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 445675us
I20260709 12:25:30.976743  7834 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:31.004278  5840 leader_election.cc:400] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4.
I20260709 12:25:30.945686  7775 consensus_queue.cc:1059] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:25:30.979660  7858 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:30.986464  7750 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:25:30.993968  7840 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.009s	sys 0.023s
I20260709 12:25:30.998576  7430 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:30.995081  7862 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:30.997321  7852 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:30.981841  7672 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:31.057304  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:31.060711  6024 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:31.006752  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:30.995607  7881 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:31.034789  6775 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:31.007509  7875 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:31.046388  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.005497  7818 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:31.025941  7867 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:30.982275  7588 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:30.994827  7880 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:31.033365  7436 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:31.064477  6015 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:25:31.034715  7755 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.011s	sys 0.009s
W20260709 12:25:31.069077  7793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.711s	user 0.025s	sys 0.020s
I20260709 12:25:31.066429  7875 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:31.034235  7592 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:25:31.090337  6022 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:31.073076  6611 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:31.071282  7562 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.000101  7799 consensus_queue.cc:260] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:31.059840  7858 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
W20260709 12:25:31.084411  7755 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.011s	sys 0.009s
W20260709 12:25:31.090768  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 476586us
I20260709 12:25:31.071794  7881 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:31.110416  7881 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:31.071875  7430 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
W20260709 12:25:31.095297  7793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.743s	user 0.026s	sys 0.021s
I20260709 12:25:31.005442  7773 consensus_queue.cc:237] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:31.074213  7867 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:31.132866  7759 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:31.133324  7867 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:31.135435  7885 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:25:31.029471  7845 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:31.074096  7818 raft_consensus.cc:697] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:31.138244  7754 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:31.139066  7818 consensus_queue.cc:237] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:31.074899  7436 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:31.119982  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:31.146600  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:31.128556  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:31.097988  7875 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:31.105952  7562 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:31.122062  7799 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:31.148164  7625 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:31.144043  7899 raft_consensus.cc:3055] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:31.152823  7672 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:31.148520  7896 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.154371  7588 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:31.155687  7880 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:31.140980  7714 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:31.129319  7592 consensus_queue.cc:1243] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:31.157099  7745 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:31.157385  5940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 218) took 2108 ms (client timeout 1734 ms). Trace:
W20260709 12:25:31.201485  5940 rpcz_store.cc:269] 0709 12:25:29.049074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.238846 (+189772us) service_pool.cc:224] Handling call
0709 12:25:31.157114 (+1918268us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b540058f554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:31.157301 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":924152576}
I20260709 12:25:31.157899  7819 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:31.159801  5944 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:25:31.203493  5944 pending_rounds.cc:85] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 2
I20260709 12:25:31.203179  7699 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:31.161090  7898 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:31.175943  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:31.210485  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:31.211369  6017 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:31.212409  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:31.212864  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:31.213274  6017 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:31.214393  6024 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:31.216276  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:31.183246  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.020s	sys 0.015s
I20260709 12:25:31.185941  7588 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:31.085426  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 189831us
I20260709 12:25:31.164855  7754 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:25:31.184036  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.194371  7436 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:31.203828  5944 pending_rounds.cc:107] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:25:31.142908  7728 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:31.224316  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.020s	sys 0.015s
I20260709 12:25:31.175297  7899 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:31.284682  7562 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:31.284314  7899 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:31.285483  7899 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:31.288406  7599 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:31.261669  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:31.262413  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:31.292002  5869 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:31.263214  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:31.194576  7414 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:31.265959  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 79391.5us
I20260709 12:25:31.267204  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 716) took 1019 ms. Trace:
I20260709 12:25:31.273546  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:31.274075  5944 raft_consensus.cc:2889] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:31.276732  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:31.278827  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 627439us
I20260709 12:25:31.310549  5868 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
W20260709 12:25:31.279465  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.282212  7436 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:31.282738  7728 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:31.254522  7898 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:31.312584  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 239225us
I20260709 12:25:31.309055  5949 rpcz_store.cc:276] 0709 12:25:30.247721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.954634 (+706913us) service_pool.cc:224] Handling call
0709 12:25:30.955004 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.955220 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:31.111662 (+156442us) log.cc:844] Serialized 14 byte log entry
0709 12:25:31.265471 (+153809us) log.cc:844] Serialized 14 byte log entry
0709 12:25:31.266144 (+   673us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:31.266375 (+   231us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:31.266541 (+   166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.266726 (+   185us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:31.267060 (+   334us) inbound_call.cc:177] Queueing success response
0709 12:25:31.290018 (+ 22958us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b48001bc258. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d7bacaa1 00007f39d7ba3b37 00007f39d7ba39ca 00007f39d7c9b26f 00007f39d7c9af37 00007f39d7caf55c 00007f39d7caf508 00007f39d7caf488 00007f39d7caf443 00007f39d7cae6ec 00007f39d6bd8a96
Metrics: {"prepare.queue_time_us":191816,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":17390,"raft.run_cpu_time_us":2067,"raft.run_wall_time_us":34771,"spinlock_wait_cycles":42553856,"thread_start_us":299142,"threads_started":2}
I20260709 12:25:31.250478  7834 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:31.350973  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:31.256098  7592 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:31.254280  7745 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:31.267478  7597 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:31.357551  6017 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:31.323905  7754 raft_consensus.cc:697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:31.265609  7588 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:31.385824  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 59448.5us
I20260709 12:25:31.382584  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:31.326087  7755 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:31.198414  7896 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:31.388453  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:31.394829  7562 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:31.386579  7894 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:25:31.396801  7811 raft_consensus.cc:993] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: : Instructing follower 1fc57407d56241b38a94d543e073f8af to start an election
I20260709 12:25:31.371515  7745 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.374384  7781 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:31.399132  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:31.402132  6611 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:31.377048  7414 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:31.388824  6785 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:31.380824  7711 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
W20260709 12:25:31.319370  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.558130  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:31.331039  5869 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:31.617125  5868 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:31.299201  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:31.574026  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.619410  7607 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:31.575279  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 429257us
I20260709 12:25:31.622152  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:31.623201  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:31.624336  5940 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:31.575546  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 204623us
I20260709 12:25:31.625689  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:31.626232  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:31.426215  6023 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:31.588092  5941 raft_consensus.cc:2410] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:25:31.588864  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.629561  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 328) took 1144 ms. Trace:
I20260709 12:25:31.630649  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:31.631138  7499 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.632192  7499 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:31.636476  7499 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:31.638351  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:31.597344  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.597371  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:31.617187  7588 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:31.642275  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:31.642920  7589 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:31.643579  6775 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 2: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:31.644256  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:31.644071  7589 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:31.645408  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:31.437880  7414 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:31.391803  7592 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 5 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:31.410130  7699 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:31.621776  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:31.630229  6023 rpcz_store.cc:276] 0709 12:25:30.485210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.423580 (+938370us) service_pool.cc:224] Handling call
0709 12:25:31.424057 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.628772 (+204715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.629285 (+   513us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:31.648301  7592 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:31.643388  7607 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.652925  6023 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:31.653716  7589 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:31.653419  7607 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:31.650437  6775 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:31.657796  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:31.658434  7589 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:31.659302  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:31.662343  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:31.662806  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:31.650841  7699 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 12:25:31.650025  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:31.664942  5949 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 6: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:31.667022  7588 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:31.599524  5872 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:31.415732  7745 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:31.692735  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:31.437583  7781 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:31.417371  7793 raft_consensus.cc:1081] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Signalling peer 1fc57407d56241b38a94d543e073f8af to start an election
I20260709 12:25:31.649619  7414 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:31.397980  7834 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:31.701252  7728 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:31.463454  7852 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4.
I20260709 12:25:31.464826  7862 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:31.476029  7880 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:31.380807  7893 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 12:25:31.409039  7896 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:31.397086  7898 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:31.513525  7908 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.100s	user 0.000s	sys 0.004s
I20260709 12:25:31.646299  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:31.637331  7499 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.730743  5942 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:31.731096  7499 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:31.734349  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:31.739750  7871 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:31.750697  7711 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:31.752694  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 350448us
I20260709 12:25:31.702677  7754 consensus_queue.cc:237] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:31.754205  7775 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:31.755349  7781 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:31.657120  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:31.668165  7607 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:31.469599  7810 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:31.724972  6775 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:31.752322  7711 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:31.755947  7775 raft_consensus.cc:3055] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:31.697021  7588 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:31.687515  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 329) took 1160 ms. Trace:
I20260709 12:25:31.799392  6023 rpcz_store.cc:276] 0709 12:25:30.521884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.651691 (+1129807us) service_pool.cc:224] Handling call
0709 12:25:31.652413 (+   722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.653535 (+  1122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.681797 (+ 28262us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:31.765924  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.767978  7818 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:31.770185  7904 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.771754  7738 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:25:31.778365  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 148785us
I20260709 12:25:31.779390  7607 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.735628  5867 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
W20260709 12:25:31.758885  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.781490  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.720180  7880 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:31.820794  5942 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
W20260709 12:25:31.813966  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.785923  7499 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:31.826869  7880 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:31.834030  7898 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.838362  7588 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:31.840180  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:31.798638  7775 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:31.860685  6020 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:31.861789  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 737) took 1140 ms. Trace:
I20260709 12:25:31.862154  6020 rpcz_store.cc:276] 0709 12:25:30.721649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.855746 (+1134097us) service_pool.cc:224] Handling call
0709 12:25:31.860206 (+  4460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.861384 (+  1178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.861686 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:31.862511  7810 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:31.864467  6020 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:31.866503  5945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 221) took 2816 ms (client timeout 2191 ms). Trace:
I20260709 12:25:31.866667  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:31.867089  5945 rpcz_store.cc:269] 0709 12:25:29.050076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.552946 (+502870us) service_pool.cc:224] Handling call
0709 12:25:31.865979 (+2313033us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54003084d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:31.866260 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2411000192}
I20260709 12:25:31.868184  5865 raft_consensus.cc:2410] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:25:31.868782  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:31.869917  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:31.870838  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 322) took 1387 ms. Trace:
I20260709 12:25:31.871197  5867 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:31.871994  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 710) took 1985 ms. Trace:
I20260709 12:25:31.872731  7818 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:31.873571  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:31.875641  7818 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:31.876695  7738 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:31.878408  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 612) took 1414 ms. Trace:
I20260709 12:25:31.879099  7738 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:31.848546  7904 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:31.882062  7607 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:31.883100  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 608) took 1362 ms. Trace:
I20260709 12:25:31.883536  6757 rpcz_store.cc:276] 0709 12:25:30.520518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.951522 (+431004us) service_pool.cc:224] Handling call
0709 12:25:30.955703 (+  4181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:30.959751 (+  4048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.882607 (+922856us) spinlock_profiling.cc:243] Waited 3.67 ms on lock 0x7b5400005850. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:31.882881 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8429824}
I20260709 12:25:31.884797  7904 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:31.887053  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:31.891769  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 523) took 2230 ms. Trace:
I20260709 12:25:31.891999  7759 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:25:31.892980  7759 consensus_queue.cc:1243] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:25:31.875834  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:31.885141  5869 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:31.889289  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:31.892350  6773 rpcz_store.cc:276] 0709 12:25:29.661726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.861147 (+199421us) service_pool.cc:224] Handling call
0709 12:25:30.507174 (+646027us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:30.541820 (+ 34646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.891213 (+1349393us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b540015efd0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:31.891505 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1484782976}
I20260709 12:25:31.865394  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 738) took 1143 ms. Trace:
I20260709 12:25:31.898739  6020 rpcz_store.cc:276] 0709 12:25:30.721888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.863439 (+1141551us) service_pool.cc:224] Handling call
0709 12:25:31.864033 (+   594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.865044 (+  1011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.865308 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:31.900511  5945 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 4: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:31.892505  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:31.904615  5940 raft_consensus.cc:2410] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:25:31.907207  5874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.006s	sys 0.004s
I20260709 12:25:31.908107  7921 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:31.908190  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:31.909106  7921 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:31.910727  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 750) took 1033 ms. Trace:
I20260709 12:25:31.911249  6022 rpcz_store.cc:276] 0709 12:25:30.877302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.958101 (+ 80799us) service_pool.cc:224] Handling call
0709 12:25:31.910243 (+952142us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400641754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:31.910452 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232440704}
I20260709 12:25:31.912030  7921 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:31.878886  6021 rpcz_store.cc:276] 0709 12:25:30.463998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.466024 (+  2026us) service_pool.cc:224] Handling call
0709 12:25:30.466278 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.950614 (+484336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.877714 (+927100us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b54005659d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:31.878108 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1102177152}
I20260709 12:25:31.910966  6793 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:31.917544  5947 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:25:31.918087  5947 pending_rounds.cc:85] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 2
I20260709 12:25:31.918365  5947 pending_rounds.cc:107] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:25:31.918671  5947 raft_consensus.cc:2889] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:31.919143  6793 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:31.919724  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.921422  6793 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:25:31.923121  5870 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:31.922577  7436 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:31.922298  7862 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.935070  7436 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:31.935396  7862 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:31.885373  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:31.936673  7880 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:31.937183  6757 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:31.924482  7900 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:31.938578  7900 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.938925  7900 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:31.872401  5943 rpcz_store.cc:276] 0709 12:25:29.886738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.334940 (+448202us) service_pool.cc:224] Handling call
0709 12:25:30.340877 (+  5937us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.360459 (+ 19582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.360495 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:30.360543 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:30.360586 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.206599 (+846013us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b18000404e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb642a6 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419
0709 12:25:31.871452 (+664853us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b54006761d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:31.871519 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:31.871737 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1565,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":3549,"spinlock_wait_cycles":48899328}
I20260709 12:25:31.864503  7810 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:31.942250  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:31.943473  7810 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:31.943794  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:31.924064  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 84053.9us
I20260709 12:25:31.947938  7867 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.948598  7867 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:31.860512  7898 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:31.951098  7867 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:31.951699  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:31.952584  7898 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:31.862203  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:31.953516  7862 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:31.927300  6766 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:31.945186  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:31.956296  6016 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:31.932195  5915 leader_election.cc:400] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6.
I20260709 12:25:31.957085  5915 leader_election.cc:403] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:25:31.956372  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:31.913357  6022 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:31.948653  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:31.957589  5943 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:31.986591  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:31.930836  7775 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:31.989961  7711 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:31.990146  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:31.871313  6024 rpcz_store.cc:276] 0709 12:25:30.483468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.422831 (+939363us) service_pool.cc:224] Handling call
0709 12:25:31.423273 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.433390 (+ 10117us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.433422 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.433962 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:25:31.845892 (+411930us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:31.870209 (+ 24317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.870346 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:31.870416 (+    70us) raft_consensus.cc:1685] finished
0709 12:25:31.870426 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:31.870599 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11958,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":126,"thread_start_us":326129,"threads_started":1,"wal-append.queue_time_us":346322}
I20260709 12:25:31.991693  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:31.992043  5942 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:31.992929  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:31.995229  7436 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:31.975157  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 736) took 1253 ms. Trace:
I20260709 12:25:31.996054  6016 rpcz_store.cc:276] 0709 12:25:30.721406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.955351 (+1233945us) service_pool.cc:224] Handling call
0709 12:25:31.955892 (+   541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.965695 (+  9803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.975086 (+  9391us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:31.997340  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:31.998548  6016 raft_consensus.cc:2410] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:31.999576  7589 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:32.000161  7589 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:32.002199  7934 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 12:25:32.003700  7799 consensus_meta.cc:287] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.659s	user 0.007s	sys 0.013s
I20260709 12:25:31.884830  7607 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:32.005399  7867 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:25:31.984791  7898 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:32.008809  5948 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:32.015506  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 746) took 1157 ms. Trace:
I20260709 12:25:32.009421  5946 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
W20260709 12:25:32.005576  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 57065.1us
I20260709 12:25:32.002239  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:32.014568  6024 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 4: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:32.017527  5946 pending_rounds.cc:85] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 2
I20260709 12:25:32.018715  5946 pending_rounds.cc:107] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:25:32.016386  6022 rpcz_store.cc:276] 0709 12:25:30.821575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.912587 (+1091012us) service_pool.cc:224] Handling call
0709 12:25:31.913067 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.968678 (+ 55611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.975222 (+  6544us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:31.993485  5942 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:32.013492  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:32.019057  5946 raft_consensus.cc:2889] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:32.006309  5868 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:32.010349  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 506) took 1491 ms. Trace:
I20260709 12:25:31.923403  7728 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:32.024358  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:32.008503  7873 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:32.023540  6768 rpcz_store.cc:276] 0709 12:25:30.518737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.609396 (+ 90659us) service_pool.cc:224] Handling call
0709 12:25:32.009842 (+1400446us) spinlock_profiling.cc:243] Waited 79.6 ms on lock 0x7b5400005354. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:32.010127 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183170304}
I20260709 12:25:32.034047  5946 pending_rounds.cc:107] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:25:32.034605  5946 raft_consensus.cc:2889] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:32.046936  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:32.048148  6017 raft_consensus.cc:2410] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:25:32.048843  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:32.050800  5941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 225) took 2999 ms (client timeout 1719 ms). Trace:
W20260709 12:25:32.051443  5941 rpcz_store.cc:269] 0709 12:25:29.051376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.450370 (+1398994us) service_pool.cc:224] Handling call
0709 12:25:32.050452 (+1600082us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b54006a3bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:32.050673 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2074161536}
I20260709 12:25:32.054147  5940 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:32.054322  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:32.011023  7875 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.836s	user 0.022s	sys 0.000s
I20260709 12:25:32.074290  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:32.084550  6019 raft_consensus.cc:2368] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4.
I20260709 12:25:32.075227  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:32.087743  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 62370.9us
I20260709 12:25:32.014036  7898 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:32.007145  7921 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 12:25:32.100126  6024 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:32.108625  5949 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:32.110824  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:32.112977  6022 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:32.114946  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:32.085850  5865 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:32.121655  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:32.075104  7867 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:25:32.086505  7875 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:32.103459  7414 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:25:32.130419  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 128319us
I20260709 12:25:32.132963  5942 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:32.096163  7904 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
W20260709 12:25:32.138664  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 149485us
I20260709 12:25:32.139415  5913 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:32.171411  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:32.173575  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 63944.6us
I20260709 12:25:32.178763  6016 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:25:32.154472  7836 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.068s	user 0.015s	sys 0.033s
W20260709 12:25:32.187564  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 255700us
I20260709 12:25:32.189363  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 336) took 1142 ms. Trace:
I20260709 12:25:32.191438  6016 rpcz_store.cc:276] 0709 12:25:31.038741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.106082 (+1067341us) service_pool.cc:224] Handling call
0709 12:25:32.145518 (+ 39436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.179370 (+ 33852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.181526 (+  2156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:32.192698  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:32.185611  7880 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
W20260709 12:25:32.185370  7836 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.232s	user 0.016s	sys 0.033s
I20260709 12:25:32.201220  5874 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
W20260709 12:25:32.207204  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.123009  6778 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:43766
I20260709 12:25:32.190641  7436 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:32.189029  7589 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:32.216102  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.216926  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:32.197711  7773 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:32.184422  7893 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6.
I20260709 12:25:32.173115  7414 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:32.222575  6766 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
W20260709 12:25:32.232551  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 93281.5us
I20260709 12:25:32.223176  6778 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:25:32.249541  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.252043  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:32.253561  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.253731  6015 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:32.254873  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:32.256906  7939 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.257279  5840 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:32.257779  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 69054.5us
W20260709 12:25:32.283406  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.288749  7414 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:32.291944  7754 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:32.294513  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 504) took 1776 ms. Trace:
I20260709 12:25:32.295329  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:32.295202  6785 rpcz_store.cc:276] 0709 12:25:30.518100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.932335 (+414235us) service_pool.cc:224] Handling call
0709 12:25:30.932786 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.433453 (+500667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.293757 (+860304us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b54002a17d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:32.294251 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1007269120}
I20260709 12:25:32.298146  7893 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:32.298740  7958 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.299638  7958 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:32.300401  7893 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:25:32.303215  7945 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:25:32.226260  7745 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:32.301364  7893 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:32.262071  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:32.317962  7893 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:25:32.319216  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 57318.6us
I20260709 12:25:32.319434  7893 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:32.303980  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:32.321254  7810 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.321991  7867 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:32.322674  7867 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:32.322187  7810 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:32.323154  7867 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:32.324121  7948 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.171s	user 0.002s	sys 0.000s
I20260709 12:25:32.325197  7810 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:32.247970  6778 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:32.316718  7871 raft_consensus.cc:3055] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:32.316936  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 984) took 3205 ms. Trace:
I20260709 12:25:32.320689  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:32.325852  7899 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:32.326124  7750 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:32.309906  7588 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:32.327178  7773 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:32.307197  7898 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:32.260222  7875 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:32.327742  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 735) took 1606 ms. Trace:
I20260709 12:25:32.345599  7958 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:32.333897  7921 raft_consensus.cc:3055] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:25:32.311932  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:32.335881  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 44255.4us
I20260709 12:25:32.342213  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 241) took 2154 ms. Trace:
I20260709 12:25:32.342590  6785 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:32.374742  6017 rpcz_store.cc:276] 0709 12:25:30.721149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.388040 (+666891us) service_pool.cc:224] Handling call
0709 12:25:32.327456 (+939416us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b54001603d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:32.327663 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1490754944}
W20260709 12:25:32.258823  7943 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:25:32.309685  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 52560.2us
I20260709 12:25:32.259287  7592 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:32.261793  5865 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:32.354586  5944 rpcz_store.cc:276] 0709 12:25:29.106866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.664588 (+557722us) service_pool.cc:224] Handling call
0709 12:25:31.159400 (+1494812us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.310465 (+151065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.309429 (+998964us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b540058f550. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:32.312182 (+  2753us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3401536384}
I20260709 12:25:32.328397  7745 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:32.306586  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 624) took 1107 ms. Trace:
I20260709 12:25:32.381955  5947 rpcz_store.cc:276] 0709 12:25:30.188281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.643350 (+455069us) service_pool.cc:224] Handling call
0709 12:25:31.916663 (+1273313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.919300 (+  2637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.341909 (+422609us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54003084d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:32.342118 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2823055616}
I20260709 12:25:32.354446  7871 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:32.391332  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 528) took 1323 ms. Trace:
I20260709 12:25:32.257855  7939 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:32.398819  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 95271us
W20260709 12:25:32.403095  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 158170us
I20260709 12:25:32.416496  7414 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.416627  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.397821  6756 rpcz_store.cc:276] 0709 12:25:31.067952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.880154 (+812202us) service_pool.cc:224] Handling call
0709 12:25:31.880757 (+   603us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.881465 (+   708us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:31.912929 (+ 31464us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.116418 (+203489us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.135663 (+ 19245us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.135830 (+   167us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:32.135877 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.135950 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.323601 (+187651us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b24001565a8. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfa1c9c4 00007f39cfa1aacb 00007f39cfbd5dc9 00007f39cfbd5a55 00007f39cfbd509d 00007f39d280710e 00007f39d281e88c 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221
0709 12:25:32.375893 (+ 52292us) raft_consensus.cc:1685] finished
0709 12:25:32.390794 (+ 14901us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.391034 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38900,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":8546,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":2717,"spinlock_wait_cycles":68841856,"thread_start_us":200761,"threads_started":1,"wal-append.queue_time_us":239194}
I20260709 12:25:32.412199  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 330) took 1881 ms. Trace:
I20260709 12:25:32.418710  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.413470  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.387305  7592 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.394631  6773 rpcz_store.cc:276] 0709 12:25:31.199094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.914898 (+715804us) service_pool.cc:224] Handling call
0709 12:25:31.915451 (+   553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.916343 (+   892us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.916413 (+    70us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.917233 (+   820us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.156585 (+239352us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:32.156714 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.156914 (+   200us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.306279 (+149365us) raft_consensus.cc:1685] finished
0709 12:25:32.306305 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.306478 (+   173us) inbound_call.cc:177] Queueing success response
0709 12:25:32.330787 (+ 24309us) spinlock_profiling.cc:243] Waited 51.9 ms on lock 0x7b480052c158. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d7bacaa1 00007f39d7ba3b37 00007f39d7ba39ca 00007f39d7c9b26f 00007f39d7c9af37 00007f39d7caf55c 00007f39d7caf508 00007f39d7caf488 00007f39d7caf443 00007f39d7cae6ec 00007f39d6bd8a96
Metrics: {"raft.queue_time_us":4264,"raft.run_cpu_time_us":4404,"raft.run_wall_time_us":408320,"spinlock_wait_cycles":119397248,"thread_start_us":221868,"threads_started":1,"wal-append.queue_time_us":297447}
I20260709 12:25:32.383980  7921 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:32.412086  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.417308  7414 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:32.429414  7799 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:32.429575  7810 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:32.396925  7899 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:32.431752  7899 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:32.433087  6020 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:32.433269  5840 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:32.433790  7588 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:32.434593  7745 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:32.434827  5840 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:32.437762  7799 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:32.438280  7799 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.438654  7799 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:32.443677  7960 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.145s	user 0.003s	sys 0.000s
I20260709 12:25:32.443748  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:32.445935  7414 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:32.446473  7811 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:32.458513  7811 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:32.459403  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:32.459884  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:32.459856  7811 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:32.462078  7810 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:32.461181  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:32.462643  7810 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:32.462971  7781 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:32.463279  7810 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:32.464139  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 143595us
I20260709 12:25:32.463894  7799 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:32.465130  5913 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:32.445947  7898 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:32.462630  7811 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:32.447325  7928 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.718s	user 0.005s	sys 0.000s
I20260709 12:25:32.467098  7799 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:32.448019  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 743) took 1631 ms. Trace:
W20260709 12:25:32.450811  7900 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.511s	user 0.014s	sys 0.001s
I20260709 12:25:32.434827  7588 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:32.482185  7588 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:32.485725  7741 raft_consensus.cc:3055] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:32.486320  7741 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:32.486482  7592 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:32.487740  7741 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:32.402961  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.452906  7871 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:32.481263  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:32.456275  7939 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:32.405018  7880 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.468950  6020 rpcz_store.cc:276] 0709 12:25:30.816609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.962554 (+1145945us) service_pool.cc:224] Handling call
0709 12:25:32.432610 (+470056us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b5800480098. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:25:32.432878 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.447689 (+ 14811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.447947 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55040768}
I20260709 12:25:32.509057  5942 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:32.435055  7745 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:32.481436  6017 pending_rounds.cc:85] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 2
I20260709 12:25:32.478082  5872 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:32.509836  6017 pending_rounds.cc:107] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:25:32.510397  6017 raft_consensus.cc:2889] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:32.493446  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 67590.6us
I20260709 12:25:32.437022  7597 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.141s
I20260709 12:25:32.418467  6023 rpcz_store.cc:276] 0709 12:25:30.522259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.858040 (+1335781us) service_pool.cc:224] Handling call
0709 12:25:31.890136 (+ 32096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.890538 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:32.110469 (+219931us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.320255 (+209786us) spinlock_profiling.cc:243] Waited 897 us on lock 0x7b18003d6e40. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:25:32.392403 (+ 72148us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:32.392480 (+    77us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:32.403593 (+ 11113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.403678 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.403846 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14180,"prepare.queue_time_us":232817,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":77,"raft.queue_time_us":10439,"spinlock_wait_cycles":5819008,"thread_start_us":278781,"threads_started":2}
I20260709 12:25:32.496907  7921 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:32.498533  7961 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.204s	user 0.003s	sys 0.000s
W20260709 12:25:32.499460  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 70554.2us
W20260709 12:25:32.500039  7944 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.446s	user 0.006s	sys 0.000s
I20260709 12:25:32.524386  7921 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:32.469990  7900 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:32.525722  6019 raft_consensus.cc:2368] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5.
I20260709 12:25:32.483620  7588 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:32.526820  7588 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:32.504959  7954 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.206s	user 0.001s	sys 0.001s
I20260709 12:25:32.529879  7773 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:32.530470  7773 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:32.530628  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 62091.8us
I20260709 12:25:32.505880  7875 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:32.526868  7900 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:32.532747  7840 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:32.534866  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:32.535104  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 107277us
I20260709 12:25:32.509711  6022 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:32.505908  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:32.513616  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 48646.3us
I20260709 12:25:32.517537  7871 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:32.517689  6015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 333) took 1479 ms (client timeout 1786 ms). Trace:
I20260709 12:25:32.511281  7745 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:32.521910  7951 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.225s	user 0.001s	sys 0.001s
I20260709 12:25:32.530912  7773 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:32.532307  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:25:32.546379  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:32.550487  7952 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.233s	user 0.005s	sys 0.000s
W20260709 12:25:32.540987  6015 rpcz_store.cc:269] 0709 12:25:31.038068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.103230 (+ 65162us) service_pool.cc:224] Handling call
0709 12:25:32.517339 (+1414109us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b5400363dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:32.517557 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2290982272}
I20260709 12:25:32.556137  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:32.561738  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:32.562515  6022 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:32.573875  6793 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:32.547842  7880 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:32.501287  7810 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:32.569931  7750 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:32.583146  7750 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.583536  7766 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:32.584182  7766 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:25:32.584231  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 52143us
I20260709 12:25:32.584600  7766 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:32.583560  7750 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:32.587869  7810 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:32.595749  7942 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.546s	user 0.003s	sys 0.000s
I20260709 12:25:32.588778  7867 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:32.593891  6762 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.010s	sys 0.027s
W20260709 12:25:32.596515  6762 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.010s	sys 0.028s
I20260709 12:25:32.596570  5865 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:32.594765  7775 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:32.564652  7900 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:25:32.595383  6793 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:32.540606  7840 consensus_queue.cc:1059] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:25:32.621744  5865 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:32.624963  5873 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:25:32.625363  5873 pending_rounds.cc:85] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 3
I20260709 12:25:32.625613  5873 pending_rounds.cc:107] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:25:32.620132  7810 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:32.626102  5873 raft_consensus.cc:2889] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:32.627063  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:32.628719  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:32.630429  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:32.631285  5874 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:32.626044  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 51368.9us
W20260709 12:25:32.635761  7957 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.340s	user 0.000s	sys 0.004s
I20260709 12:25:32.636482  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:32.638788  7840 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:32.640605  7607 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:32.641038  7607 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:32.642040  7607 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:32.643348  7607 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:32.644330  7880 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:32.617180  7799 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:32.645876  5870 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:32.580150  7728 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 12:25:32.646246  6609 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:32.646886  7728 raft_consensus.cc:3055] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:32.647289  7728 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:32.647359  7834 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:32.648394  7728 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:32.649411  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 334) took 1611 ms. Trace:
I20260709 12:25:32.649904  6021 rpcz_store.cc:276] 0709 12:25:31.038292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.916658 (+878366us) service_pool.cc:224] Handling call
0709 12:25:31.917184 (+   526us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:31.945928 (+ 28744us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.945972 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:31.946872 (+   900us) log.cc:844] Serialized 52 byte log entry
0709 12:25:32.135766 (+188894us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:32.148306 (+ 12540us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.148582 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.148687 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.458956 (+310269us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:32.648982 (+190026us) raft_consensus.cc:1685] finished
0709 12:25:32.649140 (+   158us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.649325 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9109,"prepare.queue_time_us":20172,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":37510,"raft.run_cpu_time_us":417,"raft.run_wall_time_us":3598,"spinlock_wait_cycles":36518784,"thread_start_us":188355,"threads_started":1,"wal-append.queue_time_us":348908}
I20260709 12:25:32.654047  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:32.658162  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:32.659024  7893 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:32.621428  7949 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.245s	user 0.005s	sys 0.001s
I20260709 12:25:32.659794  6778 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:32.661191  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51749.9us
I20260709 12:25:32.661278  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:32.662873  6609 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:32.668038  6023 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:32.670804  5868 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:32.677882  7728 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:32.668671  7750 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:32.607139  7862 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:32.626173  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:32.688005  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.688225  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.685237  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:32.700201  7867 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:32.701062  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.703259  5947 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:32.701467  5943 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:32.702245  7711 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:32.704530  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.705729  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:32.705978  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 57822.5us
I20260709 12:25:32.707244  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.682175  7592 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:32.708845  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.710625  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.714663  6021 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:32.704900  7711 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:32.717549  7711 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:32.695647  7773 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:32.721011  7773 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:32.721909  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.723354  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:32.724274  5913 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:32.725706  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.728826  5943 raft_consensus.cc:2368] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6.
I20260709 12:25:32.729789  5948 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:32.729348  5868 raft_consensus.cc:2368] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:25:32.738356  6024 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:32.740015  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:32.756840  6778 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:32.755312  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:32.758044  7588 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.758349  5873 raft_consensus.cc:2410] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:32.758718  7588 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:32.759395  7875 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:32.761148  7588 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:32.750350  7867 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:25:32.763228  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.763769  7875 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:32.764875  7414 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:32.766198  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:32.796906  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.771952  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:32.772863  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:32.739630  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.813891  5941 raft_consensus.cc:2393] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:25:32.781456  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 48569.5us
I20260709 12:25:32.795565  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:32.740450  7840 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:32.770769  7592 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:32.777009  5874 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
W20260709 12:25:32.803481  7707 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.732s	user 0.016s	sys 0.044s
I20260709 12:25:32.804445  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:32.834438  5868 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 1.
I20260709 12:25:32.810454  6024 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: [5.4-6.5]
I20260709 12:25:32.811090  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:32.811791  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:32.817458  5872 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:32.815356  7766 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:32.840408  7766 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:32.834236  7840 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.842130  7840 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:32.841763  7939 consensus_queue.cc:1059] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:32.842433  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.845772  7754 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:32.846182  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.846406  5868 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:32.847661  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:32.848428  5941 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:32.850103  7875 raft_consensus.cc:3055] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:32.850639  7875 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:32.850764  5942 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:32.851740  7875 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:32.853155  7875 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:32.853695  7707 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.006s	user 0.016s	sys 0.044s
I20260709 12:25:32.854049  7707 raft_consensus.cc:697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:32.855135  7707 consensus_queue.cc:237] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:32.845837  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:32.857079  6016 raft_consensus.cc:2410] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:25:32.866394  5943 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:32.866673  7750 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:32.866899  5946 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:32.799132  7834 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:25:32.869062  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.849841  7836 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:32.731513  7862 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:32.878767  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 60455.6us
I20260709 12:25:32.845959  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:32.852406  5838 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:32.849318  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:32.847662  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:32.883136  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.884055  5940 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:32.886314  5870 raft_consensus.cc:2393] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 6: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:32.880167  6021 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:32.890750  5947 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:32.883232  5868 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:32.901050  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:32.901858  5874 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:32.907373  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:32.908437  5942 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:32.915901  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:32.877040  7414 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.913442  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:32.933188  6024 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 5: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:32.934707  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:32.939953  5867 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:32.947417  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:32.920518  7811 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:32.949448  5987 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:32.953600  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:32.960418  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:32.968212  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 116046us
W20260709 12:25:32.973523  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 512363us
I20260709 12:25:32.976263  7896 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:32.977078  7896 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:32.977646  7896 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:32.980692  5838 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:32.981809  7909 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 12:25:32.982128  7592 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:32.987509  7592 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:32.988562  7896 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:33.129987  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:33.173899  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 182072us
I20260709 12:25:32.952200  7745 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:32.991078  7904 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.906755  7436 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
W20260709 12:25:33.058316  6785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.010s	sys 0.031s
W20260709 12:25:33.185891  6785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.012s	sys 0.032s
I20260709 12:25:32.966234  7867 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:32.992074  7968 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.951292  7414 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:32.990803  7955 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:32.991385  7588 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:32.920353  7818 consensus_queue.cc:1059] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 12:25:33.180821  7871 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:33.176203  6024 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:25:33.182384  7592 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.189532  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:33.188239  7867 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:33.181365  7607 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.531s	user 0.003s	sys 0.022s
I20260709 12:25:33.201536  6024 pending_rounds.cc:85] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 2
I20260709 12:25:33.175333  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:33.194799  7899 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.010s	sys 0.016s
I20260709 12:25:33.181273  7898 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:33.228858  6024 pending_rounds.cc:107] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:25:33.237699  6024 raft_consensus.cc:2889] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:33.229058  5872 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:33.173813  5874 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:33.186869  7900 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:33.231972  6020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.012s	sys 0.017s
W20260709 12:25:33.235715  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 255091us
W20260709 12:25:33.194962  7880 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.011s	sys 0.016s
W20260709 12:25:33.239065  6020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.012s	sys 0.018s
I20260709 12:25:33.186333  5866 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:33.190797  7811 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:33.245114  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:33.246445  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:33.252727  6611 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.005s	sys 0.014s
W20260709 12:25:33.253989  6611 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.005s	sys 0.014s
W20260709 12:25:33.254766  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 570895us
I20260709 12:25:33.225265  7871 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:33.194417  7499 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
W20260709 12:25:33.239030  7899 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.010s	sys 0.017s
I20260709 12:25:33.231304  7867 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:33.189617  7904 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:33.263140  7799 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:33.263839  7840 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:33.266404  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.266674  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:33.282646  6015 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
W20260709 12:25:33.268379  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 78947.7us
W20260709 12:25:33.244526  7880 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.011s	sys 0.017s
I20260709 12:25:33.253432  7597 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:33.229708  7592 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:33.192751  7968 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:33.295367  5874 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:33.268906  7871 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:33.296291  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:33.304628  5840 leader_election.cc:400] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4.
W20260709 12:25:33.305155  7971 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.089s	user 0.001s	sys 0.001s
W20260709 12:25:33.278095  7711 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.003s	sys 0.014s
W20260709 12:25:33.255762  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.249770  7811 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:33.257210  7900 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:33.323846  5942 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:25:33.329360  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 89835.3us
W20260709 12:25:33.330180  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 68374.6us
I20260709 12:25:33.308512  7862 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:33.338495  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:33.324013  7711 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.004s	sys 0.015s
I20260709 12:25:33.306339  5941 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:33.327071  7607 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:33.344529  5872 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:33.325135  7781 raft_consensus.cc:3055] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:33.346928  5838 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:33.327042  5865 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.017s	sys 0.030s
W20260709 12:25:33.351105  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 401719us
I20260709 12:25:33.353165  6024 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:33.368175  6611 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:33.377601  7974 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:33.363814  7781 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:33.381799  7900 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:33.382051  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:33.382805  6015 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:33.383054  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 78618.2us
I20260709 12:25:33.384071  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:33.386034  7976 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:25:33.386431  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:33.388243  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 344) took 1353 ms. Trace:
W20260709 12:25:33.366168  5865 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.018s	sys 0.031s
I20260709 12:25:33.388880  6017 rpcz_store.cc:276] 0709 12:25:32.035001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.396045 (+361044us) service_pool.cc:224] Handling call
0709 12:25:32.424502 (+ 28457us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:32.562719 (+138217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.562757 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:32.575284 (+ 12527us) log.cc:844] Serialized 52 byte log entry
0709 12:25:32.921238 (+345954us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:32.952240 (+ 31002us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.952520 (+   280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.952776 (+   256us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.259761 (+306985us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b480050d258. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d7bacaa1 00007f39d7ba3b37 00007f39d7ba39ca 00007f39d7c9b26f 00007f39d7c9af37 00007f39d7caf55c 00007f39d7caf508 00007f39d7caf488 00007f39d7caf443 00007f39d7cae6ec 00007f39d6bd8a96
0709 12:25:33.387788 (+128027us) raft_consensus.cc:1685] finished
0709 12:25:33.387960 (+   172us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.388158 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":181874,"prepare.queue_time_us":68254,"raft.queue_time_us":5535,"raft.run_cpu_time_us":13156,"raft.run_wall_time_us":715692,"spinlock_wait_cycles":10557568,"thread_start_us":301381,"threads_started":1,"wal-append.queue_time_us":641677}
W20260709 12:25:33.393836  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 54671us
I20260709 12:25:33.394101  7898 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:33.394922  7811 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:33.395109  7898 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:33.395516  7898 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:33.360706  7750 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:33.395846  5838 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:33.397405  7414 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:33.397920  5838 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:33.398650  5874 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 5 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:33.399431  5838 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:33.400195  5874 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:33.400954  5838 leader_election.cc:400] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6.
I20260709 12:25:33.404206  7745 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:33.396167  7750 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.374150  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:33.412843  6015 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:33.360016  7896 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:33.405300  7745 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:33.460964  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 631) took 1102 ms. Trace:
W20260709 12:25:33.469879  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 85975.8us
I20260709 12:25:33.355911  7818 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 12:25:33.473790  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:33.475220  6024 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:33.478209  5838 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:33.483848  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:33.395900  7898 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:33.485827  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.503311  6015 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:33.415405  7750 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:33.424201  7799 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:33.415508  7974 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:33.498092  7707 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.510165  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 346) took 1154 ms. Trace:
I20260709 12:25:33.467828  6766 rpcz_store.cc:276] 0709 12:25:32.359015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.366521 (+  7506us) service_pool.cc:224] Handling call
0709 12:25:32.378405 (+ 11884us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.496256 (+117851us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.496297 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.496915 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.742471 (+245556us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b1800073660. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:25:33.198134 (+455663us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:33.250317 (+ 52183us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.250739 (+   422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.250902 (+   163us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.406447 (+155545us) raft_consensus.cc:1685] finished
0709 12:25:33.460597 (+ 54150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.460852 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33560,"prepare.queue_time_us":51704,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":71180,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":14263,"spinlock_wait_cycles":26059648,"thread_start_us":449952,"threads_started":1,"wal-append.queue_time_us":812671}
I20260709 12:25:33.512804  7728 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:33.482127  7754 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:33.540884  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:33.523126  7840 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:33.462826  7414 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:33.528589  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 160846us
I20260709 12:25:33.499571  7896 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:25:33.549278  6024 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 5: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:33.407269  7862 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
I20260709 12:25:33.521708  7900 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.472013  7607 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 12:25:33.526772  7592 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:33.532421  6020 rpcz_store.cc:276] 0709 12:25:32.356192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.509414 (+153222us) service_pool.cc:224] Handling call
0709 12:25:32.538829 (+ 29415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.564461 (+ 25632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.564494 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.571952 (+  7458us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.700871 (+128919us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b180020f2c0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:25:33.244778 (+543907us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:33.244818 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.244909 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.509721 (+264812us) raft_consensus.cc:1685] finished
0709 12:25:33.509752 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.510064 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6817,"raft.queue_time_us":12274,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":6668,"spinlock_wait_cycles":5000960,"thread_start_us":543688,"threads_started":1,"wal-append.queue_time_us":807232}
I20260709 12:25:33.546825  7414 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.551301  7811 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:33.545288  7707 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:33.466281  7781 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:33.606734  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 415140us
I20260709 12:25:33.561985  7799 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:25:33.600361  7975 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.275s	user 0.000s	sys 0.002s
I20260709 12:25:33.613368  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 611) took 2970 ms. Trace:
I20260709 12:25:33.555208  7738 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
W20260709 12:25:33.627388  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 54973.4us
I20260709 12:25:33.602802  7414 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:33.569192  7900 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:33.639030  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 215168us
I20260709 12:25:33.640060  5838 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:33.637754  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 166717us
I20260709 12:25:33.591910  7840 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:25:33.620455  6762 rpcz_store.cc:276] 0709 12:25:30.635509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.952833 (+317324us) service_pool.cc:224] Handling call
0709 12:25:32.012028 (+1059195us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.012913 (+   885us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.012950 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.013597 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.597178 (+583581us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:32.597557 (+   379us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.598350 (+   793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.598448 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.763271 (+164823us) raft_consensus.cc:1685] finished
0709 12:25:32.763294 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.605515 (+842221us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400005350. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:33.605712 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":449,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":2178,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":50211,"spinlock_wait_cycles":2434759040,"thread_start_us":582988,"threads_started":1,"wal-append.queue_time_us":739824}
W20260709 12:25:33.663465  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:33.676632  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 58899.1us
I20260709 12:25:33.685233  6024 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:25:33.641215  7711 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.642554  7904 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:33.698912  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 58953.8us
I20260709 12:25:33.691267  6024 pending_rounds.cc:85] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 3
I20260709 12:25:33.709395  6024 pending_rounds.cc:107] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:25:33.709832  6024 raft_consensus.cc:2889] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:33.665964  7781 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:33.718566  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 352) took 1036 ms. Trace:
I20260709 12:25:33.666945  7728 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:33.724196  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:33.722801  6022 rpcz_store.cc:276] 0709 12:25:32.682124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.687733 (+  5609us) service_pool.cc:224] Handling call
0709 12:25:32.688190 (+   457us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:32.688770 (+   580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.688798 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:32.741049 (+ 52251us) log.cc:844] Serialized 52 byte log entry
0709 12:25:33.282778 (+541729us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:33.676367 (+393589us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.682136 (+  5769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.718182 (+ 36046us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.718257 (+    75us) raft_consensus.cc:1685] finished
0709 12:25:33.718270 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.718487 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4774,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":11548,"thread_start_us":819754,"threads_started":2,"wal-append.queue_time_us":603932}
I20260709 12:25:33.731935  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 731) took 1026 ms. Trace:
W20260709 12:25:33.737231  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.683920  7968 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:33.722401  7972 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.411s	user 0.005s	sys 0.000s
I20260709 12:25:33.705442  7970 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.767881  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:33.764654  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 636) took 1354 ms. Trace:
I20260709 12:25:33.750507  5948 rpcz_store.cc:276] 0709 12:25:32.685808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.689775 (+  3967us) service_pool.cc:224] Handling call
0709 12:25:33.711908 (+1022133us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b54003089d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:33.712171 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40218880}
W20260709 12:25:33.731003  7991 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 12:25:33.770953  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 104631us
I20260709 12:25:33.744855  7896 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.773731  7970 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:33.779788  7985 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:25:33.780138  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:33.780913  7880 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.782056  7810 raft_consensus.cc:3055] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:25:33.783206  7875 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.912s	user 0.016s	sys 0.002s
I20260709 12:25:33.744740  7607 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:25:33.789146  5945 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:33.730963  8006 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:33.791069  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:33.795989  7707 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:33.768896  6785 rpcz_store.cc:276] 0709 12:25:32.410423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.437943 (+ 27520us) service_pool.cc:224] Handling call
0709 12:25:32.438380 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.497917 (+ 59537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.497953 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.498547 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.186771 (+688224us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:33.186833 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.186922 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.764251 (+577329us) raft_consensus.cc:1685] finished
0709 12:25:33.764284 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.764550 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45832,"raft.queue_time_us":47317,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"thread_start_us":687769,"threads_started":1,"wal-append.queue_time_us":760141}
I20260709 12:25:33.803194  7996 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:25:33.806530  8005 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:25:33.760317  7904 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:33.811357  7994 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.750926  7711 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:33.770586  7799 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:33.814144  8009 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:33.818287  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 139602us
I20260709 12:25:33.861254  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1001) took 1153 ms. Trace:
W20260709 12:25:33.810573  7993 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:25:33.760874  7973 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:33.865772  5949 raft_consensus.cc:2368] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4.
I20260709 12:25:33.811522  5840 leader_election.cc:400] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5.
I20260709 12:25:33.879601  5944 rpcz_store.cc:276] 0709 12:25:32.707911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.710035 (+  2124us) service_pool.cc:224] Handling call
0709 12:25:33.779878 (+1069843us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b5800030398. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:33.861118 (+ 81240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2591104}
I20260709 12:25:33.940868  5840 leader_election.cc:403] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:25:33.877995  7994 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:33.851878  7896 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:33.885608  7989 log.cc:927] Time spent T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 12:25:33.889987  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:33.871826  7904 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:33.862318  7996 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:33.866658  7880 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:33.918715  7921 raft_consensus.cc:3055] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:33.919353  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 273) took 1145 ms. Trace:
I20260709 12:25:33.863343  7810 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:33.920095  5694 catalog_manager.cc:5697] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 4 to 5, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 5 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:31.069380  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:33.922966  7970 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:33.923759  6018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 327) took 3438 ms (client timeout 2150 ms). Trace:
I20260709 12:25:33.924331  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 347) took 1567 ms. Trace:
I20260709 12:25:33.972575  6023 rpcz_store.cc:276] 0709 12:25:32.356668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.522865 (+166197us) service_pool.cc:224] Handling call
0709 12:25:32.523342 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.670987 (+147645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.888202 (+1217215us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400641754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:33.924263 (+ 36061us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332310912}
I20260709 12:25:33.851397  7436 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.933224  7707 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:33.861457  7968 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:25:33.884140  7728 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:33.943169  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 271) took 1118 ms. Trace:
W20260709 12:25:33.971971  6018 rpcz_store.cc:269] 0709 12:25:30.484899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.620179 (+135280us) service_pool.cc:224] Handling call
0709 12:25:33.887816 (+3267637us) spinlock_profiling.cc:243] Waited 2.78 s on lock 0x7b54006b8fd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:33.923517 (+ 35701us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6396269312}
W20260709 12:25:33.951985  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 72339.1us
I20260709 12:25:33.959522  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 998) took 1252 ms. Trace:
W20260709 12:25:33.960340  6016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.010s	sys 0.013s
W20260709 12:25:33.978739  6016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.010s	sys 0.013s
I20260709 12:25:33.961061  5945 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 7: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:25:33.962033  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:33.980234  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:33.980791  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.962697  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:33.925838  7900 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:33.969820  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 46746.4us
I20260709 12:25:33.969870  7588 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:33.975520  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:33.976121  5943 rpcz_store.cc:276] 0709 12:25:32.824686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.867518 (+ 42832us) service_pool.cc:224] Handling call
0709 12:25:32.867845 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.868058 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:33.350688 (+482630us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.942674 (+591986us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:33.942757 (+    83us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:33.942830 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.942929 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.943087 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10918,"prepare.queue_time_us":678939,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":8539,"raft.queue_time_us":168280,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":141,"thread_start_us":962457,"threads_started":3,"wal-append.queue_time_us":420213}
I20260709 12:25:33.978119  5949 rpcz_store.cc:276] 0709 12:25:32.706929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.709765 (+  2836us) service_pool.cc:224] Handling call
0709 12:25:33.790862 (+1081097us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b5800030398. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:33.959447 (+168585us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2552192}
I20260709 12:25:33.965996  7921 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:33.985796  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:33.991698  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:33.959705  7750 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:34.000309  8013 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:34.001770  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 190433us
I20260709 12:25:33.980149  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:33.994352  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:34.004724  5870 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:33.997987  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:34.000918  8013 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:33.987299  7970 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.006922  7955 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:34.010839  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:33.975266  7499 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:25:34.014612  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:34.014920  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:33.985522  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 633) took 1597 ms. Trace:
W20260709 12:25:34.016254  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 232868us
I20260709 12:25:33.989672  7607 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:34.018762  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:33.977655  7436 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:34.017026  7867 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.719s	user 0.016s	sys 0.000s
I20260709 12:25:34.014250  7754 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:34.041899  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 123927us
I20260709 12:25:33.984401  7707 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:34.043545  7840 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.043944  7904 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:34.044149  8007 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:34.044647  7880 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:34.044934  7811 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.045763  7974 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:34.046010  7592 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:25:34.048040  7745 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.583s	user 0.013s	sys 0.000s
I20260709 12:25:34.026897  7810 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:34.038661  7995 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:34.050073  7738 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:33.997979  7996 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:34.109804  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:34.110471  8016 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:34.111387  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:34.099483  7592 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:34.112725  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:34.095950  7811 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:34.113962  8011 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.114789  8011 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:34.115612  7810 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:34.051172  7871 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.726s	user 0.017s	sys 0.002s
I20260709 12:25:34.052170  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:34.053794  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:34.042790  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:34.056653  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:34.123488  5874 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:34.018153  7955 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:34.061472  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:34.007329  7970 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:34.037299  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:34.061923  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:34.058856  7754 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.100447  7896 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:34.100957  7499 consensus_queue.cc:1059] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:34.102018  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:34.103176  8015 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 12:25:34.093431  8007 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:33.967801  5865 rpcz_store.cc:276] 0709 12:25:32.773389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.807262 (+ 33873us) service_pool.cc:224] Handling call
0709 12:25:32.807656 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.807980 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.808006 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.808656 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.389354 (+580698us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:33.669837 (+280483us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.745854 (+ 76017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.757917 (+ 12063us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.919071 (+161154us) raft_consensus.cc:1685] finished
0709 12:25:33.919091 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.919247 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47033,"prepare.queue_time_us":246354,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":17807,"raft.queue_time_us":62178,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":13743,"thread_start_us":822977,"threads_started":2,"wal-append.queue_time_us":741018}
I20260709 12:25:34.040778  7607 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:34.106210  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:34.169625  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:34.123662  5946 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:34.126258  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:34.208722  5868 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:34.208810  5872 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:34.058743  7818 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:25:34.093123  7904 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:34.115708  7811 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:34.130312  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:34.061669  7862 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:25:34.058076  7867 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:34.091115  7992 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.109s
I20260709 12:25:34.060815  7840 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:34.224031  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:34.163614  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:34.166504  7754 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:34.229796  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:34.205150  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:34.106716  7745 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:34.056304  5838 leader_election.cc:336] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Timed out: RequestConsensusVote RPC to 127.2.4.195:43329 timed out after 2.192s (SENT)
I20260709 12:25:34.117456  8011 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:34.235260  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:34.211355  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:34.211364  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:34.154001  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.126955  8016 consensus_queue.cc:1243] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:25:34.243997  8025 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:34.244905  8025 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:34.246405  7840 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:34.246877  8014 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:34.247141  7840 raft_consensus.cc:3055] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:34.215590  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:34.060990  7921 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:34.217499  6785 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:34.217958  6018 raft_consensus.cc:2410] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:34.219405  6766 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:34.129952  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:34.164795  7939 raft_consensus.cc:3055] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:34.205374  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:34.128672  7871 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:34.164103  7414 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:34.259953  6015 raft_consensus.cc:2410] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 12:25:34.227712  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:34.227730  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 133231us
I20260709 12:25:34.228432  6020 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 8: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:34.233101  5838 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:34.233820  6793 raft_consensus.cc:2410] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:34.240123  7994 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:34.253136  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 145871us
I20260709 12:25:34.206710  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:34.257660  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:34.259123  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:34.226024  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:34.262231  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:34.263294  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:34.264608  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:34.255841  6023 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:34.269615  6793 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:34.268786  5868 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:34.095053  7974 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:34.271824  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:34.117929  7995 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:34.274310  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:34.274621  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:34.247525  7607 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:34.250670  7711 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:34.247531  7840 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:34.254953  7939 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:25:34.278946  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.279337  6022 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:25:34.279767  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:34.220834  7904 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:34.289100  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:34.290881  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:34.277513  6775 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:34.301293  5868 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
W20260709 12:25:34.303407  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.303823  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:34.308109  5945 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:34.275938  7994 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:34.315249  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:34.293459  7745 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:34.320221  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:34.289026  7750 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:34.289781  7592 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:34.247493  8014 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:34.322038  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:34.357156  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.349409  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:34.359130  5946 raft_consensus.cc:2410] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:25:34.330524  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:34.361943  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:34.363485  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 307200us
I20260709 12:25:34.364339  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:34.368476  5838 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:34.373574  6766 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 7: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:34.312387  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:34.379324  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:34.377748  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 53258.9us
I20260709 12:25:34.379302  5838 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:34.374958  7607 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:25:34.385205  8028 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:25:34.385365  7607 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:34.316769  8025 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:34.387310  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:34.388105  7607 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:34.312029  8007 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:34.321646  8006 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:34.020303  6757 rpcz_store.cc:276] 0709 12:25:32.382277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.399871 (+ 17594us) service_pool.cc:224] Handling call
0709 12:25:32.457975 (+ 58104us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.458425 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.458474 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.459105 (+   631us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.929294 (+470189us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:32.987192 (+ 57898us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.265318 (+278126us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b18001691e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287 00007f39d2c9e208
0709 12:25:33.485742 (+220424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.530033 (+ 44291us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.549164 (+ 19131us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:33.970256 (+421092us) raft_consensus.cc:1685] finished
0709 12:25:33.972428 (+  2172us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.979674 (+  7246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":166759,"prepare.queue_time_us":39156,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":74,"raft.queue_time_us":196894,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":4773,"spinlock_wait_cycles":28525696,"thread_start_us":659002,"threads_started":2,"wal-append.queue_time_us":779320}
I20260709 12:25:34.393718  6793 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:25:34.321708  7889 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:34.415220  6773 raft_consensus.cc:2410] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:34.415485  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:34.417860  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:34.423058  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:34.310575  7818 consensus_queue.cc:1059] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
I20260709 12:25:34.427264  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:34.376613  7436 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:34.376958  7707 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:34.435331  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:34.431697  8029 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:25:34.432380  8032 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:34.389683  7968 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:34.438251  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:34.438623  8031 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:25:34.391800  8026 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:34.439123  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:34.439220  8037 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:34.439996  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:34.441061  6768 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:34.441170  8014 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:34.441720  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:34.442035  8034 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:25:34.442232  8019 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 12:25:34.447994  7970 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:34.297082  7955 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:34.445420  8035 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.445807  7889 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:34.446826  7904 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:34.443562  7974 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:34.452793  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:34.447311  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:34.411255  8006 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:34.324683  7994 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:34.487282  6793 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:34.492866  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:34.431700  7745 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:25:34.454264  7781 consensus_meta.cc:287] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.723s	user 0.009s	sys 0.018s
I20260709 12:25:34.498076  6773 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:34.501065  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 121840us
I20260709 12:25:34.676558  5838 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:34.453689  7867 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:34.466202  7818 consensus_queue.cc:1243] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:34.443200  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:34.504155  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 152584us
I20260709 12:25:34.466733  7707 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:34.508940  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:34.515883  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:34.715291  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:34.516292  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:34.502964  7898 consensus_meta.cc:287] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.956s	user 0.013s	sys 0.007s
I20260709 12:25:34.379385  7939 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:34.635754  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 183071us
I20260709 12:25:34.733183  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:34.725163  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 48680.9us
I20260709 12:25:34.611598  8037 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.749576  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 355) took 1119 ms. Trace:
I20260709 12:25:34.824393  6024 rpcz_store.cc:276] 0709 12:25:33.629998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.636857 (+  6859us) service_pool.cc:224] Handling call
0709 12:25:33.637278 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.710411 (+ 73133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.749116 (+1038705us) spinlock_profiling.cc:243] Waited 43.5 ms on lock 0x7b54003642d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:34.749481 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100110080}
I20260709 12:25:34.752427  7773 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:34.754014  6793 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:34.765198  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:34.775638  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 262705us
I20260709 12:25:34.638485  8006 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:34.818313  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:34.848500  6023 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:34.851286  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 86583.4us
I20260709 12:25:34.891225  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:34.887509  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:34.847790  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 132635us
I20260709 12:25:34.676779  7414 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:34.894232  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:34.894932  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:34.913993  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.914317  6766 raft_consensus.cc:2393] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:25:34.850576  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 114815us
I20260709 12:25:34.818373  7867 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:34.702646  5865 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:34.906418  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:34.713171  7974 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:34.907856  6019 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:34.928128  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:34.480054  8032 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:34.849543  7773 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:34.745398  7968 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:34.929235  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:34.935029  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:34.847764  7818 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:34.988786  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:34.716939  7745 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:34.993813  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:34.992451  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:34.922267  7499 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:34.932861  8024 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 12:25:35.026363  5947 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.044s	sys 0.100s
W20260709 12:25:35.036638  5947 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.045s	sys 0.101s
I20260709 12:25:34.506323  8034 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:35.030848  6762 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:34.742971  7898 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:35.039079  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:35.040401  7728 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:35.030246  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 846) took 1036 ms. Trace:
I20260709 12:25:34.988575  7904 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.677953  8035 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:35.066201  8035 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:35.041246  7499 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:35.067310  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:35.043057  6773 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:34.918417  8037 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:35.045512  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:35.077481  6766 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:25:35.053300  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 610207us
I20260709 12:25:35.054414  6019 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:35.054046  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 65403.1us
I20260709 12:25:35.034461  5867 raft_consensus.cc:2376] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4. Re-sending same reply.
W20260709 12:25:35.090091  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:35.095739  5870 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.736s	user 0.006s	sys 0.006s
I20260709 12:25:35.096518  5870 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:35.062739  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 273) took 1279 ms. Trace:
I20260709 12:25:35.115298  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:35.130205  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 843) took 1118 ms. Trace:
W20260709 12:25:35.054185  8043 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 12:25:35.194411  5873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 263) took 2474 ms (client timeout 2093 ms). Trace:
I20260709 12:25:35.139978  5870 rpcz_store.cc:276] 0709 12:25:33.979377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.979530 (+   153us) service_pool.cc:224] Handling call
0709 12:25:35.097276 (+1117746us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:35.106195  5948 rpcz_store.cc:276] 0709 12:25:33.783223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.857524 (+ 74301us) service_pool.cc:224] Handling call
0709 12:25:33.870542 (+ 13018us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.968318 (+ 97776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.968373 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.977168 (+  8795us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.125589 (+148421us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:34.125655 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.125837 (+   182us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.062484 (+936647us) raft_consensus.cc:1685] finished
0709 12:25:35.062512 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.062669 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71787,"raft.queue_time_us":89487,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":4258,"thread_start_us":110106,"threads_started":1,"wal-append.queue_time_us":284986}
I20260709 12:25:35.053164  8030 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:35.232324  6024 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:35.226169  5873 rpcz_store.cc:269] 0709 12:25:32.653556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.653717 (+   161us) service_pool.cc:224] Handling call
0709 12:25:35.112188 (+2458471us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54004a0454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:35.128024 (+ 15836us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237384960}
I20260709 12:25:35.239573  6609 leader_election.cc:400] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:25:35.240159  6609 leader_election.cc:403] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:25:35.107623  7811 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:35.065898  7745 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:35.054276  7974 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:35.089123  7414 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.058115  8040 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:35.107172  7995 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:35.113572  7728 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.052981  8036 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.425s
I20260709 12:25:35.251427  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:35.341976  5870 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:35.063908  7904 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:35.224689  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:35.114504  7436 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.174s
I20260709 12:25:35.326188  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:35.352816  5867 raft_consensus.cc:2376] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6. Re-sending same reply.
I20260709 12:25:35.399900  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 284) took 1034 ms. Trace:
W20260709 12:25:35.448359  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 332808us
I20260709 12:25:35.353827  5873 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:35.456107  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 708) took 1107 ms. Trace:
I20260709 12:25:35.456579  5873 rpcz_store.cc:276] 0709 12:25:34.348929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.249565 (+900636us) service_pool.cc:224] Handling call
0709 12:25:35.456016 (+206451us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:35.340446  5865 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:35.400429  5867 rpcz_store.cc:276] 0709 12:25:34.365293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.083477 (+718184us) service_pool.cc:224] Handling call
0709 12:25:35.399792 (+316315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:35.053025  7773 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:35.471652  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 697) took 1423 ms. Trace:
I20260709 12:25:35.450186  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:35.293067  8041 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:25:35.479717  6775 consensus_meta.cc:287] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.170s	user 0.013s	sys 0.001s
I20260709 12:25:35.483646  5870 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:35.486711  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 282) took 1122 ms. Trace:
I20260709 12:25:35.472498  5874 rpcz_store.cc:276] 0709 12:25:34.048358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.289269 (+240911us) service_pool.cc:224] Handling call
0709 12:25:34.297251 (+  7982us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.314870 (+ 17619us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.314913 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.378212 (+ 63299us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.480990 (+102778us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:34.481037 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.481115 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.471303 (+990188us) raft_consensus.cc:1685] finished
0709 12:25:35.471337 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.471561 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11430,"raft.queue_time_us":5710,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":6774,"thread_start_us":91255,"threads_started":1,"wal-append.queue_time_us":285433}
W20260709 12:25:35.294588  7588 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.304s	user 0.013s	sys 0.001s
I20260709 12:25:35.487901  5870 rpcz_store.cc:276] 0709 12:25:34.364689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.426664 (+1061975us) service_pool.cc:224] Handling call
0709 12:25:35.483418 (+ 56754us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.486249 (+  2831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.486608 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:35.522763  6775 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:35.458290  7974 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.526911  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 544) took 1271 ms. Trace:
I20260709 12:25:35.326507  7414 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:35.529541  6775 rpcz_store.cc:276] 0709 12:25:34.255199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.260487 (+  5288us) service_pool.cc:224] Handling call
0709 12:25:35.526828 (+1266341us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:35.328688  8040 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:35.489758  8041 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:35.545550  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 90565.7us
I20260709 12:25:35.540752  7773 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:35.552723  8006 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.636s	user 0.009s	sys 0.000s
I20260709 12:25:35.558681  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:35.459591  7728 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:35.527377  8030 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:35.531621  7588 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5.
W20260709 12:25:35.559837  5840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.011s	sys 0.016s
W20260709 12:25:35.572754  5840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.012s	sys 0.016s
W20260709 12:25:35.573683  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 679537us
W20260709 12:25:35.582190  5990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.023s	sys 0.024s
W20260709 12:25:35.587054  5990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.023s	sys 0.024s
W20260709 12:25:35.577615  8007 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.112s	user 0.010s	sys 0.068s
I20260709 12:25:35.535353  7974 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:35.566641  7818 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:35.560048  8041 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:35.567638  8047 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:25:35.569826  8053 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:35.591858  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:35.594980  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:35.601974  7775 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:35.602731  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:35.602918  7775 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:35.604015  5940 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:35.594967  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:35.606021  5948 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:35.606622  7775 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:35.580896  8056 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:35.610091  8056 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:35.586954  5987 connection.cc:256] RPC call timeout handler was delayed by -1.1774541499999032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1938s for a response
I20260709 12:25:35.613523  8056 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:35.584878  7992 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:35.591822  8007 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.127s	user 0.010s	sys 0.069s
I20260709 12:25:35.579669  7889 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:35.557581  8006 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:35.643887  6762 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.004s	sys 0.049s
W20260709 12:25:35.649893  6762 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.004s	sys 0.050s
W20260709 12:25:35.647845  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.016s	sys 0.029s
W20260709 12:25:35.651837  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.016s	sys 0.030s
W20260709 12:25:35.659171  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 543906us
I20260709 12:25:35.639415  8053 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.600277  7995 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:35.599890  7898 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:35.673367  5940 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:35.063436  5866 rpcz_store.cc:276] 0709 12:25:33.994083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.998193 (+  4110us) service_pool.cc:224] Handling call
0709 12:25:34.047570 (+ 49377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.048020 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.048065 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.048659 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.163639 (+114980us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:34.165739 (+  2100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.211603 (+ 45864us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.928802 (+717199us) raft_consensus.cc:1685] finished
0709 12:25:34.933380 (+  4578us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.025964 (+ 92584us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8238,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":20928,"thread_start_us":71074,"threads_started":1,"wal-append.queue_time_us":171370}
I20260709 12:25:35.669096  5948 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:35.649827  7588 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.649348  8032 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:35.638109  7818 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:35.686398  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:35.692555  5940 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:35.677856  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:35.694756  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 119036us
I20260709 12:25:35.697415  5840 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:35.700459  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 278) took 1595 ms. Trace:
I20260709 12:25:35.701099  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:35.702020  6023 consensus_meta.cc:287] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.812s	user 0.012s	sys 0.002s
I20260709 12:25:35.703101  6023 raft_consensus.cc:2410] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:25:35.701011  5947 rpcz_store.cc:276] 0709 12:25:34.104704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.317419 (+212715us) service_pool.cc:224] Handling call
0709 12:25:34.360335 (+ 42916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.360574 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:35.048626 (+688052us) log.cc:844] Serialized 14 byte log entry
0709 12:25:35.691525 (+642899us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:35.691600 (+    75us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:35.691670 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.691820 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.700382 (+  8562us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13907,"prepare.queue_time_us":67740,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":4558,"thread_start_us":1286419,"threads_started":3,"wal-append.queue_time_us":539766}
I20260709 12:25:35.674759  8053 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:35.705564  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 506409us
I20260709 12:25:35.689260  7588 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:35.711108  7818 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:35.699796  7436 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:35.711849  7588 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:35.709460  7904 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:35.684264  8052 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:35.714358  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 638) took 1272 ms. Trace:
I20260709 12:25:35.714627  8052 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:35.716728  8032 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:35.715082  6762 rpcz_store.cc:276] 0709 12:25:34.441479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.067080 (+625601us) service_pool.cc:224] Handling call
0709 12:25:35.067512 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.067957 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.067985 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.068607 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.651208 (+582601us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:35.651285 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.651409 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.714019 (+ 62610us) raft_consensus.cc:1685] finished
0709 12:25:35.714053 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.714266 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1231,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":130,"thread_start_us":581990,"threads_started":1,"wal-append.queue_time_us":642880}
I20260709 12:25:35.724515  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:35.705374  8048 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.218s	user 0.000s	sys 0.002s
I20260709 12:25:35.743909  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:35.679383  7995 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:35.703414  8055 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:35.752808  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:35.718645  8065 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:35.745223  7588 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:25:35.767493  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.771718  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:35.769290  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 709) took 1418 ms. Trace:
I20260709 12:25:35.790091  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:35.790179  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:35.794613  5940 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 7: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:25:35.798692  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 79748.7us
I20260709 12:25:35.799980  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:35.773128  7898 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:35.803756  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:35.804260  7898 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:35.804749  7898 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:35.805270  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:35.779874  8046 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.232s	user 0.003s	sys 0.006s
I20260709 12:25:35.754666  8032 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 12:25:35.806305  6775 raft_consensus.cc:2410] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:35.806885  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:35.811412  5987 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.195:43329 timed out after 1.938s (SENT) [suppressed 25 similar messages]
W20260709 12:25:35.813858  6785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.007s	sys 0.006s
W20260709 12:25:35.815280  6785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.008s	sys 0.006s
I20260709 12:25:35.777930  8068 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:35.816816  6766 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.015s	sys 0.000s
W20260709 12:25:35.817353  6766 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.016s	sys 0.000s
I20260709 12:25:35.791329  8040 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:35.820245  7811 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:35.820355  8032 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:35.846395  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:35.811710  7810 consensus_meta.cc:287] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.695s	user 0.016s	sys 0.002s
I20260709 12:25:35.826097  7974 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:35.826458  8058 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:25:35.808876  7818 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:35.854521  7818 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.854694  8069 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.855082  7818 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.815554  7728 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:35.790663  5865 rpcz_store.cc:276] 0709 12:25:34.350492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.043449 (+692957us) service_pool.cc:224] Handling call
0709 12:25:35.768927 (+725478us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54006db5d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:35.769154 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274353536}
I20260709 12:25:35.865698  8032 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:35.868747  8032 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:35.838688  7995 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:35.839169  7904 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:35.839255  7766 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:25:35.842475  8061 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:25:35.842764  7811 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:35.845348  8059 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:25:35.810168  7992 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.810999  7588 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:35.856594  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 710) took 1485 ms. Trace:
I20260709 12:25:35.875325  8055 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.861151  8073 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:35.869503  8063 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:35.870055  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:35.870745  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 275) took 2086 ms. Trace:
I20260709 12:25:35.870787  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:35.879451  7766 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:35.898586  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1011) took 1843 ms. Trace:
W20260709 12:25:35.818840  7754 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.566s	user 0.016s	sys 0.000s
I20260709 12:25:35.882830  7995 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:35.873296  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:35.874202  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 67587.1us
I20260709 12:25:35.904454  5940 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:35.904732  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:35.875244  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 711) took 1502 ms. Trace:
I20260709 12:25:35.879621  5871 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:35.790819  8065 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:35.888021  7834 raft_consensus.cc:3055] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:35.888298  5873 rpcz_store.cc:276] 0709 12:25:34.371505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.466198 (+1094693us) service_pool.cc:224] Handling call
0709 12:25:35.521260 (+ 55062us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.521775 (+   515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.521808 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.526946 (+  5138us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.683461 (+156515us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:35.683527 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.683649 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.856318 (+172669us) raft_consensus.cc:1685] finished
0709 12:25:35.856345 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.856525 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":66209,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":2907,"thread_start_us":139740,"threads_started":1,"wal-append.queue_time_us":232819}
I20260709 12:25:35.899266  5943 rpcz_store.cc:276] 0709 12:25:34.055096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.228378 (+173282us) service_pool.cc:224] Handling call
0709 12:25:34.228809 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.238751 (+  9942us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.238774 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.257055 (+ 18281us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.512163 (+255108us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:34.512206 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.512279 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.898231 (+1385952us) raft_consensus.cc:1685] finished
0709 12:25:35.898266 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.898487 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":194,"raft.queue_time_us":9544,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":8549,"thread_start_us":217728,"threads_started":1,"wal-append.queue_time_us":1017986}
I20260709 12:25:35.855515  8069 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:35.917258  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:35.909905  7834 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:35.967093  7834 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:35.905860  5867 rpcz_store.cc:276] 0709 12:25:34.372523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.466591 (+1094068us) service_pool.cc:224] Handling call
0709 12:25:35.528992 (+ 62401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.529574 (+   582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.529619 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.540390 (+ 10771us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.719174 (+178784us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:35.719242 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.719413 (+   171us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.875011 (+155598us) raft_consensus.cc:1685] finished
0709 12:25:35.875033 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.875181 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":71732,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":4240,"thread_start_us":137863,"threads_started":1,"wal-append.queue_time_us":239752}
W20260709 12:25:35.965229  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 59091us
W20260709 12:25:35.965385  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 57007.9us
I20260709 12:25:35.972759  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:35.976600  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:35.975314  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 543) took 1737 ms. Trace:
I20260709 12:25:35.995069  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:35.993851  6756 rpcz_store.cc:276] 0709 12:25:34.238115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.238249 (+   134us) service_pool.cc:224] Handling call
0709 12:25:34.260895 (+ 22646us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.487784 (+226889us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.487840 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.493876 (+  6036us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.535099 (+1041223us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:35.535143 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.535222 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.975049 (+439827us) raft_consensus.cc:1685] finished
0709 12:25:35.975074 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.975245 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8496,"raft.queue_time_us":33334,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":14588,"thread_start_us":632279,"threads_started":2,"wal-append.queue_time_us":1170604}
I20260709 12:25:35.906543  7810 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.907815  7995 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.000474  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:36.003729  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:35.916874  7904 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:35.903460  8073 raft_consensus.cc:697] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:35.972674  7974 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:36.009769  7766 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:36.010691  8062 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:36.015496  7745 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:35.973151  7834 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.019443  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:36.006209  8040 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:36.004828  7995 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:35.906651  7754 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:36.027491  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:36.014840  7811 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:36.018127  7588 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:36.027053  8084 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:35.972561  8047 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 12:25:36.008348  7810 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:35.903928  8063 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:36.046937  8077 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:25:36.054061  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 50467.3us
I20260709 12:25:36.028452  7766 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:36.052570  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 81954us
I20260709 12:25:36.055634  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:35.890659  7992 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:36.058708  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:36.067323  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:36.075137  5866 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:36.054158  7898 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:36.076526  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:36.083354  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:36.077258  7414 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:36.087347  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:35.895795  8055 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:36.089780  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:36.094426  5867 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:25:36.084460  5873 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 1.
I20260709 12:25:36.091369  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:36.077435  7810 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:36.070026  8062 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:36.075209  7997 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:36.078397  8065 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:36.119151  5870 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:36.070641  7974 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
W20260709 12:25:36.121738  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.071498  8040 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
W20260709 12:25:36.129585  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 62420.1us
I20260709 12:25:36.104429  7997 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:36.137343  7997 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:36.140354  5867 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:36.108893  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:36.142697  5865 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:36.143060  7871 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:36.143821  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.136394  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:36.145386  5866 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:36.130064  7818 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:36.146440  7995 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:36.149406  7818 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:36.150911  6762 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 3: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:36.151737  7995 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:36.154316  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:36.155005  5871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 844) took 2169 ms (client timeout 2202 ms). Trace:
W20260709 12:25:36.155526  5871 rpcz_store.cc:269] 0709 12:25:33.985238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.985445 (+   207us) service_pool.cc:224] Handling call
0709 12:25:36.154758 (+2169313us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54007350d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:36.154922 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3668580096}
W20260709 12:25:36.161036  6018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 762) took 2132 ms (client timeout 2575 ms). Trace:
W20260709 12:25:36.161582  6018 rpcz_store.cc:269] 0709 12:25:34.028541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.036235 (+  7694us) service_pool.cc:224] Handling call
0709 12:25:36.147930 (+2111695us) spinlock_profiling.cc:243] Waited 50.7 ms on lock 0x7b540041da54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:36.148096 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116640128}
I20260709 12:25:36.147017  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 520) took 1734 ms. Trace:
I20260709 12:25:36.162083  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 553) took 1342 ms. Trace:
I20260709 12:25:36.163362  6757 rpcz_store.cc:276] 0709 12:25:34.412722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.413899 (+  1177us) service_pool.cc:224] Handling call
0709 12:25:34.447616 (+ 33717us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.819616 (+372000us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.819652 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:34.819932 (+   280us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:34.819976 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.146726 (+1326750us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b5400472a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:36.146784 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.146948 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":655121792}
W20260709 12:25:36.164906  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 72623us
I20260709 12:25:36.163491  6766 rpcz_store.cc:276] 0709 12:25:34.820221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.122042 (+301821us) service_pool.cc:224] Handling call
0709 12:25:35.122695 (+   653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.122988 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:35.817735 (+694747us) log.cc:844] Serialized 14 byte log entry
0709 12:25:36.135685 (+317950us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:36.135768 (+    83us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:36.135830 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.135921 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.162004 (+ 26083us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":766090,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":186,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":101,"thread_start_us":1011567,"threads_started":2}
I20260709 12:25:36.174376  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:36.127022  8006 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:36.176988  7992 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:36.180379  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:36.188488  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 57975.8us
I20260709 12:25:36.190472  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:36.192065  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:36.192564  6757 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:36.194271  8063 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:36.189015  7754 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:36.197233  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:36.198484  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:35.897478  5941 rpcz_store.cc:276] 0709 12:25:33.783979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.789703 (+  5724us) service_pool.cc:224] Handling call
0709 12:25:33.790130 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.861414 (+ 71284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.864171 (+  2757us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.880327 (+ 16156us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.711205 (+1830878us) spinlock_profiling.cc:243] Waited 32.2 ms on lock 0x7b180018b9e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:25:35.800804 (+ 89599us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:35.802430 (+  1626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.811694 (+  9264us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.857299 (+ 45605us) raft_consensus.cc:1685] finished
0709 12:25:35.857328 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.870673 (+ 13345us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89230,"raft.queue_time_us":72140,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":7431,"spinlock_wait_cycles":74142976,"thread_start_us":80992,"threads_started":1,"wal-append.queue_time_us":1941764}
I20260709 12:25:36.089884  8073 consensus_queue.cc:237] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:36.202412  8080 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 12:25:36.204241  8057 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:36.254657  6611 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Timed out: RequestConsensusVote RPC to 127.2.4.196:34835 timed out after 2.150s (SENT)
I20260709 12:25:36.258496  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:36.255443  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:36.277807  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.198768  8069 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:36.284858  8034 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:36.291091  8091 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:25:36.197351  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:36.292760  5866 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.296250  7840 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:36.297925  6766 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
W20260709 12:25:36.302716  8038 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.308s	user 0.017s	sys 0.016s
I20260709 12:25:36.307574  5869 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:25:36.307771  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 53194.5us
I20260709 12:25:36.297530  7840 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:36.315275  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 280) took 3464 ms. Trace:
I20260709 12:25:36.315725  5869 rpcz_store.cc:276] 0709 12:25:32.849469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.975368 (+125899us) service_pool.cc:224] Handling call
0709 12:25:36.300336 (+3324968us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b58002e2f98. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:25:36.300563 (+   227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:36.308049 (+  7486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.313398 (+  5349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2791680}
I20260709 12:25:36.276976  8063 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:36.318344  8063 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:36.321251  6757 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:36.322767  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:36.324179  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 769) took 2223 ms. Trace:
I20260709 12:25:36.324594  6024 rpcz_store.cc:276] 0709 12:25:34.101065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.826315 (+725250us) service_pool.cc:224] Handling call
0709 12:25:34.826776 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.246041 (+419265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.323926 (+1077885us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b5400763754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:36.324118 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":931842816}
W20260709 12:25:36.339327  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.323405  6757 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
W20260709 12:25:36.346856  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 155181us
I20260709 12:25:36.318094  7997 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:36.358366  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:36.358721  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:36.360327  5871 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:36.360332  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:36.313524  7818 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:36.359444  6757 raft_consensus.cc:2376] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6. Re-sending same reply.
I20260709 12:25:36.342404  8063 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:36.362795  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:36.379351  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:36.380934  5943 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.349874  7893 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:36.388706  5866 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:36.342764  8013 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:36.389365  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:36.437372  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 58167.8us
W20260709 12:25:36.435757  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 71787.4us
W20260709 12:25:36.437969  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 48711.9us
I20260709 12:25:36.438350  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:36.439405  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:36.439741  5987 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:36.440429  5840 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:36.443624  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:36.448391  8006 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:36.449429  8063 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:36.450080  8063 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:36.450618  8063 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:36.365201  8080 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:36.448498  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:36.473275  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.436678  7871 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:36.363206  7889 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.084s
I20260709 12:25:36.443354  8069 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:36.446694  8040 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:36.378077  7834 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.098s
I20260709 12:25:36.472747  8006 raft_consensus.cc:697] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:36.471841  8080 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:36.508831  8080 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:36.503508  8040 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.507287  8008 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:36.507817  8006 consensus_queue.cc:237] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:36.482411  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:36.442965  7893 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:36.523681  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 83999.4us
I20260709 12:25:36.537473  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:36.521850  8040 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:36.509847  7997 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:36.552291  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.504415  8069 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:36.586359  8100 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:36.586894  7893 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:36.587258  8100 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:36.587944  8100 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.533963  8008 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:25:36.599604  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.589352  7875 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:36.601280  5941 raft_consensus.cc:2410] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:25:36.627828  7871 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:36.650115  6757 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 12:25:36.635449  7875 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:25:36.649991  8041 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.009s	user 0.013s	sys 0.003s
I20260709 12:25:36.670392  5943 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
W20260709 12:25:36.666556  8053 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.943s	user 0.007s	sys 0.009s
I20260709 12:25:36.659375  8095 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:36.669433  8097 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:36.683310  8099 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:25:36.692461  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:36.662381  8041 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:36.693329  5948 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
W20260709 12:25:36.684275  7987 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 3.110s	user 0.007s	sys 0.023s
I20260709 12:25:36.678105  7871 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:36.695119  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:36.695488  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:36.651448  8008 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:36.698423  5948 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:36.692499  7893 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:36.722685  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 305183us
W20260709 12:25:36.723402  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:36.725382  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.726756  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 624) took 3845 ms. Trace:
I20260709 12:25:36.681334  8095 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.727177  6021 rpcz_store.cc:276] 0709 12:25:32.870629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.888985 (+ 18356us) service_pool.cc:224] Handling call
0709 12:25:32.889439 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.889954 (+   515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.889996 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.900819 (+ 10823us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.331927 (+431108us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:33.576583 (+244656us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.590486 (+ 13903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.590573 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:36.683646 (+3093073us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:36.696825 (+ 13179us) raft_consensus.cc:1685] finished
0709 12:25:36.716024 (+ 19199us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.716271 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6695,"prepare.queue_time_us":464746,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":60,"raft.queue_time_us":15860,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":18993,"spinlock_wait_cycles":6398208,"thread_start_us":640084,"threads_started":2,"wal-append.queue_time_us":584959}
W20260709 12:25:36.734222  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.734436  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:36.738122  8041 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:36.677557  8053 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:36.740450  7904 consensus_meta.cc:287] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.727s	user 0.012s	sys 0.001s
I20260709 12:25:36.741017  7904 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:36.741238  8053 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:36.730533  8095 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:36.741991  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:36.742939  5947 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:36.744184  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:36.717954  8097 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:36.723664  7875 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:36.753638  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:36.730108  7997 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:36.754369  5948 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:36.728005  7893 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:36.758371  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:36.753026  8008 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:36.763684  5838 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:36.748811  8069 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:36.769063  5838 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:36.766637  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:36.767280  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:36.752491  8097 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.771569  5838 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:36.773452  7810 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.773967  7974 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:25:36.780877  5942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 737) took 3868 ms (client timeout 1938 ms). Trace:
W20260709 12:25:36.781744  5942 rpcz_store.cc:269] 0709 12:25:32.912666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.915430 (+  2764us) service_pool.cc:224] Handling call
0709 12:25:36.780452 (+3865022us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b5400309dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:36.780740 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":914442624}
I20260709 12:25:36.785611  8069 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:36.763288  7997 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:36.784457  7974 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.804368  8069 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.836691  8069 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:36.818152  8104 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 12:25:36.819897  6773 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 643) took 2310 ms (client timeout 2084 ms). Trace:
I20260709 12:25:36.821506  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 537) took 1028 ms. Trace:
I20260709 12:25:36.829737  7818 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:36.831580  7741 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:36.823805  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 358) took 3101 ms. Trace:
I20260709 12:25:36.835829  5942 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:25:36.850199  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.848785  6022 rpcz_store.cc:276] 0709 12:25:33.722848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.759875 (+ 37027us) service_pool.cc:224] Handling call
0709 12:25:33.777970 (+ 18095us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.306394 (+528424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.823435 (+2517041us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b5400172ad4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:36.823722 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1152293632}
I20260709 12:25:36.835472  8095 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:36.835892  7997 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:36.833228  8100 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:36.836148  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1015) took 2468 ms. Trace:
I20260709 12:25:36.806980  7974 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:36.843995  6773 rpcz_store.cc:269] 0709 12:25:34.509980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.927774 (+417794us) service_pool.cc:224] Handling call
0709 12:25:36.819600 (+1891826us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54002c24d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:36.819789 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262314240}
I20260709 12:25:36.844187  6762 rpcz_store.cc:276] 0709 12:25:35.792541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.018637 (+226096us) service_pool.cc:224] Handling call
0709 12:25:36.821194 (+802557us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b5400328cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:36.821392 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252004224}
I20260709 12:25:36.854426  5946 rpcz_store.cc:276] 0709 12:25:34.368061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.480993 (+112932us) service_pool.cc:224] Handling call
0709 12:25:34.481370 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.230815 (+749445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.230850 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:35.230902 (+    52us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:35.230939 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.835889 (+1604950us) spinlock_profiling.cc:243] Waited 713 ms on lock 0x7b540058fa54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:36.835942 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.836082 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1640290688}
I20260709 12:25:36.856459  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:36.859213  5948 raft_consensus.cc:2410] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:25:36.827099  7893 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:36.856459  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:36.871169  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.873154  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:36.874090  5946 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:36.876399  8095 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:36.846419  7818 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:36.878357  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:36.881917  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.887974  7818 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:36.896575  7840 consensus_meta.cc:287] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.005s
I20260709 12:25:36.898761  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:36.899160  8063 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:36.901934  7588 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:25:36.948407  6023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 765) took 2857 ms (client timeout 2213 ms). Trace:
W20260709 12:25:36.948966  6023 rpcz_store.cc:269] 0709 12:25:34.090687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.269933 (+179246us) service_pool.cc:224] Handling call
0709 12:25:36.906822 (+2636889us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b5400764154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:36.948284 (+ 41462us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1244482944}
I20260709 12:25:36.949616  8063 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:36.950796  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:36.892669  8080 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:36.961721  8107 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 12:25:36.947058  8108 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:25:36.966907  5868 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:36.969980  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.970161  8056 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.957580  8033 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:36.998811  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 704) took 2852 ms. Trace:
I20260709 12:25:36.976253  7996 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.979478  6756 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:36.979643  7799 raft_consensus.cc:3055] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:25:36.982391  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.986749  5987 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:36.999851  5868 rpcz_store.cc:276] 0709 12:25:34.146080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.309051 (+162971us) service_pool.cc:224] Handling call
0709 12:25:36.966358 (+2657307us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b58002e2f98. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:25:36.966647 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:36.998339 (+ 31692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.998709 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4699392}
I20260709 12:25:36.991323  7818 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:36.998745  8033 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.003609  8056 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:37.022101  8033 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.023705  8056 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:37.023891  7939 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:37.027479  8080 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:37.014931  7898 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:37.030382  8068 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:37.011787  7799 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:37.032469  8068 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:37.032289  7799 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:37.033748  7799 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:37.034359  5697 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40830 (request call id 147) took 26877 ms (client timeout 15000 ms). Trace:
I20260709 12:25:37.034703  7711 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:37.035297  7711 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:37.036454  7711 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:37.038302  7711 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:37.039194  8110 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:25:37.044236  8055 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:37.044669  7994 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:36.994299  8097 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:37.040745  7875 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:37.050484  8015 raft_consensus.cc:3055] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:37.051131  8015 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:37.052204  8015 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:37.052796  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:37.053323  5694 catalog_manager.cc:5697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 4 to 5, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 5 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:37.054358  5942 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:37.055830  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:37.055483  5694 catalog_manager.cc:5697] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:37.056200  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.057878  8016 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:37.057268  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:37.059268  5940 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:37.059913  8100 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:37.060361  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.061281  6757 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:37.061509  6762 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:37.062323  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:37.062692  8057 consensus_meta.cc:287] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.765s	user 0.016s	sys 0.000s
I20260709 12:25:37.063210  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 622) took 4243 ms. Trace:
I20260709 12:25:37.063444  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.064296  6773 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:37.064672  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.066053  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 516) took 2806 ms. Trace:
I20260709 12:25:37.066583  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:37.060547  8037 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:37.068284  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.062603  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.069447  5838 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:37.070158  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.065505  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.080089  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:36.944963  7588 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:37.057248  5694 catalog_manager.cc:5697] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
W20260709 12:25:37.086743  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.089306  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:37.090186  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.090199  5947 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
W20260709 12:25:37.093924  5990 connection.cc:256] RPC call timeout handler was delayed by -1.0694871610003247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20840000000000003s for a response
W20260709 12:25:37.094504  5990 connection.cc:256] RPC call timeout handler was delayed by -1.0460643189999246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
I20260709 12:25:37.094700  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:37.095082  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:37.096613  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.097568  5942 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:37.099983  5867 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:37.102648  6023 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:37.103438  5869 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:25:37.066468  6785 rpcz_store.cc:276] 0709 12:25:34.259871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.267667 (+  7796us) service_pool.cc:224] Handling call
0709 12:25:34.290472 (+ 22805us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.507288 (+216816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.507350 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.678589 (+171239us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.815629 (+1137040us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:36.110441 (+294812us) log.cc:844] Serialized 14 byte log entry
0709 12:25:36.111461 (+  1020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.065656 (+954195us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b54005294d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:37.065717 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:37.065763 (+    46us) raft_consensus.cc:1685] finished
0709 12:25:37.065851 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.065993 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12007,"prepare.queue_time_us":153119,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":5389,"raft.run_cpu_time_us":503,"raft.run_wall_time_us":112215,"spinlock_wait_cycles":486805248,"thread_start_us":1001596,"threads_started":2,"wal-append.queue_time_us":975227}
I20260709 12:25:37.105535  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.106869  6015 raft_consensus.cc:2410] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:37.103729  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.109257  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.110873  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.111774  5948 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:25:37.112226  5948 pending_rounds.cc:85] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 1
I20260709 12:25:37.112457  5948 pending_rounds.cc:107] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:25:37.040668  7898 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:25:37.120393  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.121268  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.122224  5694 catalog_manager.cc:5697] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 1 to 2, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 2 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:37.123245  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.122362  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:37.113305  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:37.122362  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.053396  8015 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:37.117957  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.132414  5874 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:37.112687  5948 raft_consensus.cc:2889] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:37.113958  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:37.123270  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 650) took 1337 ms. Trace:
I20260709 12:25:37.133534  6766 rpcz_store.cc:276] 0709 12:25:35.785435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.198224 (+412789us) service_pool.cc:224] Handling call
0709 12:25:37.123089 (+924865us) spinlock_profiling.cc:243] Waited 98.8 ms on lock 0x7b540052a154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:37.123207 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227293952}
I20260709 12:25:37.125015  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:37.127357  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:37.134415  5840 leader_election.cc:336] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Timed out: RequestConsensusVote RPC to 127.2.4.196:34835 timed out after 2.575s (SENT)
I20260709 12:25:37.063488  8057 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.134799  5946 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:37.134797  6021 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:37.130419  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:37.131301  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.132263  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.136518  6024 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:37.130334  6018 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 5. Ops: []
W20260709 12:25:37.108534  8040 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.004s
I20260709 12:25:37.137488  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.090049  8055 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:37.089231  7738 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.143476  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.143520  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:37.161422  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:37.144961  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.151925  6020 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:25:37.155341  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:37.152890  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:37.164700  5867 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:37.094184  8056 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:37.063690  6016 rpcz_store.cc:276] 0709 12:25:32.819945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.857863 (+ 37918us) service_pool.cc:224] Handling call
0709 12:25:32.858258 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.922736 (+ 64478us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:33.366499 (+443763us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.984272 (+617773us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:33.984330 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:33.984400 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.445638 (+461238us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b5400364628. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d65ed161 00007f39d668530b 00007f39d6685259 00007f39d66851e9 00007f39d66851b1 00007f39d66844ad 0000000000388634 0000000000388479 00007f39cfb96475 00007f39cfb987c1 00007f39cfb98779
0709 12:25:37.062922 (+2617284us) spinlock_profiling.cc:243] Waited 53.9 ms on lock 0x7b54003647d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:37.062979 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.063116 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38966,"prepare.queue_time_us":590628,"prepare.run_cpu_time_us":10170,"prepare.run_wall_time_us":932083,"raft.queue_time_us":5466,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":6887,"spinlock_wait_cycles":131271296,"thread_start_us":964569,"threads_started":2,"wal-append.queue_time_us":630805,"wal-append.run_cpu_time_us":1506,"wal-append.run_wall_time_us":1006661}
I20260709 12:25:37.114152  7994 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:37.172284  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:37.157953  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.099078  7592 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:37.112043  7597 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.143975  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.136972  8094 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.093116  8014 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:37.207409  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46081.4us
I20260709 12:25:37.166821  5942 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:37.170037  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.117027  7875 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:37.173895  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.156101  5694 catalog_manager.cc:5697] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:37.082662  8080 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:25:37.181708  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 68931.4us
W20260709 12:25:37.182947  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 139600us
W20260709 12:25:37.035143  5697 rpcz_store.cc:269] 0709 12:25:10.156570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.156856 (+   286us) service_pool.cc:224] Handling call
0709 12:25:37.033995 (+26877139us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:24.726720 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:25.002591 (+275871us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:25.006449 (+  3858us) write_op.cc:435] Acquired schema lock
0709 12:25:25.026566 (+ 20117us) tablet.cc:662] Decoding operations
0709 12:25:25.027535 (+   969us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:25.027595 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:25.027625 (+    30us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:25:25.028281 (+   656us) tablet.cc:701] Row locks acquired
0709 12:25:25.028301 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:25:25.028453 (+   152us) write_op.cc:270] Start()
0709 12:25:25.028581 (+   128us) write_op.cc:276] Timestamp: P: 1783599925028401 usec, L: 0
0709 12:25:25.028602 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:25:25.029296 (+   694us) log.cc:844] Serialized 3591 byte log entry
0709 12:25:25.883639 (+854343us) spinlock_profiling.cc:243] Waited 4.38 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39cfb790ac 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b
0709 12:25:27.181092 (+1297453us) op_driver.cc:464] REPLICATION: finished
0709 12:25:27.570438 (+389346us) write_op.cc:301] APPLY: starting
0709 12:25:27.573098 (+  2660us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:27.607561 (+ 34463us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:27.607594 (+    33us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:27.608964 (+  1370us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:27.609296 (+   332us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 12:25:27.609352 (+    56us) write_op.cc:312] APPLY: finished
0709 12:25:27.622775 (+ 13423us) log.cc:844] Serialized 70 byte log entry
0709 12:25:27.647303 (+ 24528us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:27.654134 (+  6831us) write_op.cc:454] Released schema lock
0709 12:25:27.660122 (+  5988us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:27.695546 (+ 35424us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:27.698046 (+  2500us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":676494,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":355398,"apply.run_cpu_time_us":4552,"apply.run_wall_time_us":129255,"num_ops":9,"prepare.queue_time_us":177151,"prepare.run_cpu_time_us":45799,"prepare.run_wall_time_us":1794217,"raft.queue_time_us":295452,"raft.run_cpu_time_us":20252,"raft.run_wall_time_us":343174,"replication_time_us":2098348,"spinlock_wait_cycles":10083712,"thread_start_us":1999693,"threads_started":4,"wal-append.queue_time_us":1256927,"wal-append.run_cpu_time_us":30983,"wal-append.run_wall_time_us":2351279}]]}
I20260709 12:25:37.186750  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:37.058511  8016 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:37.189795  8117 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.171267  7738 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:37.131714  7898 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:37.217026  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:37.190583  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56201us
W20260709 12:25:37.191020  8113 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:25:37.163911  6020 pending_rounds.cc:85] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 1
I20260709 12:25:37.237386  6020 pending_rounds.cc:107] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:25:37.237849  6020 raft_consensus.cc:2889] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:37.194237  6017 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:37.197618  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:37.204020  5940 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:25:37.183104  7499 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:37.210633  5990 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:37.211621  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 94730.5us
I20260709 12:25:37.161543  8055 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.161149  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.224099  5867 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:37.161953  8040 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:37.122341  8036 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.079s
I20260709 12:25:37.168730  8037 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:37.214540  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.234961  5694 catalog_manager.cc:5697] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:25:37.201894  8006 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:37.246774  5946 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 3.
I20260709 12:25:37.243130  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:37.218954  7900 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:37.304808  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:37.305491  6768 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:37.305594  5942 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:37.265710  5990 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:37.310204  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.313093  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.314266  6020 raft_consensus.cc:2393] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 5: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:25:37.266510  6756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 536) took 1474 ms (client timeout 1909 ms). Trace:
W20260709 12:25:37.315240  6756 rpcz_store.cc:269] 0709 12:25:35.792262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.068917 (+276655us) service_pool.cc:224] Handling call
0709 12:25:37.266256 (+1197339us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b5400004954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:37.266423 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2048933248}
I20260709 12:25:37.316748  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.244962  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:37.273650  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:37.318238  6775 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:37.275415  5947 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:37.284413  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:37.325515  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:37.326686  6015 raft_consensus.cc:2368] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 7. Current term is 8.
I20260709 12:25:37.192759  7994 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:37.346664  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:37.257840  5940 pending_rounds.cc:85] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 4
I20260709 12:25:37.347949  5940 pending_rounds.cc:107] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:25:37.348248  5940 raft_consensus.cc:2889] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:37.298499  6778 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:37.298928  5869 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:37.301828  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.187727  8056 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:37.264688  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.351934  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:37.352128  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 107244us
I20260709 12:25:37.353147  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:37.198732  7592 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:37.353935  8072 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:37.359192  8072 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:37.359308  7994 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:37.359678  8072 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.242242  8117 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:37.363750  8117 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:37.365851  7997 raft_consensus.cc:3055] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:37.366328  7997 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:37.367288  7997 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:37.368799  7997 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.369642  8052 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:37.370057  6019 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:37.350970  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.276850  8053 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:37.241046  7880 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:37.270747  8055 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.233445  8080 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:37.289737  5694 catalog_manager.cc:5697] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 5, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 5 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:37.241711  8034 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.113s
I20260709 12:25:37.209499  7597 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:37.407464  5694 catalog_manager.cc:5697] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 3 to 4, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 4 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:37.378163  6775 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
W20260709 12:25:37.409448  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 178092us
I20260709 12:25:37.379843  6021 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:37.228742  8095 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.102s
I20260709 12:25:37.383725  5869 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:37.216634  8094 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:37.387773  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.388938  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.354715  5872 raft_consensus.cc:2368] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:25:37.390916  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.418370  6015 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 2: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:37.418803  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.419680  5874 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
W20260709 12:25:37.356048  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 113070us
W20260709 12:25:37.373871  8008 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.582s	user 0.013s	sys 0.000s
W20260709 12:25:37.398053  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:37.396984  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 179407us
W20260709 12:25:37.373740  7810 consensus_meta.cc:287] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.593s	user 0.005s	sys 0.009s
W20260709 12:25:37.411073  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 58022.7us
I20260709 12:25:37.412432  6768 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:37.429104  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:37.412989  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.431427  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:37.388233  7880 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:37.440186  6609 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:37.386698  8097 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:37.382639  7970 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:37.412555  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:37.448653  6766 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:37.382143  8056 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:37.425992  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 177552us
I20260709 12:25:37.474216  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:37.424959  5694 catalog_manager.cc:5697] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 4 to 5, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 5 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:37.475176  5874 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.359344  7592 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:37.292352  8069 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:37.476516  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:37.476841  5990 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:37.430505  6019 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:37.432472  6021 raft_consensus.cc:2410] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:25:37.258309  8014 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:37.433254  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:37.433578  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:37.439126  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:37.393430  5865 raft_consensus.cc:2393] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 3: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:37.440989  7810 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.329744  8037 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:37.442354  7880 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:37.451586  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:37.502103  6756 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:37.407132  8118 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.453974  7588 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:37.454175  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.443688  7775 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:25:37.457780  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.458165  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:37.459218  7896 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.441833  7904 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:37.459270  8080 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:37.460054  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.463967  8006 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:37.431500  7436 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:37.532974  7436 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:37.469285  6762 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:37.471037  7741 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:37.414278  8053 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:37.408030  7973 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:37.482194  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.486589  5867 raft_consensus.cc:2376] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7. Re-sending same reply.
I20260709 12:25:37.418041  7875 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:37.480799  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:37.486658  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.480896  6017 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:37.484351  5694 catalog_manager.cc:5697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:37.410193  8034 consensus_queue.cc:1243] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:25:37.490900  5990 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:37.551309  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:37.551836  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 60974.4us
I20260709 12:25:37.449858  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.552524  5867 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:25:37.493791  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 60618.6us
I20260709 12:25:37.554462  5694 catalog_manager.cc:5697] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 3, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 3 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:37.495476  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 62000.6us
I20260709 12:25:37.559049  5915 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:37.496054  7996 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:37.497169  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.487706  8015 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:25:37.499109  8069 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:37.500777  5943 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
W20260709 12:25:37.507748  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.512007  5866 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:37.573127  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:37.517371  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:37.492609  7871 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:37.527499  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.533648  8040 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:37.528074  7896 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:37.505515  8050 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:37.537770  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 57279.9us
I20260709 12:25:37.545441  5874 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
W20260709 12:25:37.548715  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.522516  7904 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:37.533581  7939 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.001s
I20260709 12:25:37.562620  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:37.589375  5867 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:37.562212  7996 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.565912  5990 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:37.553692  8126 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:37.554406  7775 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:37.575294  5694 catalog_manager.cc:5697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 1 to 2, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 2 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:37.574576  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 57308us
I20260709 12:25:37.571352  8118 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:37.580822  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.610008  7970 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:37.592188  7592 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:37.611006  7711 consensus_meta.cc:287] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.572s	user 0.012s	sys 0.003s
I20260709 12:25:37.549564  7973 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:37.584903  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:37.577128  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:37.615440  5872 raft_consensus.cc:2376] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5. Re-sending same reply.
I20260709 12:25:37.586472  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:37.577509  8073 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:37.563933  8016 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:37.600414  7921 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:37.600881  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:37.601287  5990 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:37.601903  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:37.631014  5870 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:37.631582  5913 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:37.591573  7904 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:37.613451  6793 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
W20260709 12:25:37.581297  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.601397  7996 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:37.596201  7974 consensus_meta.cc:287] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.718s	user 0.010s	sys 0.003s
I20260709 12:25:37.619160  5944 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:25:37.637573  5944 pending_rounds.cc:85] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 3
I20260709 12:25:37.637910  5867 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:25:37.620769  5840 leader_election.cc:400] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 7. Current term is 8.
I20260709 12:25:37.638521  5840 leader_election.cc:403] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:25:37.564811  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:37.576658  5865 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
W20260709 12:25:37.621486  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.628846  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:37.627064  5694 catalog_manager.cc:5697] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 4 to 5, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 5 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:37.633001  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.635366  5990 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:37.641938  5840 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:37.636409  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.637995  5944 pending_rounds.cc:107] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:25:37.646683  5944 raft_consensus.cc:2889] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:37.647373  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:37.648010  6757 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:37.648367  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:37.639338  5867 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:25:37.595023  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.649343  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:37.622803  8126 raft_consensus.cc:697] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:37.650614  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.658039  5874 raft_consensus.cc:2393] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 6: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:37.647455  7592 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:37.658583  8073 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:37.642999  7900 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:25:37.661953  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.662137  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:37.662637  7900 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:37.663046  7900 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:37.663426  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.663905  7597 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:37.664026  7900 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:37.626989  8015 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:37.678429  8015 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:37.679056  8037 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:37.652860  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.654417  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:37.655097  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:37.680862  5942 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:37.655699  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:37.664989  6773 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:37.627435  8129 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.688651  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:37.690328  5913 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:37.643231  7974 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:37.666633  6775 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:25:37.669382  5865 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:37.670266  6015 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:37.670107  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.670809  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:37.696362  6019 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:37.671284  7436 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:37.671465  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:37.672458  8033 consensus_meta.cc:287] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.650s	user 0.010s	sys 0.000s
I20260709 12:25:37.649408  7818 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:37.647162  8080 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:37.713532  8080 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:37.714368  8076 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.714661  8080 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:37.715071  8076 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:37.665758  6785 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:37.651588  5694 catalog_manager.cc:5697] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 4, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 4 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:37.679522  8015 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:37.719260  5694 catalog_manager.cc:5697] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 4, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 4 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:37.720245  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:37.685514  5874 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:37.639963  7994 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:37.683070  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:37.687366  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 122626us
I20260709 12:25:37.638085  8118 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:37.698714  6020 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:37.650755  8128 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 7. Current term is 8.
I20260709 12:25:37.739326  7436 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:37.749217  5915 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:37.751976  7994 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:37.693397  8094 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:37.753255  7994 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:37.672593  7592 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:37.701959  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:37.707105  5870 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:37.708352  5838 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:37.710173  7840 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:37.710389  5872 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 5. Ops: []
I20260709 12:25:37.711421  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:37.712044  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.699489  8016 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.688386  8053 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:37.755780  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.720976  5694 catalog_manager.cc:5697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 4 to 5, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 5 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:37.756280  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 48052.5us
I20260709 12:25:37.746064  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.765671  5867 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:37.698650  8037 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 12:25:37.766237  6775 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 7: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:37.766767  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.767445  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.767731  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:37.768927  5867 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:37.741782  5874 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:37.769177  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.757517  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:37.759058  6756 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
W20260709 12:25:37.765481  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 124256us
I20260709 12:25:37.756001  5946 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:37.740602  5868 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 3: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:37.702841  8124 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:37.810338  5870 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.770810  5694 catalog_manager.cc:5697] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 1 to 2, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 2 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:25:37.812253  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:37.762898  7900 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:37.701860  8130 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:25:37.764117  8120 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:25:37.703141  8095 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 12:25:37.842881  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:37.813126  6775 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:37.740031  7974 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:37.830037  8126 consensus_queue.cc:237] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:37.699486  8055 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:37.759881  8118 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:37.847543  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:37.848584  5946 raft_consensus.cc:2410] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:25:37.814328  8053 raft_consensus.cc:697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:37.764825  8008 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:37.841317  5870 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:37.765493  8128 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:37.854233  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:37.854954  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 95701.8us
I20260709 12:25:37.857475  5870 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:37.846668  8118 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.861709  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:37.862728  5946 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:37.858932  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.863940  5873 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 5: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:37.832824  5694 catalog_manager.cc:5697] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 1 to 2, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 2 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:37.721839  8033 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:37.847024  8130 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:37.817457  7840 raft_consensus.cc:697] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:37.854490  8117 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.869256  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:37.842834  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 120047us
I20260709 12:25:37.760721  7996 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:37.871034  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:37.877131  8129 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.855852  8008 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:37.843832  7880 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:37.857838  8128 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:37.861696  8134 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:37.883903  5871 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:37.847175  8072 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:37.884586  6022 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 5: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:37.890398  7840 consensus_queue.cc:237] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:37.891794  8076 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:37.899755  5872 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:37.902287  7996 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:37.906723  5865 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:37.907588  7880 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:37.907750  7904 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:37.901281  7973 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:37.915517  5946 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:37.867098  8063 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:37.926090  6757 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:25:37.881266  8008 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:37.909574  8128 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.940464  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:37.942153  5946 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 8: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:25:37.945350  8115 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.682s	user 0.003s	sys 0.000s
I20260709 12:25:37.908771  8139 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:37.878732  8015 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:25:37.922442  8009 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:37.928344  8016 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:37.965081  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.013806  5946 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:25:38.014554  5946 pending_rounds.cc:85] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 3
I20260709 12:25:38.014881  5946 pending_rounds.cc:107] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:25:38.016029  5946 raft_consensus.cc:2889] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:38.028314  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 157563us
I20260709 12:25:37.907657  8130 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:37.952993  8009 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.042268  5871 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:38.012184  8016 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:37.938414  8063 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:38.000236  8139 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:38.042616  8009 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:38.058419  8144 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.046321  8016 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.948756  8053 consensus_queue.cc:237] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:38.044530  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:38.067719  8033 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:38.070264  8132 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 12:25:38.071224  8063 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:38.063140  8055 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:38.075417  8139 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:38.065264  7840 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:38.077991  8035 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.080389  7738 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:38.085114  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.075459  8072 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:38.086138  8144 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:38.087862  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:38.088642  8016 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:38.092731  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 848) took 1008 ms. Trace:
I20260709 12:25:38.095271  7799 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.095817  7973 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.102284  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:38.102458  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 375112us
I20260709 12:25:38.153877  6024 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:38.154963  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.156271  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:38.088719  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.089449  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.109139  8118 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:38.054636  7996 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.126257  5838 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:38.136089  6793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 639) took 3663 ms (client timeout 2025 ms). Trace:
I20260709 12:25:38.144472  6021 raft_consensus.cc:2376] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7. Re-sending same reply.
I20260709 12:25:38.164234  6021 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:38.161926  6793 rpcz_store.cc:269] 0709 12:25:34.456758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.905994 (+449236us) service_pool.cc:224] Handling call
0709 12:25:38.119592 (+3213598us) spinlock_profiling.cc:243] Waited 2.67 s on lock 0x7b540004d8d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:38.119715 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6132853120}
I20260709 12:25:38.146413  5873 rpcz_store.cc:276] 0709 12:25:37.084111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.084655 (+   544us) service_pool.cc:224] Handling call
0709 12:25:38.092450 (+1007795us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b5400653e54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:38.092642 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1701503232}
W20260709 12:25:38.157900  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:38.167984  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.157994  5865 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:38.169521  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.171774  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.134626  8066 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:25:38.172801  6015 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:38.143100  8124 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
W20260709 12:25:38.181308  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 312097us
I20260709 12:25:38.152457  7840 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:38.167264  8055 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:38.185997  6015 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:38.186054  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 115228us
I20260709 12:25:38.189435  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:38.198390  6793 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:38.150895  8035 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:38.194257  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.200932  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 277) took 4096 ms. Trace:
I20260709 12:25:38.202404  6775 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:38.155310  7799 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:38.206049  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:38.157656  7973 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:38.207501  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.204831  6609 leader_election.cc:400] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:25:38.168249  7996 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:38.250805  8145 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:25:38.251833  8144 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:38.252840  8124 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:38.254796  8118 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:38.256474  8016 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:38.256605  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:38.257418  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:38.263127  7840 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:38.200763  7773 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:38.269261  8035 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:38.268852  8128 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:38.170463  8063 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:25:38.273312  6609 leader_election.cc:403] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:25:38.378194  5870 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:38.378526  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51607.2us
I20260709 12:25:38.279026  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:38.290395  5873 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:38.201459  5945 rpcz_store.cc:276] 0709 12:25:34.104403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.316602 (+212199us) service_pool.cc:224] Handling call
0709 12:25:34.316840 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.316986 (+   146us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:34.365422 (+ 48436us) log.cc:844] Serialized 14 byte log entry
0709 12:25:36.961803 (+2596381us) spinlock_profiling.cc:243] Waited 7.59 ms on lock 0x7b1800151c00. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:25:37.944682 (+982879us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:38.200198 (+255516us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b1800151c00. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39d282b4bc 00007f39cfb7948b 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287
0709 12:25:38.200425 (+   227us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:38.200506 (+    81us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:38.200581 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.200706 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.200861 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82022,"prepare.queue_time_us":46562,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":110669,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":21502,"spinlock_wait_cycles":56910720,"thread_start_us":302945,"threads_started":2,"wal-append.queue_time_us":2826464}
I20260709 12:25:38.298756  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:38.317911  6793 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:38.318652  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:38.173985  8130 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:25:38.383987  6775 raft_consensus.cc:2410] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:38.337067  8124 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:25:38.340907  6778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 517) took 4052 ms (client timeout 2972 ms). Trace:
I20260709 12:25:38.362462  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:38.363370  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:38.270991  7974 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.297313  8055 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:38.422561  6766 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:38.369849  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:38.273542  6015 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:38.379158  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:38.387395  6611 leader_election.cc:336] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Timed out: RequestConsensusVote RPC to 127.2.4.196:34835 timed out after 1.613s (SENT)
W20260709 12:25:38.423491  6778 rpcz_store.cc:269] 0709 12:25:34.288977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.297108 (+  8131us) service_pool.cc:224] Handling call
0709 12:25:38.340645 (+4043537us) spinlock_profiling.cc:243] Waited 2.94 s on lock 0x7b54002a21d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:38.340800 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6763600256}
I20260709 12:25:38.272550  8009 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:38.270735  8008 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:38.371611  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:38.429425  6611 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:38.326581  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:38.429469  6762 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
W20260709 12:25:38.434392  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.448091  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 294) took 1329 ms. Trace:
I20260709 12:25:38.450732  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:38.450951  7973 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:38.451644  5948 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:38.452562  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:38.385759  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:38.453929  5941 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:38.454403  8149 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:25:38.455278  8055 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:38.456264  8130 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:38.456722  8130 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:38.453143  5948 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:38.458767  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:38.458972  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.445703  5940 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:38.376309  8129 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.461998  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:38.463064  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 75730.4us
I20260709 12:25:38.448784  5943 rpcz_store.cc:276] 0709 12:25:37.118536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.130096 (+ 11560us) service_pool.cc:224] Handling call
0709 12:25:38.447875 (+1317779us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b5400632254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:38.447990 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838334208}
I20260709 12:25:38.469696  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:38.476814  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:38.374764  7773 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:38.483608  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 278892us
I20260709 12:25:38.454521  8134 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:25:38.478823  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:38.486940  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.494637  5915 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:38.496431  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:38.499468  6015 raft_consensus.cc:2410] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:38.457789  8130 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:38.503144  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:38.477641  8126 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:38.477905  8117 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.507205  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:38.510368  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:38.458534  7974 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:38.528029  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.517048  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:38.522168  5915 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:38.527004  5943 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:38.483072  8015 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:25:38.480808  8129 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:38.505997  7499 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:38.546365  6015 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:38.518935  8064 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:38.557273  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.559214  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:38.503823  8016 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:25:38.554649  5948 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:25:38.578388  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:38.579276  5948 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:38.530668  8011 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:38.574252  6015 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:38.551221  7499 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.585732  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.586534  6021 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:38.582073  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.483165  8124 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:38.544183  7973 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.587512  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.519532  8152 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 12:25:38.590330  6021 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:38.555697  7607 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:38.599258  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:38.600461  5943 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 7: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:38.532333  8117 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:38.608132  6021 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:25:38.601532  8124 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:38.587337  8128 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:38.609979  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 92898.9us
W20260709 12:25:38.608986  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 124395us
I20260709 12:25:38.587070  7499 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:38.595919  8015 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:38.619046  6611 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:38.619112  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:38.627918  5943 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 3: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:38.628328  5948 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
W20260709 12:25:38.628551  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 106498us
I20260709 12:25:38.630101  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:38.631662  6018 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:38.633616  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:38.625828  8015 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:38.640280  8162 raft_consensus.cc:2764] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:25:38.601917  7973 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:38.620532  8063 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:38.642547  5915 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:38.635205  8011 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:38.663513  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:38.666600  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:38.651669  6018 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:38.668267  8035 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:38.665498  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:38.669648  5943 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:38.637807  8008 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:38.674664  8165 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:38.674942  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:38.647625  7840 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:38.678375  8129 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:38.680408  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.707139  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.707504  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:38.707811  6018 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:38.709065  6018 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:38.671450  7607 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:25:38.681500  5868 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 296) took 1596 ms (client timeout 1703 ms). Trace:
I20260709 12:25:38.682200  8011 raft_consensus.cc:697] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:25:38.682516  6022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 638) took 1579 ms (client timeout 1711 ms). Trace:
I20260709 12:25:38.684240  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:38.682327  8154 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 12:25:38.685973  8156 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:25:38.627879  8126 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:25:38.686980  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.686328  8015 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:38.688133  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 293) took 1604 ms. Trace:
W20260709 12:25:38.688975  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 61919us
I20260709 12:25:38.689931  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:38.648198  8009 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:38.647881  8164 consensus_queue.cc:1059] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 12:25:38.722656  8015 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:38.725234  5943 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 12:25:38.726014  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 298) took 1072 ms. Trace:
I20260709 12:25:38.726166  6019 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:38.726411  5943 rpcz_store.cc:276] 0709 12:25:37.653860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.724428 (+1070568us) service_pool.cc:224] Handling call
0709 12:25:38.724900 (+   472us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:38.725701 (+   801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.725942 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:38.727844  5943 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:25:38.728461  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 300) took 1058 ms. Trace:
I20260709 12:25:38.728823  5943 rpcz_store.cc:276] 0709 12:25:37.670473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.727249 (+1056776us) service_pool.cc:224] Handling call
0709 12:25:38.727608 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.728220 (+   612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.728408 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:38.730043  5868 rpcz_store.cc:269] 0709 12:25:37.085469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.122064 (+ 36595us) service_pool.cc:224] Handling call
0709 12:25:38.681283 (+1559219us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b54002f94d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:38.681428 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1417030016}
W20260709 12:25:38.716276  8160 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:25:38.715391  7973 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:38.731947  6022 rpcz_store.cc:269] 0709 12:25:37.102610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.104276 (+  1666us) service_pool.cc:224] Handling call
0709 12:25:38.682310 (+1578034us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b5400763754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:38.682449 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1696240384}
I20260709 12:25:38.729904  5866 rpcz_store.cc:276] 0709 12:25:37.083264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.083414 (+   150us) service_pool.cc:224] Handling call
0709 12:25:37.124611 (+ 41197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.576116 (+451505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.687842 (+1111726us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b54004f60d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:38.688063 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":890255232}
I20260709 12:25:38.722802  8011 consensus_queue.cc:237] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:38.736289  7974 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:38.681661  8150 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 12:25:38.670853  8063 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:38.740011  6018 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:38.743089  5943 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:38.750814  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:38.751914  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 125358us
I20260709 12:25:38.751250  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:38.752034  6785 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:38.772104  5943 pending_rounds.cc:85] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 1
I20260709 12:25:38.772548  5943 pending_rounds.cc:107] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:25:38.772841  5943 raft_consensus.cc:2889] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:38.689079  8035 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:38.773959  8035 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:38.774416  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:38.729446  8117 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:38.704499  8165 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:38.786391  6785 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:38.789368  6766 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:38.796070  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:38.796921  6766 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:25:38.783833  8036 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:25:38.799844  6775 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:38.805830  8168 consensus_queue.cc:1059] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:25:38.790411  8165 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:38.783849  8169 raft_consensus.cc:3055] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:38.783257  8063 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:38.815212  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 62257.2us
I20260709 12:25:38.818773  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:38.813033  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 291) took 1719 ms. Trace:
I20260709 12:25:38.818632  8169 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:38.823336  5947 rpcz_store.cc:276] 0709 12:25:37.094116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.094333 (+   217us) service_pool.cc:224] Handling call
0709 12:25:38.812813 (+1718480us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54006a3bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:38.812962 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407839104}
I20260709 12:25:38.834362  5947 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:38.835997  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:38.837968  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:38.840627  6773 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:38.844311  5947 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:38.826045  7799 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:38.854576  8117 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:38.855762  6022 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:38.860503  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:38.840023  5941 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:38.862018  8011 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:38.862389  8169 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:38.863798  5941 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:38.863950  8169 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:38.865321  5941 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:38.865432  8139 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:38.866348  5948 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:25:38.875771  5948 pending_rounds.cc:85] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 2
I20260709 12:25:38.876137  5948 pending_rounds.cc:107] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:25:38.876425  5948 raft_consensus.cc:2889] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:38.879699  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 151413us
I20260709 12:25:38.866237  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:38.874368  5940 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:38.857061  7499 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:38.893283  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.894569  6015 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:38.896041  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:38.896209  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:38.902091  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:38.903256  6015 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:38.907034  6609 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:38.894435  8011 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:38.900116  7799 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:25:38.911746  8011 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:38.912283  8011 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:38.913393  8035 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:38.916005  7499 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:38.875049  8015 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:25:38.922083  5948 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:25:38.925558  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:38.900326  8174 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:38.933595  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.898428  7750 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:25:38.940599  6015 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:38.940119  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:38.942126  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:38.943337  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:38.945182  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:38.946074  6015 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 7: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:38.903393  8162 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:25:38.951248  6016 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:38.951108  6766 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:38.936390  8168 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:38.955726  6757 raft_consensus.cc:3055] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:38.957067  6757 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:38.959597  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:38.966061  5868 raft_consensus.cc:2376] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7. Re-sending same reply.
I20260709 12:25:38.928020  7738 consensus_queue.cc:1059] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 12:25:38.968246  6015 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:38.973013  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:38.974107  5866 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 3: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:38.957989  6757 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:38.967432  8036 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:25:38.981539  6757 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:38.990203  6023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 642) took 1886 ms (client timeout 2077 ms). Trace:
W20260709 12:25:38.998121  6023 rpcz_store.cc:269] 0709 12:25:37.103794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.120005 (+ 16211us) service_pool.cc:224] Handling call
0709 12:25:38.989857 (+1869852us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b5400171bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:38.990104 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2051910912}
I20260709 12:25:38.993158  6020 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:38.967752  8034 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:38.958879  8173 raft_consensus.cc:2764] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:25:38.967516  8015 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:38.980067  8165 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:39.001922  8125 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 1.487s	user 0.007s	sys 0.009s
I20260709 12:25:38.983580  7799 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.987093  8126 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:39.004283  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:39.006232  8178 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:25:39.015468  6757 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:38.977576  7741 raft_consensus.cc:3055] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:38.951220  8180 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 12:25:38.983343  7773 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:38.995223  8114 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.967752  8176 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 12:25:39.007747  8034 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:39.038417  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1005) took 5074 ms. Trace:
I20260709 12:25:39.041903  5947 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:39.046908  5944 rpcz_store.cc:276] 0709 12:25:33.962981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.963181 (+   200us) service_pool.cc:224] Handling call
0709 12:25:33.974458 (+ 11277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.647388 (+3672930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.037055 (+1389667us) spinlock_profiling.cc:243] Waited 3.64 s on lock 0x7b5400467dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:39.037330 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8378194688}
I20260709 12:25:39.020049  8015 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:39.029464  7741 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:25:39.051321  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.030015  5941 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
W20260709 12:25:39.052505  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 142434us
I20260709 12:25:39.043380  7904 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:39.034567  5867 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:45236
I20260709 12:25:39.055219  5944 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:39.055083  6611 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:39.055384  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:39.028378  8168 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:39.058970  8176 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.059528  8176 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:39.059420  8168 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:39.060413  8186 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:39.060909  8186 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.061259  8186 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:39.041817  8114 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:39.063066  8165 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:39.063782  8114 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:39.056453  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.027989  7799 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:39.057289  5867 raft_consensus.cc:606] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:25:39.066819  7741 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:39.050766  8015 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:39.069618  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:39.070832  5868 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:39.069862  5867 raft_consensus.cc:614] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:39.071967  6609 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:39.074211  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 285) took 2000 ms. Trace:
I20260709 12:25:39.041519  8011 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:39.083003  8117 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:39.083567  8117 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.083901  8117 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:39.087198  8035 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:39.089466  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 302) took 1467 ms. Trace:
I20260709 12:25:39.090067  5870 rpcz_store.cc:276] 0709 12:25:37.621602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.874447 (+252845us) service_pool.cc:224] Handling call
0709 12:25:37.874808 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.378664 (+503856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.089166 (+710502us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b54002f9ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:39.089390 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1157149952}
I20260709 12:25:39.091995  8015 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:39.093891  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:39.095356  7799 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:39.042641  8034 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:39.077035  5949 rpcz_store.cc:276] 0709 12:25:37.073548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.073789 (+   241us) service_pool.cc:224] Handling call
0709 12:25:37.108193 (+ 34404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:37.126221 (+ 18028us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.126254 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:37.143472 (+ 17218us) log.cc:844] Serialized 26 byte log entry
0709 12:25:37.262937 (+119465us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:37.263005 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.263131 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.984872 (+1721741us) spinlock_profiling.cc:243] Waited 5.6 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:39.068387 (+ 83515us) raft_consensus.cc:1685] finished
0709 12:25:39.068412 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.074120 (+  5708us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18503,"raft.queue_time_us":17861,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":16727,"spinlock_wait_cycles":12878720,"thread_start_us":56455,"threads_started":1,"wal-append.queue_time_us":276996}
I20260709 12:25:39.097085  5866 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 12:25:39.097383  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 942754us
I20260709 12:25:39.092273  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 765) took 1035 ms. Trace:
I20260709 12:25:39.098627  5942 rpcz_store.cc:276] 0709 12:25:38.056822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.733563 (+676741us) service_pool.cc:224] Handling call
0709 12:25:38.813560 (+ 79997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.813820 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.813844 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:38.813887 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:38.813921 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.813976 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.091710 (+277734us) spinlock_profiling.cc:243] Waited 79.6 ms on lock 0x7b54006a3bd0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:39.091847 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183101184}
I20260709 12:25:39.096488  8034 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:39.106647  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 51655.3us
I20260709 12:25:39.075213  7904 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:39.069619  8176 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:39.107801  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:39.108204  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:39.108706  5944 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:39.116328  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:39.117113  6021 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 12:25:39.117362  6023 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:39.118691  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:39.119480  6021 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:39.120246  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:39.108664  7799 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:25:39.124341  8011 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:39.125254  8187 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:39.128564  8176 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:39.129902  7904 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:39.130395  8063 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:39.137836  5870 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:39.138289  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:39.120141  7741 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:39.147521  6023 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:39.108090  8036 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:25:39.158140  8015 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:39.111330  8114 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:25:39.144138  7799 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:39.168845  5840 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:39.169518  7973 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:39.175825  7904 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:39.191742  8190 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.192740  8170 raft_consensus.cc:3055] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:39.192788  8190 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:39.196041  7799 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:39.197114  8055 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:39.197431  8190 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:39.197207  7799 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:39.197892  8055 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:39.198360  8055 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:39.198971  8185 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:25:39.202102  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:39.202888  5870 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:39.204397  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:39.205495  8189 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.182s	user 0.001s	sys 0.002s
I20260709 12:25:39.205600  8190 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:39.206139  8190 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.206625  8190 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:39.206974  8162 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:39.207494  8162 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:39.208384  8162 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:39.210070  5867 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:45236
I20260709 12:25:39.210733  5867 raft_consensus.cc:606] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:39.210500  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:39.157119  8117 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:39.151556  8063 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:39.216521  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:39.218870  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:39.219219  7898 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:39.219583  6023 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:39.219612  8014 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:39.220829  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 787) took 1782 ms. Trace:
I20260709 12:25:39.221437  5867 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60934281a651439a87cd8c11702f67d2"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
mode: GRACEFUL
new_leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
 from {username='slave'} at 127.0.0.1:45236
I20260709 12:25:39.222138  5867 raft_consensus.cc:606] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Received request to transfer leadership to TS 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:39.223817  5867 raft_consensus.cc:614] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:39.222250  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:39.221273  6016 rpcz_store.cc:276] 0709 12:25:37.438814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.439178 (+   364us) service_pool.cc:224] Handling call
0709 12:25:39.220618 (+1781440us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54001725d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:39.220762 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3313572096}
I20260709 12:25:39.228293  8183 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:39.211585  8170 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:39.228698  6021 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:39.152017  8187 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 12:25:39.230687  5840 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:39.216435  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.228951  8183 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:39.249886  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:39.252846  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:39.256043  7974 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:39.256541  7974 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:39.256986  7974 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:39.266621  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:39.219861  8186 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:39.241765  7973 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:39.237166  7898 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:39.245057  8129 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:39.280001  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 111802us
I20260709 12:25:39.270684  7592 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:39.270489  8162 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:39.278312  8188 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:39.282296  5990 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:39.244843  8184 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:25:39.277385  8114 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:39.292310  5867 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
mode: GRACEFUL
new_leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
 from {username='slave'} at 127.0.0.1:45236
I20260709 12:25:39.294867  5867 raft_consensus.cc:606] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Received request to transfer leadership to TS 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:39.295286  5867 raft_consensus.cc:614] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:39.282905  7896 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:39.282490  7955 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:39.295480  8194 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:25:39.328135  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:39.329159  5870 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:39.290249  7592 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:39.296222  8188 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:39.296788  8063 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:39.314399  8183 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:39.298638  8100 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:39.315371  7974 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:39.316457  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:39.316987  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:39.356566  7973 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 5 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:39.318212  8176 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:39.321321  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:39.357616  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:39.322113  8184 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:39.323279  8186 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:39.323372  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:39.324836  8174 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:39.325871  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:39.272292  8170 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:39.360941  7799 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:39.363008  7799 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:39.363385  8170 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:39.365836  7799 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:39.367659  5867 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
mode: GRACEFUL
new_leader_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:45236
I20260709 12:25:39.368533  5867 raft_consensus.cc:606] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Received request to transfer leadership to TS 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:39.357707  7973 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:39.361289  8174 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:39.370541  8174 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:39.331440  6609 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:39.328239  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:39.332738  8193 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:39.372737  6766 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:39.332847  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:39.373720  6778 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:39.336823  5868 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:39.337733  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:39.374367  7898 raft_consensus.cc:993] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: : Instructing follower 1fc57407d56241b38a94d543e073f8af to start an election
I20260709 12:25:39.375064  5870 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:39.343501  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:39.299803  8036 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:39.355880  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 57119.5us
W20260709 12:25:39.355492  8198 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.111s	user 0.002s	sys 0.000s
I20260709 12:25:39.360214  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:39.342424  7592 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:39.375931  6793 raft_consensus.cc:2410] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:39.326918  7955 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:39.376601  5990 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:39.358541  8184 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:39.371891  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:39.354230  8100 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.386528  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:39.386982  7955 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:39.387894  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 365) took 1302 ms. Trace:
I20260709 12:25:39.390962  8188 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:39.391489  8117 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:39.380617  8036 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:39.420276  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.394856  6024 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:39.395382  7974 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:39.395749  8068 raft_consensus.cc:3055] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:39.384025  8183 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:39.404539  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:39.404935  5840 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:39.405980  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:39.374941  7898 raft_consensus.cc:1081] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Signalling peer 1fc57407d56241b38a94d543e073f8af to start an election
I20260709 12:25:39.415370  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:39.410166  5867 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:39.409360  6020 rpcz_store.cc:276] 0709 12:25:38.085348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.252482 (+167134us) service_pool.cc:224] Handling call
0709 12:25:38.992853 (+740371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.000656 (+  7803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.387730 (+387074us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b5400171bd0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:39.387841 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1701491456}
I20260709 12:25:39.418432  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:39.429491  5840 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:39.429803  5874 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:39.418841  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:39.431345  5840 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:39.397323  7592 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:39.394649  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 789) took 1742 ms. Trace:
I20260709 12:25:39.437172  6017 rpcz_store.cc:276] 0709 12:25:37.652411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.652555 (+   144us) service_pool.cc:224] Handling call
0709 12:25:39.394411 (+1741856us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b5400160654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:39.394583 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1195882752}
I20260709 12:25:39.424324  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:39.425511  5870 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:39.399544  8184 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:39.442112  8184 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:39.408636  8100 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:39.446708  5870 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:39.453254  6609 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:39.484042  6768 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:39.414105  7799 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:25:39.486117  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.384326  8129 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.488153  8186 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:39.419664  8199 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:39.488317  8114 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:39.490460  8195 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:25:39.408975  8200 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:39.425098  8068 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:39.414297  8034 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 12:25:39.501888  6775 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
W20260709 12:25:39.502735  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.433701  8162 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:39.482230  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:39.482048  7974 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:39.451031  8201 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:39.525875  8068 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:39.527302  8068 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:39.530117  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 588035us
I20260709 12:25:39.536911  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:39.538760  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:39.540411  5915 leader_election.cc:336] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.938s (SENT)
I20260709 12:25:39.507652  8186 raft_consensus.cc:697] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:39.543619  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:39.545962  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:39.547971  5942 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:39.554958  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:39.556299  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:39.553220  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:39.513872  8200 raft_consensus.cc:697] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:39.559592  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:39.501228  8129 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:39.559677  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:39.560145  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:39.501058  8055 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:39.562870  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:39.564900  5949 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:39.522006  8176 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.568775  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:39.564661  5940 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:39.571520  5838 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:39.571767  5942 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:39.571571  5940 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:39.504559  7799 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:39.573966  5838 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:39.555907  7750 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:39.506664  8199 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:39.576030  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 144725us
I20260709 12:25:39.555634  8011 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.520653  8162 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:39.555295  8063 consensus_queue.cc:1059] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:25:39.583525  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:39.567353  7973 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:25:39.560141  8015 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
W20260709 12:25:39.599131  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 55628.2us
I20260709 12:25:39.583241  7974 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:39.603091  7773 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:39.608153  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 59456.7us
W20260709 12:25:39.609617  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.610594  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:39.578402  8176 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:39.619354  5871 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:39.621977  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:39.624078  5874 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:25:39.624655  5867 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:39.628712  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:39.630067  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:39.602952  7898 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:39.632244  5867 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:39.607880  7974 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:39.593909  8011 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:39.644407  8208 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:39.648670  5940 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:39.613507  7750 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:39.617698  8187 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:39.645047  8210 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.667936  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:39.670471  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:39.672941  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:39.626096  8063 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:39.676317  5915 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:39.606768  8186 consensus_queue.cc:237] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:39.634168  8035 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.639093  7898 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.681461  5987 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:39.678395  5915 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:39.680485  8124 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:39.684454  7898 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:39.686009  8199 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:39.687940  8176 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:39.671658  8165 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:39.703305  8011 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:39.704411  7799 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:39.705099  7750 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:39.633769  7773 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:25:39.680030  8210 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:39.706703  8207 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:39.651441  7974 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:39.709834  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:39.645047  8055 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:39.740314  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:39.709092  8124 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:39.713122  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:39.719600  5949 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:39.673022  7896 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:25:39.709511  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 80884.1us
W20260709 12:25:39.727432  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 153536us
W20260709 12:25:39.775547  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:39.728147  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 49873.3us
I20260709 12:25:39.679745  8063 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.734380  5873 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:39.682600  8190 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:39.712792  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 328327us
I20260709 12:25:39.781070  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:39.782884  5840 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:39.783217  5869 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:39.784094  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:39.784740  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:39.743281  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:39.735875  7893 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:25:39.733144  8205 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:25:39.789120  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:39.755241  8124 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:39.771246  5874 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:39.732373  8199 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.708194  8035 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:39.739706  7955 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:39.721912  8175 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:39.790365  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:39.791949  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:39.803748  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:39.816838  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:39.651207  8200 consensus_queue.cc:237] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:39.819448  6785 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:39.808913  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:39.819465  5942 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:39.821187  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:39.822355  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:39.822887  5942 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:39.817574  5866 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:39.789472  8063 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:39.804196  8204 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:25:39.784799  8055 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.785526  8129 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:39.828259  8165 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:39.805202  8187 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:25:39.829262  8214 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:25:39.829416  8100 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:39.790611  8207 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:39.800215  7973 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.790093  8215 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:39.801041  8144 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:39.800405  7799 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:25:39.830353  7955 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:25:39.830353  8212 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:25:39.817720  8162 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:39.927939  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1045) took 1323 ms. Trace:
I20260709 12:25:39.928503  5948 rpcz_store.cc:276] 0709 12:25:38.604934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.924460 (+319526us) service_pool.cc:224] Handling call
0709 12:25:38.939688 (+ 15228us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.528914 (+589226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.528946 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:39.529526 (+   580us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.553076 (+ 23550us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:39.553152 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.553250 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.927475 (+374225us) raft_consensus.cc:1685] finished
0709 12:25:39.927507 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.927837 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":620894,"raft.queue_time_us":601801,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":9317,"spinlock_wait_cycles":57728,"thread_start_us":23058,"threads_started":1,"wal-append.queue_time_us":137898}
I20260709 12:25:39.931897  5948 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:39.826045  8011 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:39.810259  8035 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:39.812151  8117 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:25:39.943514  8117 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:39.944032  8117 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:39.834630  5941 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:39.801731  7896 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:25:39.945219  8117 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:39.948305  8117 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:39.953929  8190 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:39.837003  8211 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:39.916040  8209 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:25:39.955350  8211 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:39.955823  8211 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:39.908615  8207 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.957408  8207 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:39.955549  8210 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:39.915501  7955 raft_consensus.cc:697] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:25:39.922142  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.836339  8055 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:39.880856  8187 raft_consensus.cc:697] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:39.845203  8215 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:39.808540  8199 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:39.841533  8129 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:39.965103  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:39.881945  7973 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:39.962945  8114 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:39.940855  8100 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:39.912472  8144 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:39.982492  5869 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
W20260709 12:25:39.973026  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:39.976323  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 172609us
I20260709 12:25:39.978188  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:39.984797  8207 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:39.985646  8152 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.986346  8152 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:39.990141  7896 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.990896  7896 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:39.989501  8152 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:39.883543  8218 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.127s
W20260709 12:25:39.966979  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 180009us
I20260709 12:25:39.998003  7896 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:39.997804  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:39.975431  7738 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:39.967993  8215 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:40.013023  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:40.016494  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:40.017066  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:40.018321  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:40.019476  7799 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.019397  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:40.025480  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 297) took 2380 ms. Trace:
I20260709 12:25:40.020172  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:40.035737  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:40.038408  5866 raft_consensus.cc:2393] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 7: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:40.022323  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:40.035626  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:40.039148  6016 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:40.039355  6020 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:25:40.040097  5874 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:40.037334  5946 rpcz_store.cc:276] 0709 12:25:37.645337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.024507 (+379170us) service_pool.cc:224] Handling call
0709 12:25:38.043189 (+ 18682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.066136 (+ 22947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.066163 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:38.066199 (+    36us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:38.066223 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.025285 (+1959062us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b5400459cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:40.025328 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.025429 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52242304}
I20260709 12:25:40.043874  6019 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 5: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:40.056432  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:39.967653  8208 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:25:39.963091  8187 consensus_queue.cc:237] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:40.024492  8114 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:40.070750  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:40.026563  8100 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:40.012863  7738 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.080969  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:40.081954  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:40.083055  6023 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:40.056428  8055 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:40.081058  5987 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:40.100256  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:40.085975  7738 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:40.112852  8142 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 2.079s	user 0.013s	sys 0.015s
I20260709 12:25:40.097136  8224 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:25:40.114276  8225 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:40.114719  8225 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:40.115059  8225 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.113698  8218 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:40.116891  8218 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.117283  8218 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.045279  7799 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:40.104194  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:40.120441  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:40.127463  6023 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.121758  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:40.078866  8114 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.036330  7955 consensus_queue.cc:237] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:40.078145  8200 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:25:40.137452  5874 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:40.102340  8217 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:40.144169  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:40.126994  8224 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:40.138579  8114 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:40.151988  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 69538.2us
I20260709 12:25:40.158921  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:40.160904  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:40.092522  8207 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:25:40.093914  8100 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:40.139838  8170 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.166115  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:40.100483  8210 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:25:40.146056  7871 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:40.149755  7592 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:40.150887  8186 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:40.145257  8228 raft_consensus.cc:2764] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:25:40.174640  8055 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:40.163435  8227 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:25:40.178485  8230 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 12:25:40.180894  8222 log.cc:927] Time spent T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:25:40.189919  7871 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:40.192121  7974 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:40.194011  7799 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:40.123909  8144 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:40.198134  7973 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:40.198894  8130 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:40.199990  7996 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:40.203068  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 361) took 2118 ms. Trace:
I20260709 12:25:40.248502  6024 rpcz_store.cc:276] 0709 12:25:38.084224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.169342 (+ 85118us) service_pool.cc:224] Handling call
0709 12:25:39.394559 (+1225217us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.421857 (+ 27298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.202798 (+780941us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400160650. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:40.202991 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2816700672}
I20260709 12:25:40.204900  8201 raft_consensus.cc:3055] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:40.205775  8124 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:40.209052  5865 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:40.211488  8100 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:40.214998  7738 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:40.218197  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 301) took 2101 ms. Trace:
I20260709 12:25:40.268033  5947 rpcz_store.cc:276] 0709 12:25:38.117181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.845528 (+728347us) service_pool.cc:224] Handling call
0709 12:25:38.846110 (+   582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.048669 (+202559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.217899 (+1169230us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b5400632c54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:40.218120 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449665920}
I20260709 12:25:40.270619  8225 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:40.219502  8183 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:40.273226  8183 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:40.277735  8073 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:40.278453  8073 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:40.279346  8124 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:40.280843  8073 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:40.281327  7499 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:40.254206  8201 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:40.281539  8036 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:40.269336  7738 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:40.287516  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:40.223881  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:40.174943  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:40.230445  8182 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 1.226s	user 0.004s	sys 0.007s
I20260709 12:25:40.231191  8126 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:40.168685  8217 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:40.221191  8114 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:40.282902  7898 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.549s	user 0.013s	sys 0.000s
W20260709 12:25:40.251598  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 129934us
I20260709 12:25:40.197216  8063 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:40.176095  8170 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:40.259558  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:40.238442  8186 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:40.228036  8215 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:40.267432  8179 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 1.263s	user 0.015s	sys 0.000s
I20260709 12:25:40.267201  7974 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:40.309794  8134 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.271274  6778 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:40.272287  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:40.288321  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:40.300956  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 126087us
W20260709 12:25:40.301321  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 77537.3us
I20260709 12:25:40.223587  6023 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:40.296829  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:40.311097  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:40.319453  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.319609  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:40.294109  8162 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.293915  8036 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:40.309619  7996 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:40.219128  8199 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:40.420852  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:40.421336  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.439996  7436 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:40.440428  5913 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:40.441418  8011 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:40.441326  8035 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:40.442919  8011 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:40.314231  7711 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.314016  8126 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:40.444243  7711 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:40.447520  7711 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:40.310909  7973 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:40.425177  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:40.449110  7973 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:40.426079  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.449585  7973 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:40.450099  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.281910  7499 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:40.451040  5840 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:40.451958  8217 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:40.452411  8217 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.452755  8217 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.427747  5867 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:40.454826  7974 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:40.453114  7499 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:40.459120  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:40.459892  5874 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:40.461378  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:40.461930  5913 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:40.458774  5868 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:40.464363  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:40.285409  8218 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:40.456959  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:40.430047  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 110549us
I20260709 12:25:40.432005  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:40.475826  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:40.476931  6756 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:40.479717  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 676) took 1786 ms. Trace:
I20260709 12:25:40.433702  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:40.443373  8011 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.443349  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:40.448416  8152 raft_consensus.cc:3055] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:40.519454  8152 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:40.480361  6762 rpcz_store.cc:276] 0709 12:25:38.693040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.694091 (+  1051us) service_pool.cc:224] Handling call
0709 12:25:38.730251 (+ 36160us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.903957 (+173706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.904009 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:38.904779 (+   770us) log.cc:844] Serialized 26 byte log entry
0709 12:25:38.974454 (+ 69675us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:38.974499 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.974579 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.229699 (+1255120us) spinlock_profiling.cc:243] Waited 40.7 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:40.479389 (+249690us) raft_consensus.cc:1685] finished
0709 12:25:40.479413 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.479652 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89397,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":158,"spinlock_wait_cycles":93532544,"thread_start_us":229508,"threads_started":2,"wal-append.queue_time_us":86923}
I20260709 12:25:40.519621  8225 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:40.525202  8186 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:40.528159  8170 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:40.510180  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:40.530213  8215 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:40.530803  8230 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:25:40.532577  8114 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:40.552762  8011 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.619814  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:40.620782  5874 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.622690  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:40.626258  5872 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:25:40.631022  5990 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:40.639225  8225 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:40.422750  7893 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:40.643664  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:40.644455  6016 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 2.
I20260709 12:25:40.644410  7893 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:40.434664  8030 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:40.283663  8201 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:40.438220  8036 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:40.540759  8063 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:40.282464  7871 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:40.550606  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:40.662478  6023 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
W20260709 12:25:40.505749  8238 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
W20260709 12:25:40.555922  6775 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 585) took 1231 ms (client timeout 1581 ms). Trace:
I20260709 12:25:40.482265  6778 raft_consensus.cc:2410] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:25:40.497592  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:40.665941  6775 rpcz_store.cc:269] 0709 12:25:39.324950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.325555 (+   605us) service_pool.cc:224] Handling call
0709 12:25:40.555650 (+1230095us) spinlock_profiling.cc:243] Waited 59.6 ms on lock 0x7b540052a8d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:40.555836 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137082624}
I20260709 12:25:40.583596  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:40.324318  7898 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:40.585834  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:40.586048  6766 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:40.587275  5915 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:40.671737  6756 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:25:40.672266  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 85058us
I20260709 12:25:40.594081  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:40.597098  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:40.599871  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:40.603631  5868 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:25:40.606789  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:40.762342  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.668073  8173 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:25:40.608150  8211 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.651s	user 0.014s	sys 0.000s
I20260709 12:25:40.610790  8218 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:40.435882  8162 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:40.732404  8225 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:40.480796  7750 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:40.593652  8073 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:25:40.618055  7867 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:40.538844  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 381) took 2088 ms. Trace:
I20260709 12:25:40.645922  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:40.646843  5869 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:25:40.484707  7499 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:40.653918  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:40.667285  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:40.675657  8151 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.698s	user 0.101s	sys 0.164s
I20260709 12:25:40.701710  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:40.947803  6766 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:40.942430  6018 rpcz_store.cc:276] 0709 12:25:38.450663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.740980 (+290317us) service_pool.cc:224] Handling call
0709 12:25:38.741255 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.741673 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.741704 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:38.742184 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:25:38.929329 (+187145us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:38.929392 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.929503 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.266753 (+1337250us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:40.538390 (+271637us) raft_consensus.cc:1685] finished
0709 12:25:40.538428 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.538753 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1170,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":15482,"spinlock_wait_cycles":77412096,"thread_start_us":186671,"threads_started":1,"wal-append.queue_time_us":262152}
W20260709 12:25:40.956954  5865 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 882) took 1541 ms (client timeout 1520 ms). Trace:
W20260709 12:25:40.957662  5865 rpcz_store.cc:269] 0709 12:25:39.415150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.419530 (+  4380us) service_pool.cc:224] Handling call
0709 12:25:40.956664 (+1537134us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b540083f254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:40.956853 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1126073728}
I20260709 12:25:40.960111  8176 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.960807  8176 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:40.962404  8245 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.193s	user 0.000s	sys 0.000s
I20260709 12:25:40.938894  7499 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:40.922039  7994 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:40.964440  7994 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:40.971531  6768 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 3.
I20260709 12:25:40.767611  8170 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:40.578855  8144 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 12:25:40.736220  6019 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:40.744444  5871 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:40.737911  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:40.922821  8100 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:40.432790  8134 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:40.749325  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 886) took 1054 ms. Trace:
I20260709 12:25:41.023769  5867 rpcz_store.cc:276] 0709 12:25:39.695390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.695628 (+   238us) service_pool.cc:224] Handling call
0709 12:25:39.696034 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.454357 (+758323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.748844 (+294487us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b5400027954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:40.749196 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1682249472}
I20260709 12:25:40.750299  5874 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
W20260709 12:25:40.755941  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 282351us
I20260709 12:25:40.674892  7738 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.457708  8126 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:40.769560  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 172584us
I20260709 12:25:40.774899  8173 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:40.711416  8130 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:40.855839  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.744320  7711 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.484292  8199 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:41.061492  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 663) took 2566 ms. Trace:
W20260709 12:25:40.924882  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 325084us
I20260709 12:25:40.963761  7499 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:40.764600  8187 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:41.077149  7711 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:41.082046  8187 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:41.082788  8199 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:41.084208  8187 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:40.660394  8215 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:40.773276  8180 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:40.932107  8183 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.658s	user 0.010s	sys 0.004s
I20260709 12:25:40.739251  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:41.085021  8215 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.615643  8241 consensus_queue.cc:1059] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.112s
I20260709 12:25:40.930749  7799 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:40.965869  8151 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.988s	user 0.102s	sys 0.169s
I20260709 12:25:41.022539  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 309) took 2061 ms. Trace:
I20260709 12:25:40.772543  7893 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:40.947650  8218 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.038911  7738 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:41.041301  8126 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:41.042338  8240 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.307s	user 0.000s	sys 0.000s
I20260709 12:25:41.008108  8114 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:41.050190  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.093428  5945 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2.
I20260709 12:25:40.923242  8211 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:41.022276  8234 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.387s	user 0.001s	sys 0.001s
I20260709 12:25:41.068543  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:41.071136  8247 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.073992  8174 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:41.068802  8073 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.937528  8225 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:41.016850  8050 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.062263  6022 rpcz_store.cc:276] 0709 12:25:38.495169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.856683 (+361514us) service_pool.cc:224] Handling call
0709 12:25:39.221803 (+365120us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.222178 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:39.222588 (+   410us) log.cc:844] Serialized 14 byte log entry
0709 12:25:39.271034 (+ 48446us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:39.271109 (+    75us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:39.271153 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.271211 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.272781 (+1001570us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b18001acb00. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb642a6 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4
0709 12:25:41.061221 (+788440us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54001725d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:41.061404 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5600,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":119,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":6578,"spinlock_wait_cycles":845357824,"thread_start_us":47946,"threads_started":1,"wal-append.queue_time_us":77527,"wal-append.run_cpu_time_us":1681,"wal-append.run_wall_time_us":1109583}
I20260709 12:25:41.068604  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:40.965646  7994 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:41.157848  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 320) took 1646 ms. Trace:
W20260709 12:25:40.991575  6785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.072s	sys 0.147s
I20260709 12:25:40.934846  7750 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:41.159420  6785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.072s	sys 0.147s
W20260709 12:25:41.109041  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 430954us
I20260709 12:25:41.158617  5870 rpcz_store.cc:276] 0709 12:25:39.511493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.511647 (+   154us) service_pool.cc:224] Handling call
0709 12:25:39.511907 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.539557 (+ 27650us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:39.560683 (+ 21126us) log.cc:844] Serialized 14 byte log entry
0709 12:25:39.560870 (+   187us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:39.560927 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:39.560970 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.157407 (+1596437us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b54002f94d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:41.157480 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.157747 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":37692,"raft.queue_time_us":2532,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":1448,"spinlock_wait_cycles":68447872}
I20260709 12:25:41.014533  8170 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:41.112962  5944 rpcz_store.cc:276] 0709 12:25:38.961147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.057828 (+ 96681us) service_pool.cc:224] Handling call
0709 12:25:39.076017 (+ 18189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.115117 (+ 39100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.022086 (+1906969us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b5400672d54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:41.022425 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74407296}
I20260709 12:25:41.107591  8180 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:41.053583  8228 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:41.110023  8215 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:41.125052  6793 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:41.033398  8100 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:41.112603  8130 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:41.084215  7711 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:41.093714  8186 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.030258  8030 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:41.160524  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:41.225883  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 65456.5us
I20260709 12:25:41.227038  5915 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:41.161036  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 92539.7us
I20260709 12:25:41.136284  8241 raft_consensus.cc:3055] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:41.241813  8241 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:41.242923  8241 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:41.244904  8241 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:41.162415  8050 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:41.247961  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:41.248571  8050 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:41.251029  7750 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:41.251816  8130 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:41.179028  5913 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.520s (SENT) [suppressed 20 similar messages]
I20260709 12:25:41.128409  8218 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:41.253669  8134 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:41.254700  8218 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:41.183962  6762 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:25:41.256836  8180 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:41.153177  8247 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:41.167484  7898 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:41.234912  7893 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:41.236338  7738 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:41.162854  8073 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:41.239441  8201 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:41.239948  8030 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:41.268291  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.139452  8114 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:41.281524  6023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.021s	sys 0.060s
I20260709 12:25:41.257516  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:41.167413  8129 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:41.231485  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:41.259517  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1059) took 1238 ms. Trace:
I20260709 12:25:41.280905  7893 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:41.110231  8183 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:41.242652  8173 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:41.297765  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 66392.1us
I20260709 12:25:41.299335  5840 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:41.281975  6023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.021s	sys 0.060s
I20260709 12:25:41.084282  8199 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:41.285845  6768 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:25:41.162865  8036 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:41.290081  6785 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:41.290776  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:41.302929  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 691) took 1012 ms. Trace:
I20260709 12:25:41.303458  6785 rpcz_store.cc:276] 0709 12:25:40.290110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.289450 (+999340us) service_pool.cc:224] Handling call
0709 12:25:41.289890 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.302585 (+ 12695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.302856 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:41.245276  8100 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:41.271571  8225 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:41.302259  6768 pending_rounds.cc:85] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 2
W20260709 12:25:41.269464  8250 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:25:41.265586  8176 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:41.312093  6785 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
 from {username='slave'} at 127.0.0.1:43766
I20260709 12:25:41.301841  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 802) took 1281 ms. Trace:
I20260709 12:25:41.315956  6768 pending_rounds.cc:107] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:25:41.320377  6778 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:41.320822  6768 raft_consensus.cc:2889] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:41.320492  6785 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:41.321705  6785 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:41.322322  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 690) took 1032 ms. Trace:
I20260709 12:25:41.323455  6768 rpcz_store.cc:276] 0709 12:25:40.289858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.266124 (+976266us) service_pool.cc:224] Handling call
0709 12:25:41.266650 (+   526us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.321944 (+ 55294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.322250 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:41.170686  8228 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:41.321110  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 692) took 1030 ms. Trace:
I20260709 12:25:41.327659  6762 rpcz_store.cc:276] 0709 12:25:40.290385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.301615 (+1011230us) service_pool.cc:224] Handling call
0709 12:25:41.320570 (+ 18955us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.320774 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:41.320799 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:41.320844 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:41.320881 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.320940 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.321052 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2432}
W20260709 12:25:41.313817  8011 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.694s	user 0.013s	sys 0.001s
W20260709 12:25:41.328751  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.307492  8162 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:41.280218  8186 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:41.338799  6016 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:25:41.333174  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.333729  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:41.339943  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:41.335661  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:41.340809  6016 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:41.341470  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.342183  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.343046  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.336112  5915 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:41.336730  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.334247  6021 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:25:41.345145  6015 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:41.352180  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.347223  6015 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:41.352789  6762 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:41.353273  6022 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:41.358793  6016 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:41.330627  8173 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:41.342808  8243 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
W20260709 12:25:41.363161  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 327506us
I20260709 12:25:41.330590  8176 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:41.373647  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.378063  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:41.378964  6015 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:41.379976  8217 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.927s	user 0.005s	sys 0.009s
I20260709 12:25:41.381695  8217 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:41.382357  8183 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:41.363497  7738 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:41.384034  7738 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:41.357990  8139 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:41.385758  8244 log.cc:927] Time spent T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Append to log took a long time: real 0.181s	user 0.002s	sys 0.000s
I20260709 12:25:41.386365  7738 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:41.387687  8186 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:41.389544  5870 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:25:41.392257  6768 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:41.338553  8011 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:41.363448  7898 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:41.402566  6021 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:41.404233  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:41.380154  7973 consensus_meta.cc:287] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.927s	user 0.013s	sys 0.000s
I20260709 12:25:41.324326  7499 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:41.357003  8225 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:25:41.413805  6019 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:41.385888  8139 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.438777  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:41.360960  7867 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:25:41.445411  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:41.384035  8217 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:41.451635  8170 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:41.366671  8173 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:41.407857  8228 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:41.447619  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:41.470261  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:41.406469  8147 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:25:41.469172  5987 connection.cc:256] RPC call timeout handler was delayed by -1.9307306929999868s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20910000000000004s for a response
I20260709 12:25:41.379632  8176 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:41.367558  7893 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:41.454030  8139 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:41.446795  8130 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:41.484195  8114 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:41.487855  8130 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:41.488242  8225 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.488978  8225 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:41.490376  8130 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:41.470067  7898 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:41.488250  8030 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:41.493427  8225 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:41.494243  8011 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:41.491489  8152 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:41.495395  8152 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:41.500389  8211 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:41.502329  8249 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:25:41.502104  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.503293  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:41.503924  6021 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:41.507275  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.508077  6019 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:41.508983  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.440478  7973 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:41.510308  8247 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:41.510996  8256 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.511639  8256 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:41.512615  5866 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 6: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:41.511991  7973 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:41.484642  6022 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:41.515911  6019 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:41.530203  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:41.531344  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:41.467866  7880 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 12:25:41.542613  6022 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:25:41.544186  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:41.562659  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:41.566125  6021 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:41.579069  8184 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.484588  8147 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:41.658928  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 369117us
I20260709 12:25:41.613201  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.662879  6015 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:41.664821  6024 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:25:41.669780  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 490740us
I20260709 12:25:41.670650  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.659701  6785 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:41.670970  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:41.673940  6762 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:41.674476  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:41.677237  5874 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:41.677407  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.679881  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:41.680488  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:41.681211  6762 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:41.682207  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:41.682749  6762 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:41.682883  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:41.685384  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:41.688393  5874 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:41.690263  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:41.690389  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:41.691649  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.692322  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:41.692589  6021 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:41.596483  8144 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:41.696007  5942 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 6: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:41.658927  8247 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.703642  6785 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:41.700603  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.659792  8129 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 5 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:41.708230  5944 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:41.709231  8176 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:41.710683  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:41.711835  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:41.712145  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:41.712874  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:41.713213  5942 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
W20260709 12:25:41.714421  8251 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:41.714658  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:41.715137  7871 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:41.715488  7499 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:41.715623  7871 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:41.716815  8173 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:41.716593  7871 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:41.717406  8173 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.717926  8173 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:41.720278  8173 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:41.720639  8253 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.720971  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:41.721314  8253 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:41.721917  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:41.722657  5942 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:41.662070  8184 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:41.724646  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:41.732674  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:43766 (request call id 596) took 1233 ms. Trace:
I20260709 12:25:41.670006  8073 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:41.736483  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.736335  5990 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:41.800704  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:41.801010  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 64727.2us
I20260709 12:25:41.736994  5947 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:41.724875  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:41.729485  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.730888  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 685) took 1442 ms. Trace:
I20260709 12:25:41.784477  8180 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:41.785231  7499 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:41.730824  8152 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:25:41.788460  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:41.758798  6785 rpcz_store.cc:276] 0709 12:25:40.470989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.311888 (+840899us) service_pool.cc:224] Handling call
0709 12:25:41.704133 (+392245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:41.699131  7898 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.790680  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:41.791499  8035 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:41.792328  8130 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:41.794124  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.845355  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:41.723901  8144 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:41.799434  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.812983  6778 rpcz_store.cc:276] 0709 12:25:40.288546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.676738 (+388192us) service_pool.cc:224] Handling call
0709 12:25:40.740295 (+ 63557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.321044 (+580749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.730583 (+409539us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b54004088d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:41.730822 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1327184000}
I20260709 12:25:41.828979  8180 consensus_queue.cc:1243] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:41.702235  7973 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.853250  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:41.723827  8247 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:41.702641  8256 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:41.827347  7994 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:41.840303  5874 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:41.817334  8126 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:25:41.840276  6768 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:41.841322  6793 raft_consensus.cc:2410] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:25:41.293360  5941 rpcz_store.cc:276] 0709 12:25:40.021485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.022359 (+   874us) service_pool.cc:224] Handling call
0709 12:25:40.022982 (+   623us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:40.023594 (+   612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.023641 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:40.041363 (+ 17722us) log.cc:844] Serialized 52 byte log entry
0709 12:25:40.152578 (+111215us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:40.249834 (+ 97256us) log.cc:844] Serialized 14 byte log entry
0709 12:25:40.260156 (+ 10322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.303576 (+ 43420us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.634678 (+331102us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b18003995e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:25:41.259085 (+624407us) raft_consensus.cc:1685] finished
0709 12:25:41.259212 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.259408 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":78459,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":876,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":32517,"spinlock_wait_cycles":5119616,"thread_start_us":105426,"threads_started":1,"wal-append.queue_time_us":217870}
I20260709 12:25:41.819571  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:41.843310  8174 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.690s	user 0.010s	sys 0.003s
I20260709 12:25:41.839295  8211 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.842343  8036 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:25:41.837795  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 112987us
I20260709 12:25:41.819746  6015 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:41.928471  5945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 763) took 3872 ms (client timeout 2091 ms). Trace:
W20260709 12:25:41.946966  5945 rpcz_store.cc:269] 0709 12:25:38.056148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.381432 (+325284us) service_pool.cc:224] Handling call
0709 12:25:39.093703 (+712271us) spinlock_profiling.cc:243] Waited 6.99 ms on lock 0x7b5800030398. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:41.928202 (+2834499us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54001bdfd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:41.928359 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3612711168}
W20260709 12:25:41.901362  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.929194  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 689) took 1639 ms. Trace:
I20260709 12:25:41.949084  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 742) took 2329 ms. Trace:
I20260709 12:25:41.863517  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:41.876423  5840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.014s	sys 0.023s
W20260709 12:25:41.863071  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 461799us
I20260709 12:25:41.933736  6021 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:41.937794  5947 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:41.939505  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:41.949332  6766 rpcz_store.cc:276] 0709 12:25:40.289569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.954991 (+665422us) service_pool.cc:224] Handling call
0709 12:25:40.955372 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.085569 (+130197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:41.085607 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:41.085676 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:41.085713 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.928850 (+843137us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400007654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:41.928921 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.929115 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":298846976}
I20260709 12:25:41.927630  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 333) took 1305 ms. Trace:
I20260709 12:25:41.951136  5871 rpcz_store.cc:276] 0709 12:25:39.620787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.621212 (+   425us) service_pool.cc:224] Handling call
0709 12:25:39.621541 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.012130 (+1390589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.948748 (+936618us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400388954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:41.949012 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2581610240}
I20260709 12:25:41.955401  5866 rpcz_store.cc:276] 0709 12:25:40.622124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.622345 (+   221us) service_pool.cc:224] Handling call
0709 12:25:41.927281 (+1304936us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b540005dcd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:41.927521 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2043999488}
I20260709 12:25:41.956126  6757 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:25:41.956624  6757 pending_rounds.cc:85] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 2
W20260709 12:25:41.951452  5840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.015s	sys 0.029s
I20260709 12:25:41.956884  6757 pending_rounds.cc:107] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:25:41.957651  6757 raft_consensus.cc:2889] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:41.957959  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 658608us
I20260709 12:25:41.958873  5874 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:41.956766  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:41.963879  5913 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:41.845000  7898 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:41.968569  6016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.014s	sys 0.032s
W20260709 12:25:41.969112  6016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.014s	sys 0.032s
W20260709 12:25:41.919965  7750 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.007s	sys 0.006s
W20260709 12:25:41.919649  8244 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.004s	sys 0.011s
I20260709 12:25:41.973263  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:41.959501  5840 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:41.983183  6766 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:41.698683  8014 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:41.989809  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:41.991972  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:41.993084  5874 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:41.995415  5947 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:41.998652  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:41.999642  5874 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:41.988320  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:42.002676  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:42.003608  5946 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:25:42.003506  6018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.024s	sys 0.036s
I20260709 12:25:41.996915  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:41.936595  7994 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.848748  8130 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:42.008838  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:42.008984  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 49591.9us
I20260709 12:25:41.943847  7499 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:41.890194  8253 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:42.004529  6018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.024s	sys 0.036s
I20260709 12:25:41.943542  8073 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.984041  5944 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:41.961076  8192 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.960768  8016 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:42.020480  5866 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
W20260709 12:25:41.977605  7750 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.008s	sys 0.007s
I20260709 12:25:42.026890  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:42.027676  5944 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:41.972748  8011 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:25:42.006126  8244 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.005s	sys 0.011s
I20260709 12:25:41.858033  7973 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:42.032471  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:42.007606  8014 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:42.042474  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:41.860020  8259 consensus_queue.cc:1059] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.162s
I20260709 12:25:41.983204  8036 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:42.018278  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:41.957477  8211 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:42.028667  7799 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.902s	user 0.014s	sys 0.001s
I20260709 12:25:42.012962  7994 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:42.058617  5869 raft_consensus.cc:2410] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:42.016408  8266 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:42.109469  6785 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:25:42.112936  6785 pending_rounds.cc:85] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 4
I20260709 12:25:42.113826  6785 pending_rounds.cc:107] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:25:42.114218  6785 raft_consensus.cc:2889] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:42.111912  8266 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.056524  8011 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:42.102665  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:42.132452  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:42.132540  5947 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.133308  5944 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
W20260709 12:25:42.058516  8260 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
W20260709 12:25:42.068611  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 490370us
I20260709 12:25:42.028087  7499 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:41.983319  8129 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:42.108021  8210 raft_consensus.cc:3055] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:42.032007  8192 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:42.107782  7799 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:42.041991  8073 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:42.061820  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:42.138756  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 606) took 1001 ms. Trace:
I20260709 12:25:42.112370  8275 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:42.143836  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:42.150635  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:42.123458  8274 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:42.137486  8215 consensus_meta.cc:287] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.952s	user 0.014s	sys 0.000s
I20260709 12:25:42.063098  8256 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:25:42.107638  8180 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:25:42.155970  6778 rpcz_store.cc:276] 0709 12:25:41.137470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.850574 (+713104us) service_pool.cc:224] Handling call
0709 12:25:41.862107 (+ 11533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.862357 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:41.958070 (+ 95713us) log.cc:844] Serialized 14 byte log entry
0709 12:25:42.138376 (+180306us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:42.138432 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:42.138496 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.138566 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.138686 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8340,"prepare.queue_time_us":95905,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":10345,"raft.queue_time_us":97,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":5682,"thread_start_us":179921,"threads_started":1}
I20260709 12:25:42.132321  8173 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.135732  8147 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:42.129462  8266 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:42.163951  8269 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
I20260709 12:25:42.165465  8130 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:42.166172  8267 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 12:25:42.167519  7867 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:42.142047  8210 raft_consensus.cc:740] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:42.152129  8241 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:25:42.161290  8274 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.178295  8247 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:42.183377  8262 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 12:25:42.186429  6762 consensus_meta.cc:287] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.003s
I20260709 12:25:42.177955  8144 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:42.202327  6762 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:42.174458  8215 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:42.204437  7499 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:42.205183  8215 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:42.207381  8139 consensus_meta.cc:287] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.714s	user 0.012s	sys 0.000s
W20260709 12:25:42.207540  8270 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:25:42.157215  8036 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:42.207921  8139 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:42.210628  8139 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:42.197486  8011 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:42.215593  8201 consensus_meta.cc:287] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.927s	user 0.012s	sys 0.000s
I20260709 12:25:42.200189  8274 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.170630  8275 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:42.183040  8173 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:42.228202  8173 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:42.229815  8073 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:42.160141  8014 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:42.234059  8014 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.234217  8211 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:42.238389  7994 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:42.240794  7994 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:42.216965  8266 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:42.232822  8275 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:42.241400  8274 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:42.243782  8266 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:42.244618  8274 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:42.074164  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.165026  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1048) took 3244 ms. Trace:
I20260709 12:25:42.083946  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:42.295881  5874 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:42.297605  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:42.299667  5869 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:42.293216  5945 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:42.303524  5873 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:25:42.104794  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 86672.4us
I20260709 12:25:42.306398  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:42.308590  5868 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:42.310468  5867 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:42.294446  5943 rpcz_store.cc:276] 0709 12:25:38.920270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.046669 (+126399us) service_pool.cc:224] Handling call
0709 12:25:39.047211 (+   542us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.542265 (+495054us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:39.566555 (+ 24290us) log.cc:844] Serialized 14 byte log entry
0709 12:25:41.486707 (+1920152us) spinlock_profiling.cc:243] Waited 4.16 ms on lock 0x7b180028d380. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:25:41.936128 (+449421us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:41.948119 (+ 11991us) log.cc:844] Serialized 26 byte log entry
0709 12:25:41.948347 (+   228us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:41.948381 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.948482 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:42.164610 (+216128us) raft_consensus.cc:1685] finished
0709 12:25:42.164725 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.164938 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":564599,"prepare.queue_time_us":20454,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":512607,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":28952,"spinlock_wait_cycles":9563008,"thread_start_us":434103,"threads_started":1,"wal-append.queue_time_us":2459577}
I20260709 12:25:42.155375  5942 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:42.324004  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:42.338896  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 897) took 1050 ms. Trace:
I20260709 12:25:42.339418  5870 rpcz_store.cc:276] 0709 12:25:41.288090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.288245 (+   155us) service_pool.cc:224] Handling call
0709 12:25:41.288601 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:41.390024 (+101423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.338130 (+948106us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54007350d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:42.338638 (+   508us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231480192}
I20260709 12:25:42.340624  5874 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:42.346405  5869 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:42.346612  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.347919  6022 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:41.320676  6023 rpcz_store.cc:276] 0709 12:25:40.020422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.663616 (+643194us) service_pool.cc:224] Handling call
0709 12:25:40.664127 (+   511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.664449 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:40.702038 (+ 37589us) log.cc:844] Serialized 14 byte log entry
0709 12:25:41.301358 (+599320us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:41.301431 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:41.301490 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.301561 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.301756 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45182,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":5428,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":9291,"thread_start_us":598891,"threads_started":1}
I20260709 12:25:42.349979  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:42.350289  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:42.351531  5866 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 4: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:25:42.321427  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 177315us
I20260709 12:25:42.168999  5944 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:42.315404  5941 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:42.353865  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:42.326608  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:42.328329  6017 raft_consensus.cc:2368] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:25:42.355101  5869 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:42.356489  5872 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:42.329917  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:42.331593  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:42.333876  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:42.360464  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:42.362756  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:42.363251  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 55381.5us
I20260709 12:25:42.355124  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:42.365641  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.352880  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:42.368402  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:42.368697  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:42.369596  5948 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:42.370230  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:42.371017  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:42.371636  5940 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 4: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:42.372056  5948 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:42.372422  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:42.373093  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:42.372844  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:42.374883  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:42.375502  5940 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:42.370466  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:42.373975  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:42.378710  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:42.374209  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:42.383224  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 576) took 1023 ms. Trace:
I20260709 12:25:42.383369  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 699) took 1021 ms. Trace:
I20260709 12:25:42.383610  7994 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.384745  7994 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:42.218298  7898 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:42.388306  7994 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:42.384235  6766 rpcz_store.cc:276] 0709 12:25:41.361728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.010390 (+648662us) service_pool.cc:224] Handling call
0709 12:25:42.027178 (+ 16788us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:42.027798 (+   620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:42.027846 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:42.028534 (+   688us) log.cc:844] Serialized 26 byte log entry
0709 12:25:42.184605 (+156071us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:42.184673 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.184767 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:42.382429 (+197662us) raft_consensus.cc:1685] finished
0709 12:25:42.382503 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.383031 (+   528us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4395,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":15254,"thread_start_us":155652,"threads_started":1,"wal-append.queue_time_us":201827}
I20260709 12:25:42.389195  7994 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:42.389735  7994 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:42.390328  7994 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:42.393899  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:42.401211  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:42.404443  5943 raft_consensus.cc:2393] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 4: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:42.383234  7973 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:42.406594  5948 raft_consensus.cc:2410] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:42.399166  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:42.409852  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.410003  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.410702  8291 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:42.410960  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.412685  8036 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.413808  8036 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:42.425361  8036 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:42.421470  5874 raft_consensus.cc:2376] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6. Re-sending same reply.
I20260709 12:25:42.420284  5865 pending_rounds.cc:85] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 3
I20260709 12:25:42.427284  5865 pending_rounds.cc:107] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:25:42.427871  5865 raft_consensus.cc:2889] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:42.430311  8268 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:42.431286  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:42.433558  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:42.436722  8073 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:42.436966  8014 consensus_queue.cc:1059] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:42.437418  8289 consensus_queue.cc:1059] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:42.383903  6768 rpcz_store.cc:276] 0709 12:25:41.359562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.936472 (+576910us) service_pool.cc:224] Handling call
0709 12:25:41.977144 (+ 40672us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:42.014188 (+ 37044us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:42.014228 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:42.018228 (+  4000us) log.cc:844] Serialized 26 byte log entry
0709 12:25:42.199245 (+181017us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:42.199292 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.199364 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:42.382366 (+183002us) raft_consensus.cc:1685] finished
0709 12:25:42.382429 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.382879 (+   450us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43098,"raft.queue_time_us":37823,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":11729,"thread_start_us":134266,"threads_started":1,"wal-append.queue_time_us":195467}
I20260709 12:25:42.407048  5941 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:42.437606  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:42.466197  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.467696  7499 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:42.467780  6020 raft_consensus.cc:2410] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:25:42.468747  7499 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.469362  7499 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.470191  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:42.471336  6020 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:42.471918  7750 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:42.472352  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:42.439951  8014 raft_consensus.cc:3055] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:42.474306  6609 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:42.474717  8014 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:42.476357  8170 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:42.476588  6609 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:42.476922  8170 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:42.477483  8170 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:42.444450  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.455307  5942 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:42.486220  6021 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:42.462323  8073 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.465019  8016 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:42.490145  8016 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:42.490692  8014 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:42.496384  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.497010  8014 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.472383  7750 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:42.498179  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.498867  7750 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:42.500909  7750 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:42.502063  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:42.503180  6017 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:42.477054  8210 consensus_queue.cc:260] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:42.506553  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.507102  8210 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:42.507720  6024 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:42.508898  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.509366  8117 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:42.510030  8117 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.510466  8117 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.510345  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.512148  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:42.513005  6019 raft_consensus.cc:2376] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7. Re-sending same reply.
I20260709 12:25:42.513370  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:42.517654  6024 raft_consensus.cc:2368] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:25:42.440794  8139 raft_consensus.cc:3055] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:42.463083  8289 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.529335  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.458334  8011 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.522773  8139 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:42.532491  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:42.539320  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:42.540835  6762 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:25:42.541170  8246 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 1.902s	user 0.009s	sys 0.017s
I20260709 12:25:42.543795  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.542861  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:42.526342  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.557255  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.565907  8139 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:42.567164  8011 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:42.560159  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 63952.7us
I20260709 12:25:42.573016  8192 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:42.573213  8139 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:42.575161  6611 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:42.579346  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 568) took 2297 ms. Trace:
I20260709 12:25:42.579936  6773 rpcz_store.cc:276] 0709 12:25:40.281818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.282188 (+   370us) service_pool.cc:224] Handling call
0709 12:25:40.282579 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.283022 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.283054 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:40.283601 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 12:25:40.421857 (+138256us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:40.421909 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.421988 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:42.539897 (+2117909us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:42.578656 (+ 38759us) raft_consensus.cc:1685] finished
0709 12:25:42.578688 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.579109 (+   421us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1429,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":43529088,"thread_start_us":137891,"threads_started":1,"wal-append.queue_time_us":354545}
I20260709 12:25:42.582383  6022 raft_consensus.cc:3055] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:42.583084  6022 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:42.584661  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 688) took 2295 ms. Trace:
I20260709 12:25:42.584396  6022 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:42.546120  8073 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:42.586582  8162 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:42.556346  8289 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:42.621497  8073 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:42.621836  8271 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.452s	user 0.002s	sys 0.002s
I20260709 12:25:42.622653  6022 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.623828  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 681) took 1285 ms. Trace:
W20260709 12:25:42.625754  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 96893.8us
I20260709 12:25:42.612493  7994 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:42.628291  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.624310  6018 rpcz_store.cc:276] 0709 12:25:41.338566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.344337 (+  5771us) service_pool.cc:224] Handling call
0709 12:25:41.344704 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:41.359383 (+ 14679us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:41.359415 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:41.360603 (+  1188us) log.cc:844] Serialized 26 byte log entry
0709 12:25:42.032023 (+671420us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:42.032133 (+   110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.032261 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:42.620822 (+588561us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:42.623260 (+  2438us) raft_consensus.cc:1685] finished
0709 12:25:42.623282 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.623576 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14252,"raft.queue_time_us":93205,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":12771,"spinlock_wait_cycles":7430016,"thread_start_us":637905,"threads_started":1,"wal-append.queue_time_us":778986}
I20260709 12:25:42.645310  6015 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
W20260709 12:25:42.647416  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:42.650069  8282 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.484s	user 0.000s	sys 0.005s
I20260709 12:25:42.650480  7499 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:42.651741  8168 raft_consensus.cc:3055] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:42.652238  8168 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:42.636209  7898 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.654397  7898 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:42.594183  5947 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:42.585266  6775 rpcz_store.cc:276] 0709 12:25:40.289333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.940323 (+650990us) service_pool.cc:224] Handling call
0709 12:25:41.257758 (+317435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:41.258073 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:41.258103 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:41.258760 (+   657us) log.cc:844] Serialized 26 byte log entry
0709 12:25:41.567899 (+309139us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:41.567962 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.568070 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.861557 (+293487us) raft_consensus.cc:1685] finished
0709 12:25:41.861590 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.584027 (+722437us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b54002c24d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:42.584394 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":728844544,"thread_start_us":308703,"threads_started":1,"wal-append.queue_time_us":537629}
W20260709 12:25:42.600157  8261 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.755s	user 0.001s	sys 0.001s
I20260709 12:25:42.655793  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 590) took 3300 ms. Trace:
I20260709 12:25:42.595826  8176 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:42.606724  6021 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 8: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:42.607271  8259 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:42.666025  8259 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:42.657662  6757 rpcz_store.cc:276] 0709 12:25:39.355642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.357408 (+  1766us) service_pool.cc:224] Handling call
0709 12:25:39.357796 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.958151 (+2600355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.655223 (+697072us) spinlock_profiling.cc:243] Waited 2.59 s on lock 0x7b540004bfd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:42.655589 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5955354496}
I20260709 12:25:42.667626  8259 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:42.670145  8259 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.640458  8016 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:42.614141  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:42.617120  8011 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:42.618016  8144 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:42.672474  8253 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:42.673491  8253 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:42.619071  8272 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.471s	user 0.002s	sys 0.000s
I20260709 12:25:42.646646  8241 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.676569  8241 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:42.677963  6016 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:42.679486  8253 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:42.653453  7499 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:42.682518  8241 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:42.590873  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:42.657553  7898 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:42.625814  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.684840  6023 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:42.633955  6609 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:42.659348  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 687) took 2370 ms. Trace:
I20260709 12:25:42.686059  8147 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:42.603127  8184 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:42.686465  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 52690.1us
I20260709 12:25:42.686702  8147 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.660152  7871 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:42.640955  6024 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:42.688107  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:42.653522  8168 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:42.686308  6756 rpcz_store.cc:276] 0709 12:25:40.289058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.742711 (+453653us) service_pool.cc:224] Handling call
0709 12:25:40.770940 (+ 28229us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:41.088678 (+317738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:41.088713 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:41.108747 (+ 20034us) log.cc:844] Serialized 52 byte log entry
0709 12:25:41.462040 (+353293us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:41.462095 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.699413 (+237318us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b480052c158. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d7bacaa1 00007f39d7ba3b37 00007f39d7ba39ca 00007f39d7c9b26f 00007f39d7c9af37 00007f39d7caf55c 00007f39d7caf508 00007f39d7caf488 00007f39d7caf443 00007f39d7cae6ec 00007f39d6bd8a96
0709 12:25:42.436244 (+736831us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54002c1ad4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:42.436365 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:42.599121 (+162756us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:42.658846 (+ 59725us) raft_consensus.cc:1685] finished
0709 12:25:42.658874 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.659127 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61087,"raft.queue_time_us":2900,"raft.run_cpu_time_us":12748,"raft.run_wall_time_us":601513,"spinlock_wait_cycles":733393792,"thread_start_us":323685,"threads_started":1,"wal-append.queue_time_us":579074}
I20260709 12:25:42.689949  8184 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:42.606014  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:42.663466  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:42.663803  5946 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:42.690475  8184 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:42.691484  8184 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:42.690586  6609 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:42.657851  8176 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:42.693410  8252 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:42.693946  8252 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.694386  8252 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.694664  8176 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:42.671588  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 57796.1us
I20260709 12:25:42.618468  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:42.672216  8274 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:42.687804  8147 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:42.706936  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 88902us
I20260709 12:25:42.705389  8256 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:25:42.708417  7499 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:42.711170  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.690663  8168 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:42.712400  8256 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:42.696686  8176 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.711611  5838 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:42.684979  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 116089us
I20260709 12:25:42.716640  5865 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
W20260709 12:25:42.745011  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 79105.1us
I20260709 12:25:42.738031  6609 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:42.713083  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 340683us
I20260709 12:25:42.725118  6022 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:25:42.743613  6017 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:42.748597  6022 pending_rounds.cc:85] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 3
I20260709 12:25:42.750875  6022 pending_rounds.cc:107] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:25:42.751212  6022 raft_consensus.cc:2889] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:42.744647  8117 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:42.759361  5870 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:42.762153  8117 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:42.774539  8256 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:42.775229  8256 raft_consensus.cc:697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:42.776432  8256 consensus_queue.cc:237] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:42.777915  8192 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:42.781337  8289 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:42.787879  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:42.788714  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 79962us
I20260709 12:25:42.719544  8274 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:42.789863  8274 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:42.790493  8274 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:42.792182  8144 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:42.793269  7499 consensus_queue.cc:1059] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:42.795487  5990 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:42.632735  8170 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:42.797900  5987 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:42.740242  7994 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:42.800905  6756 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:25:42.801596  6756 pending_rounds.cc:85] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 4
I20260709 12:25:42.801973  6756 pending_rounds.cc:107] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:25:42.802235  6756 raft_consensus.cc:2889] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:42.801015  8170 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:42.801592  8016 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:42.808352  6756 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:42.812789  8241 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:42.814981  6611 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:42.815232  8241 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:42.816655  8241 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:42.816890  7994 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:42.817098  8241 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:42.819656  8241 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:42.821273  8241 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:42.822160  6773 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:42.822547  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:42.823582  8117 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:42.824129  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:42.824465  8117 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:42.826202  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:42.827512  6762 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 7: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:42.825457  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:42.827703  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:42.828835  6757 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:25:42.830232  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:42.831521  5915 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:42.826118  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:42.834254  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:42.835394  6768 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:42.834726  8016 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:42.837864  8011 raft_consensus.cc:3055] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:42.838325  8011 raft_consensus.cc:740] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:42.839440  8011 consensus_queue.cc:260] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:42.841141  8011 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:42.819334  6611 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:42.791373  8014 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
W20260709 12:25:42.868376  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 48471.5us
I20260709 12:25:42.847533  8252 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:42.870292  8170 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:42.889845  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 70617.6us
I20260709 12:25:42.884855  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:42.892783  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 107130us
I20260709 12:25:42.897439  7994 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:25:42.923041  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:42.926143  6757 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:25:42.916672  8192 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:42.892282  8170 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.929672  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:42.938730  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 56280.4us
W20260709 12:25:42.941162  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 69769.6us
I20260709 12:25:42.946858  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:42.947969  5874 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.951922  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:42.954909  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:42.962441  5868 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:42.963836  5872 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:42.935659  8015 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.952078  8162 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:42.970822  8162 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:42.971448  8162 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.954958  8192 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:42.973790  8218 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 12:25:42.976488  8192 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:42.976722  8273 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.000s	sys 0.021s
I20260709 12:25:42.977632  8016 raft_consensus.cc:3055] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:42.949446  8036 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:42.978890  8036 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.979377  8036 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:42.967406  8015 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:42.984244  8015 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:42.989923  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.991874  8144 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.992764  8144 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:42.994360  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 49288.1us
I20260709 12:25:42.996004  8144 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:42.996706  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:42.999135  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:43.000098  8050 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:43.001031  8050 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:43.002525  8050 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:42.937254  8170 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:43.008996  6020 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:42.978355  8016 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:42.901185  8114 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 12:25:43.013010  8016 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:43.014539  8114 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:43.015115  8114 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:43.015230  8016 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:43.016584  8114 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:43.018044  7750 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.517s	user 0.017s	sys 0.001s
I20260709 12:25:42.937670  8014 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.019533  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:42.977432  8273 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.000s	sys 0.021s
W20260709 12:25:43.035929  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 104840us
I20260709 12:25:43.037690  6775 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:43.044659  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.034538  7499 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:43.034430  8289 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:43.058619  5874 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:43.058682  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:43.065021  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:43.059271  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:43.077548  6611 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:43.082322  5870 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:43.062304  7994 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:43.083995  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:43.084100  7994 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:43.085882  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.086889  7994 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:43.062290  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:43.087658  8147 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:43.088506  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:43.090098  8147 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:43.090229  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.088094  5946 raft_consensus.cc:2376] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 2. Re-sending same reply.
W20260709 12:25:43.091672  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.065140  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 704) took 1469 ms. Trace:
I20260709 12:25:43.056214  7750 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:25:43.091768  8300 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 12:25:43.129379  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:43.130249  5945 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:43.080163  8291 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:25:43.100772  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 139443us
I20260709 12:25:43.136068  8291 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:43.138911  8291 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:43.139708  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.140259  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:43.107321  8015 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:43.107956  7994 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:43.095237  6785 rpcz_store.cc:276] 0709 12:25:41.595263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.115280 (+520017us) service_pool.cc:224] Handling call
0709 12:25:42.115851 (+   571us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:42.116513 (+   662us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:42.116558 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:42.117287 (+   729us) log.cc:844] Serialized 26 byte log entry
0709 12:25:42.609517 (+492230us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b1800367d20. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:25:42.894653 (+285136us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:42.910398 (+ 15745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.928732 (+ 18334us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:43.064651 (+135919us) raft_consensus.cc:1685] finished
0709 12:25:43.064675 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:43.064922 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":993,"raft.queue_time_us":99,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":10270,"spinlock_wait_cycles":5658880,"thread_start_us":197078,"threads_started":1,"wal-append.queue_time_us":930967}
I20260709 12:25:43.113688  8252 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:43.044907  8253 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:43.141852  5940 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:43.152146  8253 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:43.048827  8014 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:43.142969  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:43.122109  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 123119us
I20260709 12:25:43.153807  5944 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 7: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:43.154462  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:43.133873  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:43.156895  5941 raft_consensus.cc:2410] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:25:43.143817  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:43.116577  7750 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:43.159732  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:43.141585  5946 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:43.152060  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:43.144188  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:43.134743  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:43.146042  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 70021.8us
I20260709 12:25:43.133973  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:43.144325  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:43.100356  8114 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:43.145478  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:43.164235  5945 raft_consensus.cc:2376] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4. Re-sending same reply.
I20260709 12:25:43.162822  5873 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 4.
I20260709 12:25:43.170285  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:43.172080  5948 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:43.143627  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:43.172608  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:43.174258  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 121904us
I20260709 12:25:43.179890  5868 raft_consensus.cc:2393] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 6: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:43.180958  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:43.177284  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:43.183723  5949 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:43.184247  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:43.144028  7499 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:43.169617  8011 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:43.143450  8015 raft_consensus.cc:697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:43.196346  5869 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:43.093571  8184 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:43.196346  5940 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:43.178607  8289 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:43.190143  7499 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.231128  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:43.232888  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:43.234302  5865 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:43.235742  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:43.236994  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:43.237649  5943 raft_consensus.cc:2376] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6. Re-sending same reply.
I20260709 12:25:43.238579  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:43.239977  5943 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:43.240890  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:43.231159  7499 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:43.243507  8289 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:43.246819  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:43.248570  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:43.212499  8274 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:43.214733  6611 leader_election.cc:400] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:25:43.293738  6611 leader_election.cc:403] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:43.140511  8144 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:25:43.218103  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:43.219506  8015 consensus_queue.cc:237] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:43.191980  8139 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:43.223479  5869 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:43.181926  8147 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:43.224951  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:43.188983  8176 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:43.228412  8241 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:43.304984  8176 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.217083  8170 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:43.229832  8211 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.230239  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:43.225033  7750 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:43.223580  8014 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:43.208997  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 45069.1us
I20260709 12:25:43.257586  5870 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 12:25:43.350713  5866 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:43.292580  6768 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:43.244848  8289 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:43.301275  5873 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:43.229902  8259 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.559s	user 0.007s	sys 0.007s
I20260709 12:25:43.194288  8291 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.122s
I20260709 12:25:43.357573  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:43.306301  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 46693.3us
I20260709 12:25:43.308136  8147 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.311558  8011 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:43.313068  5990 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:43.315656  5874 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 6: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:43.234225  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:43.366493  5948 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:43.229210  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:43.353102  5868 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:43.311407  5869 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:25:43.364583  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:43.305423  7871 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:43.348001  8176 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:43.369493  5870 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:43.352794  8253 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:25:43.375046  5866 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:43.301252  8210 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.071s
W20260709 12:25:43.381713  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.305215  8241 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:43.379606  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 272593us
I20260709 12:25:43.369235  8147 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:43.380812  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 67851us
I20260709 12:25:43.385694  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:43.378702  8114 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:43.395833  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:43.357347  8211 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:43.389909  8259 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:43.396704  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:43.432847  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 288450us
W20260709 12:25:43.412606  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.399979  8014 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:43.401574  8114 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.382670  7871 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:43.440230  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 225823us
I20260709 12:25:43.445119  7955 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:43.446702  5872 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:43.383319  8162 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:43.454502  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 72672.9us
I20260709 12:25:43.447206  8114 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:43.476421  5865 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:43.466517  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:43.455138  7955 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:43.454974  8259 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:25:43.492799  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 52903us
I20260709 12:25:43.496763  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:43.505239  7871 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:43.505975  8184 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:43.506680  8259 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:43.510475  7880 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:43.511113  7880 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:43.512303  7880 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:43.514073  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:43.515182  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.515858  5915 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:43.498692  8011 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:43.519914  8011 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:43.520329  8011 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:43.524650  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:43.502344  8036 consensus_meta.cc:287] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.005s
I20260709 12:25:43.526705  7750 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:43.547876  8319 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:25:43.526890  8144 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:43.525616  5872 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:43.547353  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.563544  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:43.527864  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:43.568677  5873 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:43.539388  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:43.570039  5868 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:43.571187  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:43.572557  5870 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:43.550462  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:43.576071  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.578991  6017 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:43.581286  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 87104.6us
W20260709 12:25:43.532526  8309 log.cc:927] Time spent T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:25:43.585427  5868 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:43.589648  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:43.594451  6022 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:43.513579  7880 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:43.586463  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:43.582487  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:43.596988  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:43.600850  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:43.609200  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:43.539484  8162 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:43.611025  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:43.604206  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 239691us
I20260709 12:25:43.605306  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:43.612800  6768 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:43.530859  8241 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:43.616109  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 168253us
I20260709 12:25:43.550153  8274 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:43.527256  8211 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:43.615197  6762 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:43.615089  5838 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:25:43.617524  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.621207  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:43.613746  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 86021.1us
I20260709 12:25:43.617174  6611 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:43.587195  7750 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:43.625586  5870 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:43.533099  8036 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:43.603422  8016 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.630664  6020 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:25:43.608043  8170 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:43.650820  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 149370us
I20260709 12:25:43.634392  6019 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:43.649123  5990 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:43.538239  7955 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:43.656148  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 116922us
I20260709 12:25:43.624922  8253 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:25:43.658762  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 43858.7us
I20260709 12:25:43.659412  8274 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:43.665552  8274 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:43.666828  6775 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:25:43.667409  8274 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:43.647620  8184 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:43.674055  8184 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.674479  8184 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:43.620172  7871 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:25:43.677336  5838 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:43.678532  7871 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:43.679033  7871 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:43.680114  7871 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:43.658542  8306 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:25:43.665426  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:43.700994  6778 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:43.702744  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:43.646343  7750 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:43.681594  7871 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:43.711133  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:43.711557  8170 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:43.711138  6020 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:43.721616  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:43.700253  6017 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 3.
I20260709 12:25:43.723994  5872 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:25:43.726127  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:43.659631  8218 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:43.730803  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 77364.6us
W20260709 12:25:43.780262  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 58808.9us
W20260709 12:25:43.780432  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.789705  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 178081us
I20260709 12:25:43.790561  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:43.726929  6015 raft_consensus.cc:2376] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9. Re-sending same reply.
I20260709 12:25:43.794027  6022 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:43.706352  8139 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.652070  8016 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:43.689728  8100 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:43.798503  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:43.847059  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:43.727941  8241 raft_consensus.cc:2764] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:25:43.860376  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:43.849750  8253 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:43.884539  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:43.873543  5941 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.016s	sys 0.008s
W20260709 12:25:43.887579  5941 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.017s	sys 0.008s
I20260709 12:25:43.727350  8011 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:43.898983  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 100555us
I20260709 12:25:43.915026  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:43.915704  6609 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:43.888815  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:43.917114  6768 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:43.800940  8162 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 12:25:43.733659  8218 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.919804  6773 raft_consensus.cc:2393] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:43.925673  6023 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:43.888815  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:43.877081  8252 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:43.946327  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.813776  8114 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:43.933604  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:43.938731  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:43.875144  8211 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:43.854358  8050 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.065s
W20260709 12:25:43.883697  7499 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.641s	user 0.013s	sys 0.001s
W20260709 12:25:43.934880  8176 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.002s
I20260709 12:25:43.964144  6016 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:43.861109  8139 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:43.970491  6021 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:43.936528  8218 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:43.979305  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.984845  5874 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.018s	sys 0.021s
I20260709 12:25:43.986434  5866 raft_consensus.cc:2368] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7.
W20260709 12:25:43.987558  5874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.018s	sys 0.021s
I20260709 12:25:43.986235  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:43.963552  8253 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:43.919225  8100 consensus_queue.cc:1059] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
W20260709 12:25:44.080543  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 402400us
I20260709 12:25:44.064073  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:43.969435  7994 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:44.065475  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:44.063444  7898 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:44.075394  8011 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:43.985365  8320 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
I20260709 12:25:44.081902  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:43.982275  8036 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:44.080571  5871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.029s	sys 0.067s
I20260709 12:25:44.076632  8016 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:44.092217  6775 raft_consensus.cc:2393] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:25:44.094532  5871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.029s	sys 0.067s
I20260709 12:25:43.969769  7750 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:44.068121  8173 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:44.101109  8173 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:44.103590  8173 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:44.104382  8016 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:43.985914  8274 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:44.107743  8274 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:44.115247  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 414610us
I20260709 12:25:43.968600  7499 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:44.124325  6023 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:43.975101  8176 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:44.132290  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.123641  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:44.133900  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 52102.4us
W20260709 12:25:44.137549  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 408212us
I20260709 12:25:44.139806  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:44.079056  8324 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:25:44.156629  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 311634us
W20260709 12:25:44.121996  8147 consensus_meta.cc:287] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.725s	user 0.011s	sys 0.008s
W20260709 12:25:44.142771  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 410911us
I20260709 12:25:43.991787  8228 consensus_queue.cc:1059] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.419s
I20260709 12:25:44.160356  5990 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:44.172457  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 81030.5us
I20260709 12:25:44.188098  5867 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 3: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:25:44.191320  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.206741  8173 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:25:44.207274  8050 raft_consensus.cc:3055] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:44.208192  8050 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:44.209439  8114 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:44.213397  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 422840us
I20260709 12:25:44.209507  8050 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:44.215482  8050 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:44.216569  5915 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:44.169611  7994 raft_consensus.cc:3055] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:44.220660  6021 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:44.225756  7994 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:25:44.214047  8325 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 12:25:44.233763  8114 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:44.257771  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:44.259514  7499 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:44.259725  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:44.261152  7499 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.170693  8011 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:44.183383  8147 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:44.239552  8050 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.261560  5915 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:44.298766  8050 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:44.186806  8170 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
W20260709 12:25:44.243670  8184 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.569s	user 0.013s	sys 0.000s
I20260709 12:25:44.251554  8274 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:44.234728  8329 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 12:25:44.278520  7499 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:44.191259  8210 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:44.281095  8139 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:44.282709  5940 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:44.301755  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 752) took 1073 ms. Trace:
I20260709 12:25:44.380401  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:44.413774  5867 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:44.296280  8241 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:44.291512  7898 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:25:44.356007  6019 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.695s	user 0.009s	sys 0.003s
I20260709 12:25:44.440156  6019 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:44.253086  7994 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:44.375329  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.285849  8011 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:44.380481  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:44.299487  8254 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
W20260709 12:25:44.241452  8014 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.797s	user 0.004s	sys 0.005s
I20260709 12:25:44.475839  5872 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:44.476574  8011 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:44.294904  8016 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:44.404953  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 810) took 1240 ms. Trace:
W20260709 12:25:44.438706  5914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.008s	sys 0.025s
I20260709 12:25:44.413455  5874 rpcz_store.cc:276] 0709 12:25:43.228267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.373802 (+145535us) service_pool.cc:224] Handling call
0709 12:25:43.374229 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.393270 (+ 19041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:43.393305 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.403267 (+  9962us) log.cc:844] Serialized 26 byte log entry
0709 12:25:43.988237 (+584970us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:43.992636 (+  4399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.992799 (+   163us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:44.301374 (+308575us) raft_consensus.cc:1685] finished
0709 12:25:44.301402 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.301652 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90262,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":4776,"thread_start_us":533422,"threads_started":1,"wal-append.queue_time_us":864400}
W20260709 12:25:44.482789  5914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.009s	sys 0.025s
I20260709 12:25:44.251068  8176 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:44.297292  8100 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.475947  8014 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:44.331431  8322 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.618s	user 0.004s	sys 0.001s
I20260709 12:25:44.481884  5913 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:44.454233  6024 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.008s	sys 0.051s
W20260709 12:25:44.489521  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 228047us
W20260709 12:25:44.500108  6024 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.009s	sys 0.051s
W20260709 12:25:44.499366  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 361165us
I20260709 12:25:44.359575  8342 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:44.405907  8144 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:44.481961  5941 rpcz_store.cc:276] 0709 12:25:43.138697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.178926 (+ 40229us) service_pool.cc:224] Handling call
0709 12:25:43.192611 (+ 13685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:43.207167 (+ 14556us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:43.888174 (+681007us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.378424 (+490250us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:44.378495 (+    71us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:44.378560 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.378646 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.378843 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22199,"prepare.queue_time_us":439197,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":121372,"thread_start_us":1008959,"threads_started":2}
I20260709 12:25:44.406425  8344 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:44.437412  8241 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:44.485325  7898 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:44.379606  8184 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:44.528375  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:44.536171  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.510960  8100 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:44.539052  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:44.539273  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:44.540031  5945 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:44.540271  5942 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:44.542217  5946 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:44.488269  8210 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:44.531301  8241 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:44.532838  7994 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:44.515084  8274 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
W20260709 12:25:44.578967  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.573546  5838 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:44.541386  8259 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.585026  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 345) took 1355 ms. Trace:
W20260709 12:25:44.591617  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 376059us
I20260709 12:25:44.592621  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:44.585675  5940 rpcz_store.cc:276] 0709 12:25:43.222958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.232554 (+  9596us) service_pool.cc:224] Handling call
0709 12:25:44.577875 (+1345321us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540058fa54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:44.578117 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2411891456}
I20260709 12:25:44.536164  8343 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:44.546751  8014 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:44.627673  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.630570  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:44.631551  6019 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:44.633105  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:44.634128  6021 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:44.580825  7880 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.966s	user 0.009s	sys 0.008s
I20260709 12:25:44.608024  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:44.609612  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 719) took 1015 ms. Trace:
I20260709 12:25:44.556659  8342 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:44.574182  8016 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:44.524600  8345 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:44.677675  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 85120us
W20260709 12:25:44.679744  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 71814.4us
I20260709 12:25:44.582598  8147 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:44.730435  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:44.732400  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:44.733947  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:44.735128  6020 raft_consensus.cc:2410] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:44.735239  8210 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:44.736557  6016 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:44.737061  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:44.737591  8331 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.434s	user 0.001s	sys 0.001s
I20260709 12:25:44.737804  6023 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 3: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:44.738914  8349 raft_consensus.cc:3055] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:44.739434  8349 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:44.740548  8349 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:44.588739  8211 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:44.758617  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:44.773582  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44480.7us
I20260709 12:25:44.816956  6778 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:44.817438  6611 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:44.824026  6019 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:25:44.815681  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:44.816118  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:44.829386  6773 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:44.601372  7750 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.019s	sys 0.018s
I20260709 12:25:44.829756  6775 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:44.835064  8050 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:44.842509  8336 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.355s	user 0.001s	sys 0.001s
I20260709 12:25:44.844336  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:44.847630  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 925) took 1391 ms. Trace:
I20260709 12:25:44.675619  8342 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.853674  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:44.855080  6757 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.006s	sys 0.049s
W20260709 12:25:44.855549  6757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.006s	sys 0.050s
I20260709 12:25:44.850637  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:44.857172  5867 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:44.848265  5873 rpcz_store.cc:276] 0709 12:25:43.456160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.580550 (+124390us) service_pool.cc:224] Handling call
0709 12:25:43.581029 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.603292 (+ 22263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:43.603347 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.617294 (+ 13947us) log.cc:844] Serialized 26 byte log entry
0709 12:25:43.936955 (+319661us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:43.937085 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.937277 (+   192us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:44.847281 (+910004us) raft_consensus.cc:1685] finished
0709 12:25:44.847313 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.847530 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11350,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":18849,"thread_start_us":259922,"threads_started":1,"wal-append.queue_time_us":723783}
I20260709 12:25:44.773043  8210 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.865100  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:44.854730  6793 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:44.760614  8036 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:44.857316  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:44.881966  6019 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:44.746771  8343 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:44.596258  8259 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:44.812471  8170 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:44.832720  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:44.894277  6021 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
W20260709 12:25:44.879104  5840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.033s	sys 0.067s
I20260709 12:25:44.727895  8016 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:44.679206  8014 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:44.937775  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:44.938668  8351 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 12:25:44.917929  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.921826  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:44.839931  7750 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.019s	sys 0.020s
W20260709 12:25:44.925900  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.926605  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.943127  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.943557  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.943899  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.927933  6778 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
W20260709 12:25:44.897125  5944 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.939s	user 0.029s	sys 0.073s
I20260709 12:25:44.828840  8349 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:44.930750  6785 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:25:44.931687  5840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.034s	sys 0.067s
W20260709 12:25:44.945261  5944 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.995s	user 0.029s	sys 0.073s
W20260709 12:25:44.945588  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 760650us
I20260709 12:25:44.946375  6775 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 7: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:25:44.864424  8228 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.026s	sys 0.029s
W20260709 12:25:44.899469  6609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.959s	user 0.121s	sys 0.108s
W20260709 12:25:44.947170  6609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.122s	sys 0.109s
I20260709 12:25:44.814790  8211 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:25:44.937389  8097 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.018s	sys 0.029s
I20260709 12:25:44.863884  8050 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:44.932467  6023 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:25:44.947474  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:44.865473  8348 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.865017  8342 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:44.875897  8147 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:44.960515  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 807) took 1822 ms. Trace:
W20260709 12:25:44.953778  8228 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.026s	sys 0.029s
I20260709 12:25:44.811295  8345 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:44.939960  7499 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:45.003415  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 1.08776e+06us
I20260709 12:25:45.013306  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:44.925643  8173 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:25:44.965726  5944 rpcz_store.cc:276] 0709 12:25:43.137743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.154650 (+ 16907us) service_pool.cc:224] Handling call
0709 12:25:43.155118 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:43.155623 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:43.581038 (+425415us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.952381 (+1371343us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.952586 (+   205us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:44.952643 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:44.960162 (+  7519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.960269 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.960441 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1978,"prepare.queue_time_us":437780,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":318856,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"thread_start_us":1731511,"threads_started":3,"wal-append.queue_time_us":618376}
I20260709 12:25:44.939520  7880 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.009720  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 527570us
I20260709 12:25:44.969842  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:45.019409  6785 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:44.954402  8211 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:45.027251  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.927650  8036 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:44.955654  8097 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.018s	sys 0.029s
I20260709 12:25:44.896165  8210 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:45.033074  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:45.038501  5873 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:44.966233  8218 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.975s	user 0.013s	sys 0.003s
I20260709 12:25:45.043440  6775 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:44.965412  8176 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:44.965004  8346 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.955778  8170 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:25:45.064239  5946 consensus_meta.cc:287] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.518s	user 0.011s	sys 0.000s
I20260709 12:25:45.065276  6609 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:45.066079  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:44.964308  8357 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:44.955924  8347 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 12:25:45.004047  8147 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:45.071703  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:45.072620  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 716) took 1496 ms. Trace:
I20260709 12:25:45.074433  5874 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:45.022675  8184 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:45.077067  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 259741us
I20260709 12:25:45.079564  5946 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:45.070201  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:45.081980  6757 rpcz_store.cc:276] 0709 12:25:43.576634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.576813 (+   179us) service_pool.cc:224] Handling call
0709 12:25:43.596317 (+ 19504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:43.622240 (+ 25923us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:43.650457 (+ 28217us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.095157 (+444700us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b1800295de0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:25:45.052579 (+957422us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:45.052648 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:45.072312 (+ 19664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.072402 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.072555 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41773,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":4966,"spinlock_wait_cycles":26950912,"thread_start_us":930261,"threads_started":2,"wal-append.queue_time_us":1358490}
W20260709 12:25:45.053571  8335 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:25:45.097437  5873 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:45.087720  6023 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
W20260709 12:25:45.090852  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 332382us
W20260709 12:25:45.099936  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 92149.9us
I20260709 12:25:44.969844  8348 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:45.101929  5872 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:45.076809  8274 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.097267  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:45.037243  7880 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:45.095836  8356 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:25:45.101910  8344 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:45.099165  8352 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.141324  8352 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:45.143844  8253 consensus_meta.cc:287] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.051s	user 0.013s	sys 0.003s
I20260709 12:25:45.144407  8253 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:45.144579  7955 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:45.145700  8350 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.324s	user 0.000s	sys 0.001s
I20260709 12:25:45.148933  8147 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:45.150940  8345 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:45.151723  8355 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.055s
W20260709 12:25:45.152347  7871 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.426s	user 0.007s	sys 0.007s
I20260709 12:25:45.153667  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:45.156144  5915 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:45.158090  6762 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:45.158742  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 750) took 1931 ms. Trace:
W20260709 12:25:45.135576  8338 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.418s	user 0.000s	sys 0.002s
I20260709 12:25:45.113720  8274 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:45.136209  8377 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:45.136866  8365 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:25:45.138906  8210 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:45.140252  8184 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.086462  8346 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:45.164783  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 51409.2us
I20260709 12:25:45.068583  8218 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:45.159327  5869 rpcz_store.cc:276] 0709 12:25:43.227556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.372853 (+145297us) service_pool.cc:224] Handling call
0709 12:25:43.383260 (+ 10407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.740433 (+357173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:43.740481 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.848731 (+108250us) log.cc:844] Serialized 26 byte log entry
0709 12:25:44.371179 (+522448us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:44.371236 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.371338 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.158423 (+787085us) raft_consensus.cc:1685] finished
0709 12:25:45.158451 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.158652 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108651,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":141,"thread_start_us":791825,"threads_started":2,"wal-append.queue_time_us":706554}
I20260709 12:25:45.131192  8011 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.205982  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:45.134359  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:45.099519  8100 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:45.186192  6021 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:45.165920  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:45.135579  8333 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.480s	user 0.000s	sys 0.002s
I20260709 12:25:45.148736  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:45.194095  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:45.198679  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:45.201675  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 45593us
W20260709 12:25:45.169937  5945 consensus_meta.cc:287] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.618s	user 0.010s	sys 0.003s
I20260709 12:25:45.094674  8357 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:45.144593  8352 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:45.215941  5945 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:45.182592  6020 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:45.214797  6785 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:45.185609  8176 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:45.187412  8377 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:25:45.242448  5987 connection.cc:256] RPC call timeout handler was delayed by -2.0365760240001691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18480000000000002s for a response
W20260709 12:25:45.280566  5987 connection.cc:256] RPC call timeout handler was delayed by -1.995361053000579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2042s for a response
W20260709 12:25:45.281267  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:45.281731  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:45.282539  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.282970  5872 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:45.264531  8344 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:45.287693  6773 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:45.200728  8016 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:45.267791  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 153704us
I20260709 12:25:45.267150  8100 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.271355  8173 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:45.297716  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 721) took 1694 ms. Trace:
W20260709 12:25:45.289667  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 141003us
I20260709 12:25:45.300151  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:45.213835  8253 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.110678  8343 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:45.290455  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:45.304064  6023 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:45.256560  8210 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.292376  6015 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.559s	user 0.013s	sys 0.001s
I20260709 12:25:45.219498  8139 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:45.202391  8364 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:25:45.217222  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 49361.7us
W20260709 12:25:45.294932  5987 connection.cc:256] RPC call timeout handler was delayed by -1.8270610530003069s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
I20260709 12:25:45.293687  8390 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:45.309316  8253 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:45.294817  8259 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:44.660213  6756 rpcz_store.cc:276] 0709 12:25:43.593936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.602971 (+  9035us) service_pool.cc:224] Handling call
0709 12:25:43.624045 (+ 21074us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:43.624299 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:44.114752 (+490453us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.599783 (+485031us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.608961 (+  9178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:44.609112 (+   151us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:44.609204 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.609316 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.609518 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26980,"prepare.queue_time_us":85730,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":9679,"raft.run_cpu_time_us":474,"raft.run_wall_time_us":88809,"thread_start_us":860094,"threads_started":2}
I20260709 12:25:45.305226  8367 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:45.305382  8036 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:45.307031  6015 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:45.359037  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:45.360051  6015 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 8: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:45.202910  8184 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:45.298238  6766 rpcz_store.cc:276] 0709 12:25:43.603702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.603852 (+   150us) service_pool.cc:224] Handling call
0709 12:25:43.623830 (+ 19978us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.669514 (+ 45684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:43.669564 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.670262 (+   698us) log.cc:844] Serialized 26 byte log entry
0709 12:25:43.957840 (+287578us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:43.957917 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.958055 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.297373 (+1339318us) raft_consensus.cc:1685] finished
0709 12:25:45.297402 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.297613 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9954,"raft.queue_time_us":42238,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":267870,"threads_started":1,"wal-append.queue_time_us":858322}
I20260709 12:25:45.277096  8352 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:45.296695  8173 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.369097  8173 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:45.355810  8346 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:45.362481  8367 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.281725  8377 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:45.352336  8210 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:45.305879  8368 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.105s
I20260709 12:25:45.299242  8016 consensus_queue.cc:1059] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:25:45.269116  8170 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:45.302098  7499 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:45.343017  8390 raft_consensus.cc:697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:25:45.343856  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 43848.8us
I20260709 12:25:45.344377  8381 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:45.314062  8176 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:45.324404  8117 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:45.334582  8344 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:45.344849  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 440) took 1564 ms. Trace:
I20260709 12:25:45.406275  6018 rpcz_store.cc:276] 0709 12:25:43.780290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.783540 (+  3250us) service_pool.cc:224] Handling call
0709 12:25:43.845012 (+ 61472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.408903 (+563891us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:44.408963 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:44.516102 (+107139us) log.cc:844] Serialized 26 byte log entry
0709 12:25:45.009919 (+493817us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:45.009970 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.011047 (+  1077us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.344564 (+333517us) raft_consensus.cc:1685] finished
0709 12:25:45.344591 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.344769 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40026,"raft.queue_time_us":191247,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":19196,"thread_start_us":907856,"threads_started":2,"wal-append.queue_time_us":727456}
W20260709 12:25:45.348533  7898 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.804s	user 0.014s	sys 0.002s
I20260709 12:25:45.349311  8340 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:45.349745  8144 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:45.346024  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:45.363482  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 924) took 1958 ms. Trace:
I20260709 12:25:45.367863  8352 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:45.321767  8100 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:45.357141  8358 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:45.351312  8274 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:45.356355  8218 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:45.500546  5866 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 758) took 2078 ms (client timeout 1583 ms). Trace:
I20260709 12:25:45.217916  7871 consensus_queue.cc:1059] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.168s
W20260709 12:25:45.402009  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 236329us
I20260709 12:25:45.402235  8388 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.403313  8391 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.404922  8343 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.383813  8383 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:45.377048  8210 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:45.374003  8385 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.456174  6023 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:45.511234  8391 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:45.511807  8330 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 12:25:45.513105  6023 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:45.455595  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 48558.5us
I20260709 12:25:45.514101  8343 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:45.516443  8343 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:45.501827  8362 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:45.519394  8362 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:45.520898  8349 consensus_meta.cc:287] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.567s	user 0.007s	sys 0.002s
I20260709 12:25:45.383922  8368 raft_consensus.cc:3055] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:45.371992  8253 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:45.461465  5990 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:45.456974  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:45.384675  8348 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:45.491721  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:45.481295  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 265567us
I20260709 12:25:45.497614  8358 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:45.456051  5871 rpcz_store.cc:276] 0709 12:25:43.405255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.535863 (+130608us) service_pool.cc:224] Handling call
0709 12:25:43.536248 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.536704 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:43.536732 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.537350 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:25:44.095870 (+558520us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:44.095943 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.096054 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.363233 (+1267179us) raft_consensus.cc:1685] finished
0709 12:25:45.363257 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.363408 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1162,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"thread_start_us":557998,"threads_started":1,"wal-append.queue_time_us":953567}
I20260709 12:25:45.399430  8117 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:45.498006  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:45.379873  8184 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:45.374464  8367 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:45.492632  8352 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:45.370591  8211 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:45.501909  5866 rpcz_store.cc:269] 0709 12:25:43.384013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.384138 (+   125us) service_pool.cc:224] Handling call
0709 12:25:45.462481 (+2078343us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b54001ed054. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:45.462655 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1345202944}
W20260709 12:25:45.504405  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 94131.6us
W20260709 12:25:45.469702  7994 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.877s	user 0.014s	sys 0.000s
I20260709 12:25:45.507120  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:45.507679  5913 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:45.518607  6766 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:45.560822  5941 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:45.217224  8011 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:45.459916  8144 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.521826  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:45.392302  7499 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:45.563200  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 54896.2us
I20260709 12:25:45.564088  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:45.526296  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68963.6us
I20260709 12:25:45.532159  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:45.455013  7898 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:45.570317  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.522265  8383 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:45.564038  7499 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:45.579380  8383 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:45.580402  8100 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:45.580976  8383 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:45.581804  8369 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:25:45.582123  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:45.582405  8114 raft_consensus.cc:3055] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:45.582552  6766 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:25:45.582839  8114 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:45.583197  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 436) took 1843 ms. Trace:
I20260709 12:25:45.535633  8390 consensus_queue.cc:237] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:45.584026  8114 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:45.528599  8259 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:45.588775  8259 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.589246  8259 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.511952  8385 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:45.589941  6773 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:25:45.591573  8385 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:45.592298  8342 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.585s	user 0.006s	sys 0.006s
I20260709 12:25:45.592762  8342 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:45.538664  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:45.539384  8050 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.575s	user 0.014s	sys 0.001s
I20260709 12:25:45.594045  8050 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:45.540709  8117 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:45.597862  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 95305.9us
I20260709 12:25:45.541740  8139 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:45.542102  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:45.532332  8358 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:45.545753  8388 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:45.554972  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:45.610504  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.523032  8368 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:45.399262  8381 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:45.615177  8211 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:45.535463  6023 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:25:45.575415  5915 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:45.616182  6023 pending_rounds.cc:85] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 2
I20260709 12:25:45.595175  6019 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:45.616446  6023 pending_rounds.cc:107] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:25:45.563868  8011 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:45.616704  6023 raft_consensus.cc:2889] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:45.617416  8117 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:45.618404  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:45.620236  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:45.623957  8342 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:45.628700  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:45.629331  8184 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:45.631665  8391 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:45.596117  6018 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:45.583616  6024 rpcz_store.cc:276] 0709 12:25:43.739911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.740107 (+   196us) service_pool.cc:224] Handling call
0709 12:25:43.854411 (+114304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.500664 (+646253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.500702 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:44.512848 (+ 12146us) log.cc:844] Serialized 26 byte log entry
0709 12:25:45.010087 (+497239us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:45.010147 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.010252 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.582914 (+572662us) raft_consensus.cc:1685] finished
0709 12:25:45.582941 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.583122 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2480,"raft.queue_time_us":115120,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":11430,"thread_start_us":1091974,"threads_started":2,"wal-append.queue_time_us":875359}
W20260709 12:25:45.612126  5840 connection.cc:256] RPC call timeout handler was delayed by -1.7036110650001319s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
W20260709 12:25:45.656850  5840 connection.cc:256] RPC call timeout handler was delayed by -1.6892110650001086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
W20260709 12:25:45.657366  5840 connection.cc:256] RPC call timeout handler was delayed by -1.6415110650000315s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
W20260709 12:25:45.657804  5840 connection.cc:256] RPC call timeout handler was delayed by -1.551511064999886s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1708s for a response
W20260709 12:25:45.658187  5840 connection.cc:256] RPC call timeout handler was delayed by -1.5371110650003175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
W20260709 12:25:45.658664  5840 connection.cc:256] RPC call timeout handler was delayed by -1.4489110650001749s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18220000000000003s for a response
W20260709 12:25:45.659065  5840 connection.cc:256] RPC call timeout handler was delayed by -1.3454110650000075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
W20260709 12:25:45.659425  5840 connection.cc:256] RPC call timeout handler was delayed by -1.2581110650003211s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2034s for a response
W20260709 12:25:45.659786  5840 connection.cc:256] RPC call timeout handler was delayed by -1.0655110650000097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22480000000000003s for a response
I20260709 12:25:45.612221  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:45.612731  8340 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:45.575017  8386 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.613721  8368 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:45.649719  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:45.650684  6793 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
W20260709 12:25:45.670009  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 105997us
W20260709 12:25:45.581529  8354 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.485s	user 0.001s	sys 0.002s
I20260709 12:25:45.649931  6773 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:45.660696  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:45.664743  8359 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.665160  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:45.673280  5941 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.665483  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:45.673575  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 98313.7us
W20260709 12:25:45.667832  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 67059.7us
I20260709 12:25:45.610792  8114 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.675180  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:45.652592  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:45.607699  8343 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:45.602946  8362 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:45.687162  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 705) took 1026 ms. Trace:
I20260709 12:25:45.696722  6023 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.707062  6022 rpcz_store.cc:276] 0709 12:25:44.648232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.648425 (+   193us) service_pool.cc:224] Handling call
0709 12:25:44.730384 (+ 81959us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:44.744056 (+ 13672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.744108 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:44.745013 (+   905us) log.cc:844] Serialized 52 byte log entry
0709 12:25:45.091598 (+346585us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:45.111320 (+ 19722us) log.cc:844] Serialized 14 byte log entry
0709 12:25:45.662239 (+550919us) spinlock_profiling.cc:243] Waited 2.95 ms on lock 0x7b18001f69c0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39d282b4bc 00007f39cfb7948b 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287
0709 12:25:45.662462 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.662580 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.662649 (+    69us) raft_consensus.cc:1685] finished
0709 12:25:45.674578 (+ 11929us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.674744 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13081,"prepare.queue_time_us":14475,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":243571,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":17735,"spinlock_wait_cycles":6777600,"thread_start_us":309021,"threads_started":2,"wal-append.queue_time_us":494560}
W20260709 12:25:45.744154  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 50487.4us
I20260709 12:25:45.673825  8253 raft_consensus.cc:2764] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:25:45.663256  8014 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:45.750133  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:45.748980  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:45.752748  6022 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:45.750065  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:45.756353  6762 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 7: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:45.686853  8343 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:45.767253  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 92121.2us
I20260709 12:25:45.740350  8368 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:45.768204  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:45.770715  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:45.747960  8241 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.143s	user 0.014s	sys 0.001s
I20260709 12:25:45.741097  8346 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:45.743391  8391 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.794014  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:45.799094  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:25:45.796341  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 135748us
I20260709 12:25:45.800365  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:45.800616  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:45.747182  8011 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:45.813251  5915 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:45.815768  6022 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:25:45.755126  8352 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:45.824824  6768 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:45.827942  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 147500us
I20260709 12:25:45.824168  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:45.835577  6023 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:45.775089  8343 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.678879  8340 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:45.808782  8342 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:45.842736  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:25:45.839681  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.846385  6024 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:45.763221  8362 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:25:45.859724  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:45.861117  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:45.862090  6762 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:45.877614  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:45.860738  5943 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:45.822122  8402 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:45.815826  8241 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:45.879617  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:45.910444  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:45.910775  8393 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.236s	user 0.000s	sys 0.000s
I20260709 12:25:45.911459  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:45.878733  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:45.818746  8391 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:45.920244  5940 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 7: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:45.845659  8330 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:45.826391  8289 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
W20260709 12:25:45.924945  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 78854.4us
W20260709 12:25:45.922538  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 43909.6us
I20260709 12:25:45.997488  8391 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:45.978597  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:45.911301  8402 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:45.889827  8342 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:45.967551  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:45.879798  8211 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:45.861411  8011 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:45.844561  8173 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:45.852315  8388 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:45.912199  8385 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:45.926862  7499 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.996201  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:46.079176  6022 raft_consensus.cc:2410] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:45.996984  7898 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:45.979880  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:45.967442  8346 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:45.983376  8362 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.998078  8398 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.417s	user 0.002s	sys 0.000s
I20260709 12:25:46.016842  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:45.687493  8386 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:45.672935  8359 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:45.920507  8389 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.409s
I20260709 12:25:45.983345  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:46.063201  6024 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:46.063257  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:46.087280  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 70517.7us
I20260709 12:25:46.088357  5943 raft_consensus.cc:2410] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:25:46.033778  8402 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:46.101477  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 435) took 2349 ms. Trace:
I20260709 12:25:46.102969  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:46.103267  8407 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:25:46.103034  8340 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:46.106629  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:46.107026  8399 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 12:25:46.153250  6016 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:46.154947  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:46.155827  5945 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:46.082775  8404 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:46.158111  8404 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:46.158756  7880 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:46.167701  8361 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 1.032s	user 0.012s	sys 0.000s
I20260709 12:25:46.087586  7499 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:46.113165  8173 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:46.113359  8381 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:46.114586  8253 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:46.073093  8342 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:46.070622  8391 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:46.098348  5940 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:46.124782  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:46.119222  6017 rpcz_store.cc:276] 0709 12:25:43.738968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.797790 (+ 58822us) service_pool.cc:224] Handling call
0709 12:25:43.798144 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.916689 (+118545us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:43.916725 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.951675 (+ 34950us) log.cc:844] Serialized 26 byte log entry
0709 12:25:44.514279 (+562604us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:44.514330 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.080437 (+1566107us) spinlock_profiling.cc:243] Waited 80.7 ms on lock 0x7b5400661cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:46.080527 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.080617 (+    90us) raft_consensus.cc:1685] finished
0709 12:25:46.080640 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.087782 (+  7142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40787,"raft.queue_time_us":272180,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":27713,"spinlock_wait_cycles":185504000,"thread_start_us":383116,"threads_started":1,"wal-append.queue_time_us":863961}
I20260709 12:25:46.083266  8385 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:46.128423  8389 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:46.130338  5695 catalog_manager.cc:5697] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:46.132076  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:46.116182  8349 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:46.217289  5948 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:46.134078  8324 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:25:46.136173  8383 consensus_meta.cc:287] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.555s	user 0.011s	sys 0.003s
I20260709 12:25:46.138607  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 587) took 1395 ms. Trace:
I20260709 12:25:46.222824  6768 rpcz_store.cc:276] 0709 12:25:44.743349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.745090 (+  1741us) service_pool.cc:224] Handling call
0709 12:25:44.745524 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.826583 (+1081059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.138191 (+311608us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54002a17d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:46.138505 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2481540480}
W20260709 12:25:46.212900  8259 consensus_meta.cc:287] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.623s	user 0.010s	sys 0.006s
W20260709 12:25:46.142612  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 79443.7us
I20260709 12:25:46.226460  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:46.226908  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:46.144011  5941 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
W20260709 12:25:46.145524  8117 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.001s
W20260709 12:25:46.148008  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.151913  8346 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.172559  8218 raft_consensus.cc:3055] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:46.109380  8386 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:46.083985  8011 raft_consensus.cc:697] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:46.233718  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:46.234508  5941 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:46.190971  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:46.217680  8173 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:46.192485  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 731) took 1029 ms. Trace:
I20260709 12:25:46.210438  6022 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:46.100536  8362 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:46.213400  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 180574us
W20260709 12:25:46.217881  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 155676us
I20260709 12:25:46.224452  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:46.141501  8352 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:46.257402  8352 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:46.257755  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:46.230290  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.258503  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:46.235823  5949 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 7: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:46.227227  5940 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:46.202675  8359 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:46.237335  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 935) took 1125 ms. Trace:
W20260709 12:25:46.234325  8377 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.859s	user 0.015s	sys 0.000s
I20260709 12:25:46.240206  8114 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:46.269290  8274 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:46.270100  8274 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:46.271064  5838 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:46.271188  8274 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:46.272768  8340 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:46.267340  5872 rpcz_store.cc:276] 0709 12:25:45.112380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.112885 (+   505us) service_pool.cc:224] Handling call
0709 12:25:45.113260 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.283651 (+170391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.237069 (+953418us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54001d0454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:46.237261 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343060352}
W20260709 12:25:46.241520  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.274138  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.216660  8389 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:46.247246  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:46.254768  6768 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:25:46.219828  8349 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:46.256978  8346 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:46.222313  8324 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:46.236824  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:46.286238  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:46.233741  8241 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:46.265156  8409 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:25:46.258371  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:46.255941  8383 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 12:25:46.273370  8340 raft_consensus.cc:697] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:46.233536  8218 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:25:46.279840  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.233006  8259 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:46.278900  8349 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:46.287034  6768 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:46.280061  5948 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:46.285941  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:46.261601  5941 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:46.247365  6778 rpcz_store.cc:276] 0709 12:25:45.163491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.163648 (+   157us) service_pool.cc:224] Handling call
0709 12:25:45.193937 (+ 30289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:45.382637 (+188700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:45.382671 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:45.383263 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 12:25:45.404032 (+ 20769us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:45.404102 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.184083 (+779981us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b5400007154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:46.184133 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.184187 (+    54us) raft_consensus.cc:1685] finished
0709 12:25:46.184201 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.192329 (+  8128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2285,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":124,"spinlock_wait_cycles":428009344,"thread_start_us":20215,"threads_started":1,"wal-append.queue_time_us":110222}
I20260709 12:25:46.288993  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:46.342876  5869 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:46.291059  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:46.300346  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 715) took 1343 ms. Trace:
I20260709 12:25:46.259155  8173 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:46.316651  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:46.347728  6757 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:46.343919  6021 rpcz_store.cc:276] 0709 12:25:44.957086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.957326 (+   240us) service_pool.cc:224] Handling call
0709 12:25:44.957930 (+   604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.207722 (+249792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.299905 (+1092183us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54001072d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:46.300249 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":523886848}
W20260709 12:25:46.337988  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 101256us
I20260709 12:25:46.352385  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:46.288571  8324 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:46.354849  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 128023us
I20260709 12:25:46.219280  8391 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:46.340961  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:46.355665  5840 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:46.357863  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:46.337674  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 85338.7us
I20260709 12:25:46.363651  5838 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:46.367709  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:46.368582  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:46.370417  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:46.371672  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:46.373728  7898 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:46.373973  8050 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.374326  7898 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:46.374933  7898 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:46.378232  6024 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:46.378362  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:46.379041  5872 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:46.379557  8406 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:46.374660  8050 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:46.362694  7880 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:46.380699  7880 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:46.382920  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:46.378950  8372 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:46.313202  8289 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:46.340886  8241 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.389068  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:46.388613  5872 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:46.390367  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:46.368257  8412 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:46.316813  8176 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:46.392642  6773 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:46.279449  8377 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:46.395784  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:46.347167  8359 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.354287  8330 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.427910  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:46.428267  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:46.387532  7750 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:46.433573  6017 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:46.391322  8372 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:46.401566  8412 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.436762  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1098) took 1040 ms. Trace:
I20260709 12:25:46.422575  8385 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:46.422787  6785 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:46.423480  8144 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.861s	user 0.018s	sys 0.006s
I20260709 12:25:46.423580  8377 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:46.411489  8358 consensus_meta.cc:287] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.790s	user 0.012s	sys 0.009s
I20260709 12:25:46.424757  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:46.443280  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:46.422612  6020 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:46.444243  5941 raft_consensus.cc:2376] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4. Re-sending same reply.
I20260709 12:25:46.380107  8406 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.428979  6015 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:46.437248  5942 rpcz_store.cc:276] 0709 12:25:45.395886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.396026 (+   140us) service_pool.cc:224] Handling call
0709 12:25:46.427685 (+1031659us) spinlock_profiling.cc:243] Waited 891 us on lock 0x7b5800030398. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:25:46.436681 (+  8996us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2048896}
I20260709 12:25:46.439921  6757 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:46.440479  6022 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:46.365088  8016 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:25:46.443588  5871 raft_consensus.cc:2410] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:25:46.426913  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 71318.9us
I20260709 12:25:46.452811  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:46.425427  8211 raft_consensus.cc:2764] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:25:46.463189  8385 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:46.388625  7955 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
W20260709 12:25:46.492902  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 81737.5us
I20260709 12:25:46.467020  5838 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:46.441027  7750 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.439015  8412 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:46.441951  8383 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:46.409755  8241 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:46.341084  8389 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:46.523169  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 155557us
I20260709 12:25:46.527381  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:46.528399  5872 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 9: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:46.532128  6015 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:46.427752  8359 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:46.462679  8406 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:46.391692  8011 consensus_queue.cc:237] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:46.544531  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 172921us
I20260709 12:25:46.466817  7499 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:46.502143  8385 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.488500  8342 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:46.548486  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:46.564519  5869 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:46.565577  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:46.541788  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 154892us
I20260709 12:25:46.567133  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:46.550570  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:46.567766  5872 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:46.553334  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:46.568783  5874 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:46.556052  5870 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:46.558311  5865 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:46.521605  8413 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:25:46.439201  8330 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:46.572106  5990 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:46.523694  8415 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:46.501566  8100 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.448083  8144 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:46.504860  8274 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:46.513007  7750 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:46.561450  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:46.575253  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:46.461776  8358 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:46.582963  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:46.590353  5874 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:46.570221  8211 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:46.562865  8362 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:46.450419  8218 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:46.564525  8385 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:46.623921  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 62581.2us
I20260709 12:25:46.618619  5869 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:46.460245  8340 consensus_queue.cc:237] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:46.582149  8415 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:46.611543  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 158860us
I20260709 12:25:46.588151  8274 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:46.620586  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1079) took 3477 ms. Trace:
W20260709 12:25:46.637568  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 143502us
W20260709 12:25:46.612522  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.634301  5940 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:46.642233  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:46.643764  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:46.644497  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:46.644618  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:46.645008  6756 raft_consensus.cc:2393] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:46.645484  6022 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:46.637814  5947 rpcz_store.cc:276] 0709 12:25:43.129287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.135272 (+  5985us) service_pool.cc:224] Handling call
0709 12:25:43.135605 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:43.190532 (+ 54927us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:43.466592 (+276060us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.123022 (+656430us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:44.123091 (+    69us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:44.129553 (+  6462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.748591 (+1619038us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b18003cacc0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb642a6 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419
0709 12:25:46.606548 (+857957us) spinlock_profiling.cc:243] Waited 46.6 ms on lock 0x7b54003093d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:46.606630 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.606830 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67592,"prepare.queue_time_us":425522,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":386812,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":9742,"spinlock_wait_cycles":110615168,"thread_start_us":722955,"threads_started":3,"wal-append.queue_time_us":475209,"wal-append.run_cpu_time_us":13239,"wal-append.run_wall_time_us":1533285}
I20260709 12:25:46.658546  5915 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:46.630290  8384 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:46.606034  8100 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:46.630367  8211 raft_consensus.cc:697] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:46.669377  5915 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:46.683122  8211 consensus_queue.cc:237] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:46.685143  8428 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:25:46.683825  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 456) took 1302 ms. Trace:
I20260709 12:25:46.686281  6018 rpcz_store.cc:276] 0709 12:25:45.380975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.415233 (+ 34258us) service_pool.cc:224] Handling call
0709 12:25:46.683539 (+1268306us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b5400363dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:46.683716 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84284160}
W20260709 12:25:46.687261  8421 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 12:25:46.691401  8367 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.096s	user 0.005s	sys 0.011s
I20260709 12:25:46.657277  8426 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:25:46.692019  8367 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:46.693496  8359 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:46.637192  8415 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:46.642372  8274 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:46.699410  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 127512us
I20260709 12:25:46.654874  8390 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:46.671172  8346 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:46.674894  7499 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:46.722673  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:46.713614  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:46.732903  6768 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:25:46.646826  8388 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:46.679104  8391 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:46.633204  8419 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:25:46.674826  8430 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:46.732059  6766 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
W20260709 12:25:46.740247  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 90790.7us
I20260709 12:25:46.743429  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:46.745302  6775 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:46.745973  6768 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:46.751850  6611 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:46.747725  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:46.676165  8384 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:46.774102  6611 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:46.708393  8437 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:46.738262  8388 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.741397  8289 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:46.747937  8114 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:46.699218  8434 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.062s
W20260709 12:25:46.784113  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 139707us
I20260709 12:25:46.734735  8349 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:46.780862  8437 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.795665  5840 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:46.770897  7499 raft_consensus.cc:697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:46.773273  8430 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:25:46.802670  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.775205  8372 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:46.788053  8388 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:46.810034  8414 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.314s	user 0.001s	sys 0.001s
I20260709 12:25:46.816455  8173 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:46.816862  8357 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:46.758036  8170 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
W20260709 12:25:46.792697  8368 consensus_meta.cc:287] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.970s	user 0.012s	sys 0.004s
I20260709 12:25:46.807567  8184 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:46.801826  8241 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:46.802260  8437 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:46.816506  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 735) took 1218 ms. Trace:
I20260709 12:25:46.822898  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:46.814579  8358 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:46.822800  6762 rpcz_store.cc:276] 0709 12:25:45.598505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.928201 (+329696us) service_pool.cc:224] Handling call
0709 12:25:45.969737 (+ 41536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:45.970096 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:45.970139 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:45.997710 (+ 27571us) log.cc:844] Serialized 26 byte log entry
0709 12:25:46.206219 (+208509us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:46.206272 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.206422 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.816204 (+609782us) raft_consensus.cc:1685] finished
0709 12:25:46.816230 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.816423 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":207737,"threads_started":1,"wal-append.queue_time_us":318605}
I20260709 12:25:46.790608  8114 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:46.829748  6019 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:46.830866  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.820875  8412 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:46.807834  8346 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:25:46.843415  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 52319.9us
I20260709 12:25:46.821363  8218 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:46.808370  8419 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:46.822043  8384 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:46.835939  8184 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:46.863817  8430 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:46.835868  8330 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:46.866124  7499 consensus_queue.cc:237] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:46.853998  8289 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:46.898471  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:46.835997  8114 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:46.921169  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.896718  8431 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:25:46.923811  8368 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:46.930510  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:46.900101  8419 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.942221  5867 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:25:46.946133  5869 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:46.975916  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:46.927194  8346 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:46.946928  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:46.951975  5872 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:46.946573  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:46.989848  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:46.974728  6762 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:46.992825  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:46.993831  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:46.995435  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:46.990000  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:46.997488  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:46.993018  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:46.998530  5872 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:46.998663  5941 raft_consensus.cc:2410] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:25:47.000299  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44805.3us
I20260709 12:25:46.976893  5873 raft_consensus.cc:2376] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8. Re-sending same reply.
I20260709 12:25:46.934645  8434 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.000391  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:47.008854  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.009531  5949 raft_consensus.cc:2410] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:25:47.009665  5873 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:47.009130  8434 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:47.001560  5947 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:47.010805  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:47.002907  5942 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:46.995930  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.012430  5869 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:25:46.952600  8384 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:47.001591  5874 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 6: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:47.012916  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:46.990597  8419 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:46.942338  8100 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:47.015350  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.021876  5942 raft_consensus.cc:2368] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9.
I20260709 12:25:47.012027  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:46.976608  8330 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:46.949927  8367 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:47.024246  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.305332  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.198982  7898 consensus_meta.cc:287] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.823s	user 0.004s	sys 0.012s
W20260709 12:25:47.275542  8383 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.752s	user 0.014s	sys 0.000s
W20260709 12:25:47.186560  8456 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:25:47.288048  7880 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:47.285295  6778 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
W20260709 12:25:47.182749  8408 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 12:25:47.301244  8385 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.658s	user 0.012s	sys 0.002s
W20260709 12:25:47.252450  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 181624us
I20260709 12:25:47.201722  8274 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:47.347074  8383 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:47.181131  8452 log.cc:927] Time spent T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.098s	user 0.001s	sys 0.001s
W20260709 12:25:47.284607  8402 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.112s	user 0.006s	sys 0.017s
I20260709 12:25:47.147785  8415 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:47.100615  5913 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:47.519629  5871 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:47.078404  8367 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:47.206043  8324 consensus_meta.cc:287] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.814s	user 0.007s	sys 0.006s
W20260709 12:25:47.270408  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.060438  8330 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:46.948257  8432 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
W20260709 12:25:47.185293  8440 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:25:47.062266  8412 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:47.097292  8352 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.811s	user 0.009s	sys 0.004s
I20260709 12:25:47.183238  8435 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.213s
W20260709 12:25:47.194396  8438 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.293s	user 0.000s	sys 0.007s
I20260709 12:25:47.007566  8368 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:47.084225  8346 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:47.067776  8184 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:47.207508  8100 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.271332  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:47.202255  7750 consensus_meta.cc:287] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.581s	user 0.012s	sys 0.001s
I20260709 12:25:47.081383  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:47.208356  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.142437  8342 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:25:47.055882  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:47.301646  8381 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:47.065997  8349 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:47.284190  8259 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:47.208684  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:47.056129  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:47.204864  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:47.279578  8343 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.425s	user 0.017s	sys 0.008s
I20260709 12:25:47.338994  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.348018  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:47.349119  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:47.554113  6775 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:47.554396  5873 raft_consensus.cc:2376] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8. Re-sending same reply.
I20260709 12:25:47.360306  6778 raft_consensus.cc:606] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
W20260709 12:25:47.381758  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.390026  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:47.555644  6778 raft_consensus.cc:614] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:46.942704  8358 consensus_queue.cc:1059] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 12:25:47.303218  8144 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:47.390969  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.557109  5866 raft_consensus.cc:2368] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6.
I20260709 12:25:47.424479  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:47.424942  6020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.000s	sys 0.016s
W20260709 12:25:47.446664  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.371903  8377 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.014s	sys 0.023s
I20260709 12:25:47.290427  8437 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:47.527325  5868 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:47.556288  6793 raft_consensus.cc:2393] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:25:47.529687  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 429075us
W20260709 12:25:47.427203  8406 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.861s	user 0.008s	sys 0.008s
I20260709 12:25:47.344283  7898 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:47.544936  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:47.548691  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.533846  8324 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:47.552340  5943 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:47.421952  8385 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:47.538630  8383 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:47.557751  5872 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 3.
I20260709 12:25:47.569418  5943 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
W20260709 12:25:47.557888  6020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.001s	sys 0.017s
I20260709 12:25:47.563541  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:47.550112  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.569972  5943 pending_rounds.cc:85] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 4
I20260709 12:25:47.571350  5943 pending_rounds.cc:107] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:25:47.571636  5943 raft_consensus.cc:2889] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:47.559500  8377 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.015s	sys 0.023s
I20260709 12:25:47.572014  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:47.534587  8367 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:47.566632  5987 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.576s (SENT) [suppressed 26 similar messages]
I20260709 12:25:47.574805  5949 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:47.575369  8437 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:47.553646  8100 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:47.575915  8412 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:47.590896  5870 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 5: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:47.591822  5869 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:47.568323  8406 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:47.577710  5866 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:47.527448  8402 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:47.579957  7499 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:47.568535  8383 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.584429  5872 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:47.566159  8340 raft_consensus.cc:3055] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:47.585515  6793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:25:47.584493  5867 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:25:47.587498  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.611099  5874 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 7: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:47.546260  8352 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:47.553664  7750 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:47.552326  8381 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:47.524129  8404 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:47.621708  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 55106.5us
I20260709 12:25:47.610787  6793 raft_consensus.cc:606] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:25:47.610739  5867 pending_rounds.cc:85] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 2
W20260709 12:25:47.627439  5987 leader_election.cc:336] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.583s (SENT)
I20260709 12:25:47.627892  5867 pending_rounds.cc:107] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:25:47.628005  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:47.628320  5867 raft_consensus.cc:2889] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:47.559440  8343 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:47.621258  8340 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:47.635247  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.657513  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.658349  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.615027  8383 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:47.636040  8388 consensus_meta.cc:287] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.800s	user 0.006s	sys 0.009s
I20260709 12:25:47.636598  8437 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.623682  8367 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:47.660382  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:47.673270  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:47.673358  6775 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:47.666965  5947 raft_consensus.cc:2368] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9.
I20260709 12:25:47.659178  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:47.680714  5874 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:47.668416  5948 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:47.625684  8100 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:47.660034  8377 raft_consensus.cc:3055] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:47.688165  5944 raft_consensus.cc:2368] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7.
I20260709 12:25:47.694504  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.711429  5866 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:47.675678  8404 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:47.716372  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:47.661152  8330 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:47.728121  8432 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:47.730334  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.730732  8402 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:47.731299  5866 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:47.732913  5866 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:47.683301  8437 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:47.728339  8469 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
W20260709 12:25:47.735080  8114 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.743s	user 0.012s	sys 0.000s
I20260709 12:25:47.735749  8437 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:47.686270  8377 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:47.735971  8114 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:47.738065  8453 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:25:47.739292  7898 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:47.740831  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.741889  5866 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:47.743328  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:47.738202  8466 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 12:25:47.689541  8419 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.631s	user 0.011s	sys 0.001s
I20260709 12:25:47.681553  8388 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:47.758204  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.759258  5866 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
W20260709 12:25:47.759692  8462 log.cc:927] Time spent T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:25:47.760416  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.761175  5866 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:25:47.758224  8419 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:47.762703  8114 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:47.762601  8384 consensus_meta.cc:287] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.734s	user 0.011s	sys 0.002s
W20260709 12:25:47.758605  8464 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:25:47.756946  5874 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:25:47.770428  8352 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:47.775458  5865 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:47.777895  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 726) took 1764 ms. Trace:
I20260709 12:25:47.703800  8368 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:47.779595  6024 rpcz_store.cc:276] 0709 12:25:46.022326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.102572 (+ 80246us) service_pool.cc:224] Handling call
0709 12:25:47.777360 (+1674788us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b5400160b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:47.777554 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591654144}
W20260709 12:25:47.743975  8467 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:25:47.702994  8385 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:47.782876  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:47.805922  8415 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:47.743944  8324 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:47.695194  8274 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:47.721699  8435 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.792363  8343 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:47.794881  8406 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:47.760478  8388 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:47.737303  8377 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:47.799309  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:47.799978  6793 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:47.800458  6778 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:47.699344  8340 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:47.810997  6768 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:47.814801  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:47.821398  6762 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:47.825098  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:47.826416  6768 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:47.823362  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:47.845039  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:47.846485  6778 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:47.855720  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:47.856045  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.809155  7898 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:47.858413  6762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:25:47.871783  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1095) took 2478 ms. Trace:
I20260709 12:25:47.871824  6762 raft_consensus.cc:606] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:25:47.872519  5946 rpcz_store.cc:276] 0709 12:25:45.393355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.393513 (+   158us) service_pool.cc:224] Handling call
0709 12:25:45.393965 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.394280 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:45.415753 (+ 21473us) log.cc:844] Serialized 14 byte log entry
0709 12:25:46.588040 (+1172287us) spinlock_profiling.cc:243] Waited 4.48 ms on lock 0x7b18003caf60. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:25:46.708334 (+120294us) log.cc:844] Serialized 14 byte log entry
0709 12:25:46.897432 (+189098us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b18003caf60. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:25:47.702541 (+805109us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b18003caf60. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287 00007f39d2c9e208
0709 12:25:47.870820 (+168279us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.871121 (+   301us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:47.871239 (+   118us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:47.871315 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.871456 (+   141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.871679 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33291,"prepare.queue_time_us":65513,"prepare.run_cpu_time_us":2688,"prepare.run_wall_time_us":25389,"raft.queue_time_us":48605,"raft.run_cpu_time_us":360,"raft.run_wall_time_us":41304,"spinlock_wait_cycles":116807552,"thread_start_us":263069,"threads_started":2,"wal-append.queue_time_us":1476109}
I20260709 12:25:47.875039  6762 raft_consensus.cc:614] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:47.861141  8330 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:47.877368  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 158307us
I20260709 12:25:47.818089  8324 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.711831  8381 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:47.910735  8381 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:47.883811  8014 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:47.930099  8389 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:47.928865  8383 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:47.814330  8435 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:47.938575  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.888556  8377 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:47.915827  8340 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:47.950660  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 134328us
W20260709 12:25:47.953958  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.955284  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 98142us
I20260709 12:25:47.959129  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:47.960260  5942 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
W20260709 12:25:47.983160  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.984912  8463 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.309s	user 0.003s	sys 0.000s
I20260709 12:25:47.986260  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:47.986636  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:47.987107  6778 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:25:47.987208  6021 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:47.988363  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:47.988945  6021 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:47.948781  8404 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:47.990430  6762 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:47.990317  8404 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:47.991705  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:47.992511  6762 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 6: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:47.992588  8404 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:47.947456  8477 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.993552  8477 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:47.993994  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:47.995311  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:47.995965  6793 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:47.996204  8477 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:47.996906  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:47.957737  8114 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.997494  6775 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:47.997825  8114 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:47.951506  8343 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.000267  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 369) took 1474 ms. Trace:
I20260709 12:25:47.958511  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:48.000917  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:47.986286  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:48.003373  5871 rpcz_store.cc:276] 0709 12:25:46.526095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.526462 (+   367us) service_pool.cc:224] Handling call
0709 12:25:46.526980 (+   518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.524249 (+997269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.999803 (+475554us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b54002f99d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:48.000142 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2282030848}
I20260709 12:25:48.005407  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:48.006811  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.000571  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:48.007902  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 49496.5us
I20260709 12:25:47.993317  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:48.009220  6762 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:48.009496  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:48.005191  6756 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:47.885003  7898 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:47.889235  8324 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:48.015578  6775 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:48.018529  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:47.946143  8383 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:48.028720  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.998143  8434 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:48.002032  8343 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:48.041462  6756 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:48.012945  8477 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:48.036785  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 778) took 1022 ms. Trace:
I20260709 12:25:48.047308  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:48.048041  6018 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:48.047065  5869 rpcz_store.cc:276] 0709 12:25:47.014868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.015071 (+   203us) service_pool.cc:224] Handling call
0709 12:25:48.036445 (+1021374us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b540043f654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:48.036688 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1309694208}
I20260709 12:25:48.028028  8404 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.034770  8274 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:48.068166  6021 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:25:48.048447  8477 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:48.079048  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:48.105980  8477 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:48.106245  6756 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:48.106631  5873 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:48.088662  5944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 815) took 2229 ms (client timeout 1953 ms). Trace:
W20260709 12:25:48.108829  5944 rpcz_store.cc:269] 0709 12:25:45.859567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.925367 (+ 65800us) service_pool.cc:224] Handling call
0709 12:25:48.088273 (+2162906us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b5400309dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:48.088512 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3799546752}
I20260709 12:25:48.114460  8477 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:48.116701  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:48.090245  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1103) took 1815 ms. Trace:
I20260709 12:25:48.118151  5915 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:48.118760  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 825) took 2104 ms. Trace:
I20260709 12:25:48.119158  5940 rpcz_store.cc:276] 0709 12:25:46.014177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.269747 (+255570us) service_pool.cc:224] Handling call
0709 12:25:46.345673 (+ 75926us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.642032 (+296359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.118490 (+1476458us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b5400188b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:48.118690 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656098560}
I20260709 12:25:48.119597  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:48.120872  8383 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:48.120807  8426 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:48.099005  8324 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:48.121680  8383 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:48.122076  8324 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.066571  8274 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.101130  5941 raft_consensus.cc:2410] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:25:48.053066  8486 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:48.122855  8477 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:48.122897  8426 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:48.117110  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:48.123036  8324 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:48.124918  8139 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:48.125516  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:48.126329  8324 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:48.127106  8391 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.127753  8391 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:48.126372  6021 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:48.123883  8426 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:48.130817  8426 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:48.131657  8391 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:48.134275  8419 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:48.127058  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:48.136464  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 468) took 1333 ms. Trace:
I20260709 12:25:48.137789  8342 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:48.138285  8342 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:48.138751  8342 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:48.102428  6773 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 605) took 1879 ms (client timeout 1948 ms). Trace:
W20260709 12:25:48.103241  6609 connection.cc:256] RPC call timeout handler was delayed by -1.8438943249998374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
I20260709 12:25:48.103653  8343 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:48.054999  8404 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:48.105057  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:48.122130  8383 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:48.123857  8274 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.101929  8435 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:48.101380  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 739) took 2435 ms. Trace:
I20260709 12:25:48.143314  6778 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:25:48.151798  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 469) took 1325 ms. Trace:
I20260709 12:25:48.157552  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:48.167055  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:48.166415  8404 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:48.141562  7898 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:48.117796  5948 rpcz_store.cc:276] 0709 12:25:46.274395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.355571 (+ 81176us) service_pool.cc:224] Handling call
0709 12:25:46.391995 (+ 36424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.683302 (+1291307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.089866 (+406564us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400602f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:48.090153 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2918930688}
I20260709 12:25:48.268836  6785 rpcz_store.cc:276] 0709 12:25:45.666093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.152221 (+486128us) service_pool.cc:224] Handling call
0709 12:25:46.186627 (+ 34406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.439389 (+252762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.101006 (+1661617us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b5400005d54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:48.101273 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":542530432}
I20260709 12:25:48.272437  5946 raft_consensus.cc:2410] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:25:48.150986  8342 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:48.165339  8383 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:48.167289  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:48.309813  8342 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:48.156419  6021 raft_consensus.cc:2410] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:48.166796  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:48.313738  5949 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:25:48.158656  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:48.159896  6609 connection.cc:256] RPC call timeout handler was delayed by -1.7277943250001044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1845s for a response
W20260709 12:25:48.315102  6609 connection.cc:256] RPC call timeout handler was delayed by -1.7106943250000768s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1864s for a response
W20260709 12:25:48.315521  6609 connection.cc:256] RPC call timeout handler was delayed by -1.707094325000071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 12:25:48.316170  6609 connection.cc:256] RPC call timeout handler was delayed by -1.6827943250000317s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1895s for a response
W20260709 12:25:48.316478  6609 connection.cc:256] RPC call timeout handler was delayed by -1.5756943249998585s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2014s for a response
I20260709 12:25:48.316629  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:48.165656  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 767) took 1220 ms. Trace:
I20260709 12:25:48.163446  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:48.150450  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.144914  6762 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:48.317595  5868 rpcz_store.cc:276] 0709 12:25:46.945709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.946267 (+   558us) service_pool.cc:224] Handling call
0709 12:25:48.165383 (+1219116us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b5400119754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:48.165542 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1229549312}
I20260709 12:25:48.203496  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:48.281807  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 272356us
I20260709 12:25:48.165019  6775 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
W20260709 12:25:48.159545  6773 rpcz_store.cc:269] 0709 12:25:46.223095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.234210 (+ 11115us) service_pool.cc:224] Handling call
0709 12:25:48.102158 (+1867948us) spinlock_profiling.cc:243] Waited 99.1 ms on lock 0x7b5400003554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:48.102314 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227963776}
I20260709 12:25:48.147516  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:48.284058  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:48.163818  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:48.322238  8364 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.322433  5944 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:25:48.322743  8435 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:48.322898  8364 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:48.323866  8435 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:48.267521  6022 rpcz_store.cc:276] 0709 12:25:46.826350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.023675 (+197325us) service_pool.cc:224] Handling call
0709 12:25:47.067538 (+ 43863us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.077278 (+  9740us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.077328 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.083516 (+  6188us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.051884 (+968368us) spinlock_profiling.cc:243] Waited 6.33 ms on lock 0x7b180011b4e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:25:48.151236 (+ 99352us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:48.151298 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.151385 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.151434 (+    49us) raft_consensus.cc:1685] finished
0709 12:25:48.151444 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.151725 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12620,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":35264,"spinlock_wait_cycles":14564096,"thread_start_us":99106,"threads_started":1,"wal-append.queue_time_us":1015926}
I20260709 12:25:48.299125  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:48.401515  6756 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:48.307318  5947 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 7: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:48.162214  8050 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:48.316759  6609 connection.cc:256] RPC call timeout handler was delayed by -1.5540943250002783s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
I20260709 12:25:48.325323  8346 raft_consensus.cc:3055] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:48.326957  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:48.404069  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:48.327549  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:48.327821  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.325083  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:48.328729  8259 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.413975  7898 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:48.413815  8259 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:48.414559  7898 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.415004  7898 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:48.414836  5695 catalog_manager.cc:5697] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 5 to 6, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 6 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:48.416462  8259 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:48.154662  6018 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:48.331187  8100 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:48.404556  8426 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:25:48.331111  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 703599us
I20260709 12:25:48.418982  8426 raft_consensus.cc:697] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:48.333794  8014 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:48.420044  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 728) took 1779 ms. Trace:
I20260709 12:25:48.417083  5695 catalog_manager.cc:5697] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 5 to 6, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 6 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:48.339488  8457 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.664s	user 0.002s	sys 0.000s
I20260709 12:25:48.394452  8147 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.403816  8016 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:25:48.420082  8426 consensus_queue.cc:237] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:48.421100  8147 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:48.422065  5695 catalog_manager.cc:5697] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 3 to 5, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 5 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:25:48.423508  8147 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:48.423722  5695 catalog_manager.cc:5697] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 5 to 6, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 6 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:48.424950  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:48.425549  5695 catalog_manager.cc:5697] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 3 to 5, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 5 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:25:48.420506  6015 rpcz_store.cc:276] 0709 12:25:46.643583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.658293 (+ 14710us) service_pool.cc:224] Handling call
0709 12:25:46.658741 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.742505 (+ 83764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.742542 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:48.110700 (+1368158us) spinlock_profiling.cc:243] Waited 3.88 ms on lock 0x7b2c001ddd28. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:25:48.111402 (+   702us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.253590 (+142188us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:48.278048 (+ 24458us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.338067 (+ 60019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.338184 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.419695 (+ 81511us) raft_consensus.cc:1685] finished
0709 12:25:48.419796 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.419970 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59642,"prepare.queue_time_us":23867,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":76569,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":229,"spinlock_wait_cycles":8921344,"thread_start_us":210467,"threads_started":2,"wal-append.queue_time_us":273714}
I20260709 12:25:48.335368  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:48.428758  6016 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
W20260709 12:25:48.344045  8424 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.885s	user 0.006s	sys 0.009s
I20260709 12:25:48.330183  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 730) took 1683 ms. Trace:
I20260709 12:25:48.429847  5695 catalog_manager.cc:5697] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 3 to 5, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 5 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:48.431604  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:48.348600  5870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 769) took 1397 ms (client timeout 1638 ms). Trace:
W20260709 12:25:48.432052  5870 rpcz_store.cc:269] 0709 12:25:46.950770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.950960 (+   190us) service_pool.cc:224] Handling call
0709 12:25:48.348316 (+1397356us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b54001d06d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:48.348520 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1370537344}
W20260709 12:25:48.342845  5941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 351) took 1369 ms (client timeout 1684 ms). Trace:
I20260709 12:25:48.349403  8114 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:48.360045  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:48.325197  8364 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:48.364917  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:48.366600  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:48.441453  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 467) took 1638 ms. Trace:
I20260709 12:25:48.367805  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:48.372304  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:48.440968  5874 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:48.372515  5948 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:48.376451  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:48.443410  5869 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:48.446231  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:48.385965  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:48.447418  6773 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:48.393072  5944 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:48.395308  5838 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:48.395936  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:48.396061  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 248646us
W20260709 12:25:48.448839  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 53621.9us
I20260709 12:25:48.449199  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:48.331266  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:48.443441  5695 catalog_manager.cc:5697] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 4 to 6, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 6 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:48.450544  6775 raft_consensus.cc:2410] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:25:48.430750  6020 rpcz_store.cc:276] 0709 12:25:46.647334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.647536 (+   202us) service_pool.cc:224] Handling call
0709 12:25:46.667362 (+ 19826us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.667698 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:46.784933 (+117235us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.329818 (+1544885us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:48.329879 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:48.329931 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.329993 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.330118 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":819153,"prepare.queue_time_us":228004,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":757585,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":904724,"threads_started":2,"wal-append.queue_time_us":837375}
I20260709 12:25:48.455286  5940 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:48.397715  5913 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:48.325114  8435 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:25:48.404556  6609 connection.cc:256] RPC call timeout handler was delayed by -1.4523943250001139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
I20260709 12:25:48.409005  6017 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
W20260709 12:25:48.409152  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 82278.9us
W20260709 12:25:48.475873  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.406800  5990 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:48.479835  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:48.481070  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:48.409768  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:48.479823  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:48.483768  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 77065us
I20260709 12:25:48.420223  8014 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:48.432832  5941 rpcz_store.cc:269] 0709 12:25:46.973027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.016791 (+ 43764us) service_pool.cc:224] Handling call
0709 12:25:48.342597 (+1325806us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b5400632254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:48.342769 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1269100416}
I20260709 12:25:48.136879  6023 rpcz_store.cc:276] 0709 12:25:46.803305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.975544 (+172239us) service_pool.cc:224] Handling call
0709 12:25:46.975964 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.976204 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:47.054776 (+ 78572us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.315818 (+261042us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.135915 (+820097us) spinlock_profiling.cc:243] Waited 86.4 ms on lock 0x7b1800052840. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb10c0a 00007f39d9250219 00007f39cfb98a40 00007f39cfb934e4 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287 00007f39d2c9e208
0709 12:25:48.136083 (+   168us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:48.136139 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:48.136191 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.136275 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.136400 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56308,"prepare.queue_time_us":123068,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":2484,"raft.queue_time_us":871349,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":17957,"spinlock_wait_cycles":198756736,"thread_start_us":226027,"threads_started":1,"wal-append.queue_time_us":556790}
I20260709 12:25:48.485246  5990 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:48.418138  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:48.436875  5945 raft_consensus.cc:2368] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6.
I20260709 12:25:48.441907  6019 rpcz_store.cc:276] 0709 12:25:46.802949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.861900 (+ 58951us) service_pool.cc:224] Handling call
0709 12:25:46.887874 (+ 25974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.895093 (+  7219us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:47.148002 (+252909us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.149349 (+1001347us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b1800008d40. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:25:48.441015 (+291666us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:48.441078 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:48.441129 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.441212 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.441371 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53489,"prepare.queue_time_us":188494,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"spinlock_wait_cycles":55272192,"thread_start_us":406638,"threads_started":2}
I20260709 12:25:48.469439  5874 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:48.487877  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:48.386919  7750 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:48.433547  8346 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:48.493397  5947 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:48.495200  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:48.503638  6768 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 7: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:48.504065  6775 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:25:48.497963  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:48.486954  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 89332.9us
I20260709 12:25:48.505102  5869 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:48.505820  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:48.506506  6019 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.511186  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:48.511919  6020 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:25:48.492123  8412 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:48.512123  5990 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:48.512265  6762 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:48.513880  8386 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:48.514460  8386 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.514545  5869 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:45236
I20260709 12:25:48.514784  8050 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:48.515002  8386 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:48.515399  5869 raft_consensus.cc:606] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:25:48.515918  5869 raft_consensus.cc:614] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:48.521098  5990 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:48.524165  6762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:25:48.525121  6762 raft_consensus.cc:606] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:25:48.525563  6762 raft_consensus.cc:614] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:48.526994  8384 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:48.527937  8384 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:48.529129  8384 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:48.530589  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:48.532253  8486 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:48.533495  8340 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:48.534153  8340 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:48.535307  8340 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:48.409752  8342 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 12:25:48.502344  8404 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:48.541112  8404 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.541577  8404 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:48.473173  8492 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:48.543385  8492 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.543859  8492 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:48.550172  8346 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:48.502956  8364 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:48.553311  8364 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.553817  8364 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:48.555583  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:48.556684  6023 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 9: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:48.564062  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:48.501798  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:48.566967  5869 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 7: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:25:48.565235  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.568198  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:48.568943  6023 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:48.503998  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:48.571189  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 67272.1us
I20260709 12:25:48.573108  6609 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:48.500394  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:48.574293  6022 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:48.575930  6773 raft_consensus.cc:3055] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:48.576514  6773 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:48.563925  5695 catalog_manager.cc:5697] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 3, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 3 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:25:48.579149  6785 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 12:25:48.593402  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 88264.1us
I20260709 12:25:48.595234  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:48.563990  8340 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:48.597134  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:48.598500  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:48.599087  6015 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:48.599501  6018 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:48.600689  6017 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:48.601542  8430 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:48.602548  8430 raft_consensus.cc:3055] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:48.602893  8430 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:48.603822  8430 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:48.605440  8430 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:48.606052  5695 catalog_manager.cc:5697] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 3, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 3 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:48.473474  8435 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:48.515256  8050 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:48.629606  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:48.620388  5694 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50588 (request call id 54) took 26170 ms (client timeout 15000 ms). Trace:
I20260709 12:25:48.620422  8434 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:25:48.620952  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 57749.8us
I20260709 12:25:48.621829  5695 catalog_manager.cc:5697] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 3 to 5, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 5 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:48.485795  8147 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:48.630856  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:48.635708  5869 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:48.637486  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:48.638944  6766 raft_consensus.cc:3055] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:48.639453  6766 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:25:48.632269  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.617251  6015 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:48.494980  8377 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:48.635938  5840 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:48.583753  8346 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:48.606598  8358 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:48.645733  5695 catalog_manager.cc:5697] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 3 to 5, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 5 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:25:48.647672  5915 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:48.654831  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:48.642661  8114 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:25:48.659058  8434 raft_consensus.cc:993] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: : Instructing follower 6add331326ca47038308a447bf798fa7 to start an election
I20260709 12:25:48.659770  8486 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:25:48.664165  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:48.651332  6766 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:48.672667  5915 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:48.697418  5695 catalog_manager.cc:5697] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 5 to 6, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 6 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:48.634153  8050 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:48.698889  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:48.699793  6766 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:48.710083  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:48.701799  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:48.711539  6016 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:48.700506  8434 raft_consensus.cc:1081] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Signalling peer 6add331326ca47038308a447bf798fa7 to start an election
I20260709 12:25:48.708069  8389 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:48.656143  7750 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:48.640036  7898 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:48.702219  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:48.719493  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 124332us
I20260709 12:25:48.721828  5838 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:48.618096  6773 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:48.700745  8147 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:48.660104  8477 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:48.653451  8432 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:48.649994  8377 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:48.760479  6773 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:48.770315  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.771255  6019 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:48.781782  5695 catalog_manager.cc:5697] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 3 to 6, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 6 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
W20260709 12:25:48.794953  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 51599.4us
I20260709 12:25:48.753170  6775 raft_consensus.cc:3055] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:48.799940  6775 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:48.723842  8343 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:48.799696  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:48.806818  6756 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:48.807087  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:48.795058  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 64694.4us
I20260709 12:25:48.807875  6793 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:25:48.803249  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:48.808130  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:48.812757  6015 raft_consensus.cc:2368] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6.
I20260709 12:25:48.814709  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:48.723676  8389 raft_consensus.cc:697] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:48.824034  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:48.824760  6768 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:48.826980  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:48.829054  6015 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:48.731971  8357 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:48.732872  8342 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:48.843781  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:48.800915  6775 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:48.838256  5840 leader_election.cc:400] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6.
I20260709 12:25:48.845036  5840 leader_election.cc:403] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:25:48.845273  6775 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.754810  8147 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.775067  8016 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:48.631759  5694 rpcz_store.cc:269] 0709 12:25:22.492744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.493175 (+   431us) service_pool.cc:224] Handling call
0709 12:25:48.620087 (+26126912us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:38.461785 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:38.634225 (+172440us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:38.642136 (+  7911us) write_op.cc:435] Acquired schema lock
0709 12:25:38.644616 (+  2480us) tablet.cc:662] Decoding operations
0709 12:25:38.860145 (+215529us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:38.860245 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:38.860275 (+    30us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:25:38.909481 (+ 49206us) tablet.cc:701] Row locks acquired
0709 12:25:38.923606 (+ 14125us) write_op.cc:260] PREPARE: finished
0709 12:25:38.940376 (+ 16770us) write_op.cc:270] Start()
0709 12:25:38.967902 (+ 27526us) write_op.cc:276] Timestamp: P: 1783599938937803 usec, L: 0
0709 12:25:38.967929 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:25:38.977666 (+  9737us) log.cc:844] Serialized 7407 byte log entry
0709 12:25:40.273836 (+1296170us) spinlock_profiling.cc:243] Waited 54.9 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39cfb790ac 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b
0709 12:25:42.124217 (+1850381us) op_driver.cc:464] REPLICATION: finished
0709 12:25:42.609105 (+484888us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39cfb790ac 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d66b9ef0
0709 12:25:43.065537 (+456432us) write_op.cc:301] APPLY: starting
0709 12:25:43.065714 (+   177us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:43.066289 (+   575us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:43.066327 (+    38us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:43.068683 (+  2356us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:43.069166 (+   483us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=19
0709 12:25:43.069230 (+    64us) write_op.cc:312] APPLY: finished
0709 12:25:43.167492 (+ 98262us) log.cc:844] Serialized 131 byte log entry
0709 12:25:43.405887 (+238395us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:43.506822 (+100935us) write_op.cc:454] Released schema lock
0709 12:25:43.506913 (+    91us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:43.507605 (+   692us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:43.507644 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4355060,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":928612,"apply.run_cpu_time_us":44439,"apply.run_wall_time_us":442029,"num_ops":19,"prepare.queue_time_us":293841,"prepare.run_cpu_time_us":281900,"prepare.run_wall_time_us":2756314,"raft.queue_time_us":632891,"raft.run_cpu_time_us":24430,"raft.run_wall_time_us":1056450,"replication_time_us":3090615,"spinlock_wait_cycles":136473216,"thread_start_us":3952676,"threads_started":5,"wal-append.queue_time_us":2413750,"wal-append.run_cpu_time_us":21884,"wal-append.run_wall_time_us":2993375}]]}
I20260709 12:25:48.868265  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:48.869148  6785 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
W20260709 12:25:48.857800  8289 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.054s	sys 0.068s
I20260709 12:25:48.859565  6762 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:48.839934  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.850422  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:48.838294  8340 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:48.806433  8343 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:48.879232  6017 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:48.726874  8324 consensus_queue.cc:1059] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.109s
I20260709 12:25:48.811343  8477 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:48.751071  6021 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:48.903252  6756 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:48.844732  6019 raft_consensus.cc:2368] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6.
I20260709 12:25:48.910476  5695 catalog_manager.cc:5697] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 4 to 5, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 5 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:48.880779  8289 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.054s	sys 0.069s
I20260709 12:25:48.916901  8477 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:48.918601  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:48.919432  6021 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:48.920428  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:48.921064  6021 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:48.922392  6021 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:48.922657  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:48.869189  8147 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:48.880584  8016 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:48.889444  8274 consensus_meta.cc:287] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.724s	user 0.019s	sys 0.000s
I20260709 12:25:48.927778  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:48.849900  8342 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:25:48.930341  8426 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.007s	sys 0.032s
I20260709 12:25:48.897223  8343 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:48.951610  8434 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:48.959718  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.967487  6018 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:48.913241  8432 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:25:48.800665  8435 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:49.037657  7898 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:49.029459  8426 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.008s	sys 0.032s
I20260709 12:25:48.878677  8357 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:48.939215  8340 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:49.010720  6017 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:48.982581  8358 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:48.986260  5695 catalog_manager.cc:5697] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 1 to 4, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 4 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:48.973213  6020 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:49.012578  6793 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:25:48.977754  8377 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:48.996230  8412 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:49.019353  6766 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:25:48.997326  8383 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.681s	user 0.010s	sys 0.000s
W20260709 12:25:49.021118  5987 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.006s	sys 0.011s
I20260709 12:25:48.975004  8491 raft_consensus.cc:3055] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:25:49.028378  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 413220us
I20260709 12:25:49.021242  6019 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:49.029794  8342 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:49.096766  5695 catalog_manager.cc:5697] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 2 to 4, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 4 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:48.995220  8386 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:49.020115  8016 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:49.038031  8435 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:49.017077  6775 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:25:49.112185  6775 pending_rounds.cc:85] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 3
I20260709 12:25:49.112430  6018 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:25:49.112661  6775 pending_rounds.cc:107] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:25:49.113440  6775 raft_consensus.cc:2889] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:49.102286  5987 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.008s	sys 0.012s
W20260709 12:25:49.115360  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 695722us
I20260709 12:25:49.103484  6611 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:49.092118  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 742) took 1085 ms. Trace:
I20260709 12:25:49.040623  8357 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:48.975004  8389 consensus_queue.cc:237] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:49.035375  6015 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
W20260709 12:25:49.055589  5871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.016s	sys 0.036s
W20260709 12:25:49.122382  5871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.018s	sys 0.037s
I20260709 12:25:49.107710  8412 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:49.119874  6020 rpcz_store.cc:276] 0709 12:25:47.984471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.480859 (+496388us) service_pool.cc:224] Handling call
0709 12:25:49.069381 (+588522us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.015193  6773 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:49.119347  6017 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
W20260709 12:25:49.170845  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.150573  6015 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:49.127425  6019 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:49.179308  6018 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:49.136186  6021 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:49.019661  8364 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:49.068064  8358 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:49.129838  8501 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:25:49.031630  8274 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:49.131091  8500 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:25:49.117507  7898 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:49.097445  8383 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:49.195936  5944 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:49.198180  8383 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:49.198931  8383 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:49.199497  8383 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:49.196048  7898 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:25:49.141906  7880 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.014s	sys 0.014s
I20260709 12:25:49.175392  8357 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:49.072082  8491 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:49.190197  5945 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:49.191967  5941 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:49.192968  5947 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:49.195492  6022 raft_consensus.cc:2368] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 9. Current term is 10.
I20260709 12:25:49.174569  8412 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:49.207672  7880 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.014s	sys 0.014s
W20260709 12:25:49.232461  6023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.003s	sys 0.037s
W20260709 12:25:49.212631  8014 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.673s	user 0.008s	sys 0.004s
W20260709 12:25:49.210263  8430 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.584s	user 0.008s	sys 0.005s
W20260709 12:25:49.228240  6016 consensus_meta.cc:287] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.513s	user 0.013s	sys 0.000s
I20260709 12:25:49.252601  6016 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
W20260709 12:25:49.243994  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.255311  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 444151us
I20260709 12:25:49.250528  8383 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:49.249773  7750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.000s	sys 0.027s
W20260709 12:25:49.235980  6023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.003s	sys 0.037s
W20260709 12:25:49.232499  8492 consensus_meta.cc:287] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.688s	user 0.012s	sys 0.002s
W20260709 12:25:49.258330  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.271736  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.271996  5838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.006s	sys 0.045s
I20260709 12:25:49.268313  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 732) took 1286 ms. Trace:
W20260709 12:25:49.244869  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 141461us
I20260709 12:25:49.221879  8357 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:49.265358  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:49.258206  8381 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:49.207321  7898 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:49.234127  8412 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:49.243894  5987 leader_election.cc:336] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.638s (SENT)
W20260709 12:25:49.268608  7750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.000s	sys 0.028s
I20260709 12:25:49.254833  8014 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:49.274161  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 737) took 1290 ms. Trace:
I20260709 12:25:49.274691  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:49.298280  5913 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:49.299256  5913 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:49.272997  8383 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:25:49.304275  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.291271  7750 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:25:49.290004  8404 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.748s	user 0.009s	sys 0.006s
I20260709 12:25:49.272843  6024 rpcz_store.cc:276] 0709 12:25:47.981960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.985150 (+  3190us) service_pool.cc:224] Handling call
0709 12:25:47.985591 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:48.336081 (+350490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:48.336116 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:48.336630 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.734085 (+397455us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:48.734147 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.734431 (+   284us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.267925 (+533494us) raft_consensus.cc:1685] finished
0709 12:25:49.267960 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.268210 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":349934,"raft.queue_time_us":351589,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":62,"thread_start_us":396970,"threads_started":1,"wal-append.queue_time_us":665851}
I20260709 12:25:49.284972  8324 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:25:49.274824  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 439386us
I20260709 12:25:49.335726  6015 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:49.283524  8381 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.287690  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:49.309953  8386 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:49.311117  8358 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:49.341401  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:49.343222  6609 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:25:49.273799  5838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.007s	sys 0.046s
W20260709 12:25:49.346808  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 617558us
I20260709 12:25:49.315654  8342 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:49.321731  8377 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:49.276021  8492 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:49.301476  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.348731  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.302758  8364 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:49.295519  6023 rpcz_store.cc:276] 0709 12:25:47.983315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.608176 (+624861us) service_pool.cc:224] Handling call
0709 12:25:48.608620 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.608900 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:48.641924 (+ 33024us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.273608 (+631684us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.273715 (+   107us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:49.273772 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.273876 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.274074 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34462,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":5240,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":8904,"thread_start_us":626777,"threads_started":1}
W20260709 12:25:49.312098  8050 consensus_meta.cc:287] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.587s	user 0.010s	sys 0.003s
I20260709 12:25:49.313658  8357 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:49.333454  5840 leader_election.cc:400] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6.
I20260709 12:25:49.360277  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:49.312100  8346 consensus_meta.cc:287] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.657s	user 0.011s	sys 0.000s
I20260709 12:25:49.354212  6609 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:49.355403  5838 leader_election.cc:400] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6.
I20260709 12:25:49.371351  5840 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:49.344480  8381 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:49.371917  5838 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:49.345403  8114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.020s	sys 0.033s
I20260709 12:25:49.351702  8430 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:49.332151  8404 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:49.332454  8383 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:49.350224  6022 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519"
dest_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:34888
I20260709 12:25:49.381400  6022 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:49.381824  6022 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:49.358208  8358 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:49.380402  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 264134us
I20260709 12:25:49.344250  8324 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:49.377924  8114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.020s	sys 0.033s
I20260709 12:25:49.374418  8357 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:25:49.396545  5987 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.681s (SENT)
I20260709 12:25:49.396660  5990 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:49.379508  8346 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.345829  8491 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:49.382462  8274 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:49.400743  8289 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:49.413113  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:49.418344  6016 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:49.367888  8050 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:49.408951  8364 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:25:49.421056  8364 raft_consensus.cc:3055] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:49.421408  8364 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:49.384044  7750 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:49.396121  8117 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:49.384282  8511 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:49.422216  8383 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:49.414202  8506 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:25:49.414949  8432 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:49.412609  8494 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.224s	user 0.000s	sys 0.002s
I20260709 12:25:49.444483  8014 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:49.408315  8016 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:49.451109  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.452172  8050 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:49.437139  6022 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:49.462677  8324 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:49.407809  8430 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:49.407713  8274 raft_consensus.cc:2764] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:49.430006  8419 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:49.470788  8117 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:49.439832  6016 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
W20260709 12:25:49.473724  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.425971  8289 raft_consensus.cc:697] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:49.449350  8340 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6.
I20260709 12:25:49.414350  8346 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:49.429378  8513 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:25:49.493378  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.522830  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 151007us
W20260709 12:25:49.523758  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.476946  8432 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.405337  8114 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.102s
I20260709 12:25:49.527794  6022 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:49.539285  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:49.539706  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:49.540328  6773 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:49.540807  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:49.483310  8430 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.541498  6766 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:49.514784  8324 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:49.541900  6766 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:49.540809  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:49.546185  6775 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:49.545486  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 174240us
I20260709 12:25:49.548403  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:49.513967  8419 raft_consensus.cc:697] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:49.559017  5941 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:49.422390  8364 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:49.560953  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.558760  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:49.563396  6766 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:49.566296  8289 consensus_queue.cc:237] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:49.567243  8346 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:49.567984  8346 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:49.568370  8346 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.568710  8346 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:49.536661  8432 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:49.573267  5947 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:49.544872  8430 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:49.535691  8513 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:49.545456  8324 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:49.552413  8516 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:49.580353  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 226236us
I20260709 12:25:49.589453  8516 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:49.589923  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:49.590620  8505 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:25:49.591403  6609 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:49.593024  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:49.589977  8513 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:49.595960  8519 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:49.581032  6773 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:25:49.582290  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:49.584626  6775 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:49.543716  8340 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:25:49.613890  8117 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:49.616225  8419 consensus_queue.cc:237] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:49.625377  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.626210  8404 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:49.647416  5990 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:49.639034  8324 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:49.632429  8509 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 12:25:49.634150  8377 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:49.575753  8512 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 12:25:49.643622  8517 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:25:49.636577  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.654495  8512 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:49.655090  8512 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:49.656709  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:25:49.657600  6773 raft_consensus.cc:606] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:25:49.658017  6773 raft_consensus.cc:614] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:49.637385  8016 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:49.569968  8274 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 12:25:49.574357  8510 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 12:25:49.638244  8364 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:49.638705  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.639209  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 339989us
I20260709 12:25:49.563638  8114 consensus_queue.cc:1243] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:49.601840  8518 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.648361  5990 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:49.630157  8358 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:49.586375  8492 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:49.652011  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 69873.3us
I20260709 12:25:49.677084  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:49.679190  5867 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:25:49.680702  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.658303  6775 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:49.656836  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:49.659442  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.660796  8486 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:49.680454  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:49.658459  8384 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:49.579733  8342 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:49.703666  5868 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:49.703816  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:49.705858  5941 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:49.710059  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:49.711212  5948 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 8: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:49.712064  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:49.686590  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:49.692751  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 89291.8us
I20260709 12:25:49.694804  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:49.714874  5867 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:49.695439  8330 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:49.695353  8524 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:49.696264  8492 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:49.698210  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:49.698686  8526 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:49.733318  8526 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:49.700495  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:49.700769  8384 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:49.676307  8358 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:25:49.702389  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.890s	user 0.016s	sys 0.042s
W20260709 12:25:49.743031  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.016s	sys 0.042s
W20260709 12:25:49.743323  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 940066us
I20260709 12:25:49.684324  5942 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:49.717487  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:49.743505  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 64470.1us
I20260709 12:25:49.744123  5947 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:49.674211  7880 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:49.745682  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:49.746157  7880 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:49.685010  8486 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:49.720849  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:25:49.722821  8147 consensus_meta.cc:287] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.691s	user 0.014s	sys 0.001s
I20260709 12:25:49.679389  8518 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:49.683986  8016 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.727370  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:49.729858  8330 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:49.730664  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:49.779217  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:49.700012  8521 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:49.735188  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:49.740214  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:49.764120  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 81259.7us
I20260709 12:25:49.775074  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:49.788697  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:49.789920  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:49.790689  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.790673  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:49.747431  7880 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:49.780285  5940 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:49.758198  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:49.783152  8514 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:25:49.783849  8404 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:49.674412  8525 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 12:25:49.779877  5944 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:49.798945  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:49.787281  5941 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:49.800217  5840 leader_election.cc:400] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 9. Current term is 10.
I20260709 12:25:49.804533  5840 leader_election.cc:403] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:25:49.733968  8526 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:49.805603  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:49.748126  8358 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:49.658069  8512 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:49.820498  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.775588  7750 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 12:25:49.782864  5867 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:49.808020  5947 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:25:49.844635  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:49.847072  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.733618  8430 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:49.798427  8016 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:49.848987  8340 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:49.849710  8340 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:49.850404  8430 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:49.851208  8016 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:49.837884  8477 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:49.881151  8477 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:49.806681  8274 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:49.852084  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:49.885741  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 164972us
I20260709 12:25:49.854585  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:49.856662  8526 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:49.860809  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:49.887989  6609 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:49.805842  8533 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:49.863511  8511 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:25:49.863962  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 57709.5us
I20260709 12:25:49.866228  8524 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:49.806838  8383 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:49.877369  5940 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:49.801584  8014 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:49.742066  8381 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:49.851910  8340 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:49.806342  8342 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:49.886943  6775 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:49.919703  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:49.920253  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:49.920547  6019 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:25:49.920866  6017 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:49.888991  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:49.862732  8514 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:49.884171  8419 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.923847  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:49.923388  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:49.924997  5867 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:25:49.887928  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:49.781744  8147 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:49.867595  8510 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:49.897979  6773 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:49.807106  8389 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.086s
I20260709 12:25:49.786594  8412 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:49.904527  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:49.905537  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:49.908072  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:49.904527  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:49.888936  8486 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:49.913223  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:49.923405  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:49.949237  6021 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:49.926014  6757 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:25:49.879094  8114 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 12:25:49.930186  6018 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:25:49.929560  6017 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:25:49.901234  7898 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:49.894102  8511 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:49.935196  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:49.958837  6756 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
W20260709 12:25:49.913259  8343 consensus_meta.cc:287] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.857s	user 0.011s	sys 0.004s
I20260709 12:25:49.960915  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:49.961534  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:49.962381  6762 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:49.962570  8511 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:49.963501  6757 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:49.902813  8526 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:49.899132  8383 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.939702  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:49.941146  6775 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
W20260709 12:25:49.927186  8435 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.810s	user 0.009s	sys 0.006s
I20260709 12:25:49.942173  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:49.910187  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:49.946471  6022 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:49.950258  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:49.954149  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:49.959506  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:49.960837  8518 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:49.960366  8477 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:49.967001  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.005015  5868 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:49.946861  8426 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:49.887868  8384 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:50.006177  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:49.899808  8330 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:50.019068  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.948273  8510 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:49.953289  8342 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:50.030660  5867 raft_consensus.cc:2410] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:25:49.927968  8539 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:50.027187  6785 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:50.005036  5866 raft_consensus.cc:2368] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7.
I20260709 12:25:50.026561  6756 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:50.035574  6020 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:49.944777  8274 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:50.040207  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:49.970714  8383 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:49.965246  8343 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:49.971100  8486 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:50.010864  8426 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:49.941196  8419 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:50.056636  8529 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 12:25:50.059509  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:49.968365  8514 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.035573  8539 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:50.067446  5866 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 7: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:50.069602  5945 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:50.039129  8357 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:25:50.035764  7750 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
W20260709 12:25:50.083670  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.081691  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:50.085238  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:50.086103  6768 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:50.086933  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:25:50.087837  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.087116  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 219170us
I20260709 12:25:50.095160  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:50.098619  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:50.100922  6775 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:50.104990  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:50.107308  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:50.081147  8530 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.186s	user 0.001s	sys 0.001s
I20260709 12:25:50.130165  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:50.073567  8014 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:50.129240  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:50.133812  8114 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:50.069697  8539 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:50.140223  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.068181  8324 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:50.116731  7750 consensus_queue.cc:1243] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:25:50.157090  6768 raft_consensus.cc:2410] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:50.084906  8514 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:50.142983  8014 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:25:50.195765  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.196619  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.158706  8346 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.590s	user 0.007s	sys 0.007s
W20260709 12:25:50.188917  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 301004us
I20260709 12:25:50.148468  8544 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.164634  8381 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:50.193338  8014 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:50.202148  8535 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.271s	user 0.000s	sys 0.003s
I20260709 12:25:50.152196  8114 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:50.238164  8541 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:25:50.157426  8274 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:50.148136  8342 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:50.242550  8537 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.201s	user 0.001s	sys 0.001s
W20260709 12:25:50.216692  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.289693  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.290184  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.150010  8510 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:50.228950  8514 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:50.390604  6785 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
W20260709 12:25:50.324823  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 284648us
W20260709 12:25:50.310807  6609 connection.cc:256] RPC call timeout handler was delayed by -2.1671425169997747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1684s for a response
W20260709 12:25:50.391904  6609 connection.cc:256] RPC call timeout handler was delayed by -2.1563425169997572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
W20260709 12:25:50.392411  6609 connection.cc:256] RPC call timeout handler was delayed by -2.0384425170000213s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
W20260709 12:25:50.392760  6609 connection.cc:256] RPC call timeout handler was delayed by -2.0249425169999995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18420000000000003s for a response
W20260709 12:25:50.393114  6609 connection.cc:256] RPC call timeout handler was delayed by -1.9826425169999311s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1889s for a response
W20260709 12:25:50.393455  6609 connection.cc:256] RPC call timeout handler was delayed by -1.9574425169998904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
W20260709 12:25:50.393846  6609 connection.cc:256] RPC call timeout handler was delayed by -1.7387425169999915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21600000000000003s for a response
W20260709 12:25:50.394234  6609 connection.cc:256] RPC call timeout handler was delayed by -1.7360425169999871s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2163s for a response
I20260709 12:25:50.209076  8412 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:50.290485  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.394908  6609 connection.cc:256] RPC call timeout handler was delayed by -1.7243425169999682s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21760000000000002s for a response
W20260709 12:25:50.434899  6609 connection.cc:256] RPC call timeout handler was delayed by -1.1366425169999275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28290000000000004s for a response
W20260709 12:25:50.285549  8500 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:25:50.392771  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:50.312412  8545 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
W20260709 12:25:50.458536  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 65826.1us
I20260709 12:25:50.296487  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:50.459630  5874 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:50.353609  8546 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 12:25:50.462158  8555 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 12:25:50.462386  8551 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.210541  8544 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:50.345021  8381 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:50.357818  8343 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:50.358821  8324 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:50.463364  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:50.510646  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 47373.2us
I20260709 12:25:50.511817  6609 leader_election.cc:400] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9.
I20260709 12:25:50.512115  6609 leader_election.cc:403] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:50.513047  6609 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:50.207183  8346 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:50.515445  8346 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:50.516496  8558 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:50.517318  8558 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:50.518911  8432 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.935s	user 0.009s	sys 0.005s
I20260709 12:25:50.519152  8558 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9.
I20260709 12:25:50.519419  8432 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:50.520071  8558 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:50.520793  5869 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:50.521916  8357 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:50.523255  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:50.524385  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:50.526098  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:50.526970  6023 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:50.467783  8532 raft_consensus.cc:3055] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:25:50.469175  8364 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.800s	user 0.011s	sys 0.000s
I20260709 12:25:50.206588  8553 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:50.322137  8117 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:50.502146  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.396224  8419 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:50.517943  5871 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:50.531546  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:50.535537  6020 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:50.555716  6016 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 12:25:50.556217  6016 pending_rounds.cc:85] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 2
I20260709 12:25:50.556463  6016 pending_rounds.cc:107] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:25:50.556680  6016 raft_consensus.cc:2889] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:50.541501  6609 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:50.542580  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:50.543982  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.500857  8404 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:50.563651  5947 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:50.550089  8358 consensus_meta.cc:287] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.672s	user 0.009s	sys 0.005s
I20260709 12:25:50.550520  8117 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:50.551093  8114 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:50.508750  8491 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:50.540351  5874 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:50.555785  6018 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:50.560102  5943 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 8. Ops: []
I20260709 12:25:50.590916  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:50.592305  5940 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:50.565832  8553 raft_consensus.cc:697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:50.505962  8342 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.562242  6024 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:25:50.568231  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:50.583545  6609 leader_election.cc:400] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9.
I20260709 12:25:50.534943  8532 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:50.539422  8562 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 12:25:50.507751  8551 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:50.535306  8381 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:50.574183  8377 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:50.617728  6609 leader_election.cc:403] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:50.604053  5943 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:50.617733  6024 pending_rounds.cc:85] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 4
I20260709 12:25:50.619150  6024 pending_rounds.cc:107] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:25:50.619483  6024 raft_consensus.cc:2889] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:50.648307  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 64963.2us
I20260709 12:25:50.649173  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:50.589396  8533 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.681s	user 0.008s	sys 0.005s
I20260709 12:25:50.561254  6023 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:25:50.655086  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 766) took 1115 ms. Trace:
I20260709 12:25:50.653761  6018 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:50.669039  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:50.667992  5947 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:50.670184  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:50.678508  5940 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:50.591691  8357 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.594383  8404 raft_consensus.cc:697] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:25:50.653386  7898 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.686s	user 0.013s	sys 0.000s
I20260709 12:25:50.595134  8432 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:50.680881  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:50.681572  5943 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 6: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:25:50.681882  6611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.008s	sys 0.053s
W20260709 12:25:50.695345  6611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.008s	sys 0.053s
I20260709 12:25:50.612025  8117 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:50.675477  6793 rpcz_store.cc:276] 0709 12:25:49.539343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.539464 (+   121us) service_pool.cc:224] Handling call
0709 12:25:50.654832 (+1115368us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b5400006254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:50.655021 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":600435072}
I20260709 12:25:50.583267  8358 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:50.685106  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:50.695403  5987 connection.cc:256] RPC call timeout handler was delayed by -1.8267724090001138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 12:25:50.701134  5987 connection.cc:256] RPC call timeout handler was delayed by -1.7975235190001513s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
I20260709 12:25:50.584697  8419 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.073s
W20260709 12:25:50.696030  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 555135us
I20260709 12:25:50.698475  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:50.705108  5946 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:50.705359  5944 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 9: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:50.669603  8533 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 9. Current term is 10.
I20260709 12:25:50.674582  8558 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9.
W20260709 12:25:50.698763  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.702996  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:50.674960  8559 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:50.646850  8342 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:50.715886  6778 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:50.706208  8383 consensus_meta.cc:287] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.648s	user 0.011s	sys 0.003s
I20260709 12:25:50.717864  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:50.718534  5943 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
W20260709 12:25:50.709851  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 166270us
I20260709 12:25:50.691804  8357 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:50.689493  8544 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:50.715476  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:50.726824  5946 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:50.693966  8346 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:50.712641  8548 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.211s	user 0.004s	sys 0.000s
I20260709 12:25:50.706595  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:50.732352  5940 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:50.731200  5874 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:50.720535  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:50.712723  8117 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.732234  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:50.713738  8508 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.735169  5943 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:25:50.727875  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:50.741104  5913 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:50.756510  5944 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:50.740787  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:50.759728  5874 raft_consensus.cc:2410] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:25:50.760840  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:50.765977  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:50.769919  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 68152.7us
W20260709 12:25:50.773216  5987 connection.cc:256] RPC call timeout handler was delayed by -1.7196016250004504s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
W20260709 12:25:50.773725  5987 connection.cc:256] RPC call timeout handler was delayed by -1.6925878160000138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2831s for a response
I20260709 12:25:50.774855  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:50.755676  8289 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:50.721675  8383 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:50.777966  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:50.779619  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:50.765264  8562 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:50.778293  5987 leader_election.cc:336] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.848s (SENT)
I20260709 12:25:50.783260  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:50.783516  5948 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:25:50.784564  5987 connection.cc:256] RPC call timeout handler was delayed by -1.6841235190004227s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2042s for a response
W20260709 12:25:50.789906  8486 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.736s	user 0.010s	sys 0.003s
W20260709 12:25:50.777745  8526 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.807s	user 0.010s	sys 0.009s
I20260709 12:25:50.801057  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:50.806941  6793 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 7: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:50.796227  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:50.808166  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:50.809058  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:50.771906  8519 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:25:50.796808  6018 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:50.809430  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:50.810935  6017 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:50.812810  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:50.813560  6768 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:50.814721  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:50.814932  8566 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:50.740264  8553 consensus_queue.cc:237] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:50.815593  8566 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.816409  8566 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:50.815488  8477 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.814735  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:50.818159  8477 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:50.814961  8567 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:25:50.820341  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:50.769819  8114 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.709888  8532 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:50.821660  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:50.822016  8114 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:50.822741  8532 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:50.824384  8114 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:50.825053  8289 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:50.798826  8547 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:50.825884  8547 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:50.826470  8547 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:50.826133  8289 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:50.737170  8404 consensus_queue.cc:237] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:50.827692  6016 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:50.812031  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:50.832816  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:50.834614  6017 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
W20260709 12:25:50.805128  5987 connection.cc:256] RPC call timeout handler was delayed by -1.6835878159999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2841s for a response
I20260709 12:25:50.835968  5873 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:25:50.839291  5987 connection.cc:256] RPC call timeout handler was delayed by -1.6264548520002791s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
W20260709 12:25:50.839929  5987 connection.cc:256] RPC call timeout handler was delayed by -1.6264548520002791s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
W20260709 12:25:50.840397  5987 connection.cc:256] RPC call timeout handler was delayed by -1.6121235190003063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
I20260709 12:25:50.842873  5869 raft_consensus.cc:2410] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:50.738353  8346 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:50.847688  5987 connection.cc:256] RPC call timeout handler was delayed by -1.5158235190001506s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
I20260709 12:25:50.790143  8384 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:50.757052  8117 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:50.849370  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 853) took 1003 ms. Trace:
I20260709 12:25:50.790616  8419 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:50.849414  5872 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:25:50.859102  5872 pending_rounds.cc:85] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 3
I20260709 12:25:50.855793  6015 rpcz_store.cc:276] 0709 12:25:49.845820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.845984 (+   164us) service_pool.cc:224] Handling call
0709 12:25:49.846231 (+   247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.846688 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.846715 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.847122 (+   407us) log.cc:844] Serialized 26 byte log entry
0709 12:25:50.139712 (+292590us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:50.139758 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.139827 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.553579 (+413752us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:50.849058 (+295479us) raft_consensus.cc:1685] finished
0709 12:25:50.849079 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.849301 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":135,"raft.queue_time_us":19483,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":75,"spinlock_wait_cycles":46058752,"thread_start_us":246202,"threads_started":1,"wal-append.queue_time_us":441485}
I20260709 12:25:50.852478  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:50.859920  5838 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:50.859457  5872 pending_rounds.cc:107] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:25:50.861243  5872 raft_consensus.cc:2889] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:50.805737  8558 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:50.860608  5940 raft_consensus.cc:2368] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 7. Current term is 8.
I20260709 12:25:50.844441  8477 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:50.850095  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.859314  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:50.850034  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:50.862921  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:50.845199  8114 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:50.851518  8569 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:25:50.863310  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:50.869575  8526 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:50.874657  6017 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
W20260709 12:25:50.875226  5987 connection.cc:256] RPC call timeout handler was delayed by -1.3479724090002492s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21330000000000002s for a response
I20260709 12:25:50.875910  6015 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:50.875916  5838 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:50.878674  6775 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:50.766247  8508 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:50.881034  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.900484  6016 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:50.883411  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:50.902216  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:50.848783  8565 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:50.903051  5941 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:50.903134  8565 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.903744  8565 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:50.873289  8492 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:50.910463  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:50.875623  8419 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:50.913796  8357 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:50.914588  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.915037  8383 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:50.917309  8014 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.669s	user 0.012s	sys 0.001s
W20260709 12:25:50.882529  5987 connection.cc:256] RPC call timeout handler was delayed by -1.3479724090002492s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21330000000000002s for a response
I20260709 12:25:50.904386  7898 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:50.925094  8547 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:50.880916  8526 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.878599  8381 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:50.931493  5872 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:50.935369  5866 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:25:50.886154  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:50.909300  8551 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:50.882808  8510 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:25:50.884207  8486 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:25:50.929066  8358 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:50.948137  5987 leader_election.cc:400] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6.
W20260709 12:25:50.947264  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 61186us
I20260709 12:25:50.958643  5987 leader_election.cc:403] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:50.954457  8381 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:50.959940  8364 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6.
I20260709 12:25:50.960268  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:50.962544  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:50.963899  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:50.966480  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:50.967555  8364 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:50.968911  8581 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:50.969772  8364 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:50.970254  8364 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:50.970701  8364 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:50.945791  7898 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:50.972222  8358 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:50.974370  7898 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:50.945190  8556 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.982326  8556 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:50.982506  8358 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:50.984750  8419 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:50.935024  8014 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:50.967636  8539 consensus_meta.cc:287] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.804s	user 0.012s	sys 0.003s
I20260709 12:25:50.945938  8526 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:50.987429  8539 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:50.988615  8508 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:50.990008  6756 raft_consensus.cc:2393] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 8: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:50.991056  8547 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:50.979393  8581 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:51.007290  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:51.008203  5941 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:51.007324  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:50.992142  8539 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:51.010454  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.955225  8384 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 12:25:50.980284  7880 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:51.016621  8014 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:51.017223  8561 consensus_queue.cc:1059] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:51.014272  7880 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:51.035741  5949 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:51.047133  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:51.049364  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:51.050168  5941 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:51.051352  5870 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:51.050424  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 87950.7us
I20260709 12:25:51.058913  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:51.033504  7880 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:51.019104  8565 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:51.064786  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:51.064168  8386 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:51.074101  5870 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:51.075956  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:51.077276  8532 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:51.077852  8532 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:51.078862  8532 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:51.080101  8147 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:51.080791  8386 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:51.081292  8386 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.084367  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:51.088732  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:51.089344  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 967) took 1421 ms. Trace:
I20260709 12:25:51.090425  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1124) took 1535 ms. Trace:
I20260709 12:25:51.090252  5871 rpcz_store.cc:276] 0709 12:25:49.668034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.668258 (+   224us) service_pool.cc:224] Handling call
0709 12:25:49.668499 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.555041 (+886542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.089051 (+534010us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54007f30d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:51.089256 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952755968}
I20260709 12:25:51.090930  5945 rpcz_store.cc:276] 0709 12:25:49.555326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.555476 (+   150us) service_pool.cc:224] Handling call
0709 12:25:49.555910 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.070185 (+514275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.090197 (+1020012us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b5400467dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:51.090367 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1180842368}
I20260709 12:25:51.091868  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:51.092453  5871 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.096891  8346 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:51.104136  8556 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:51.114756  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 798) took 1253 ms. Trace:
I20260709 12:25:51.115068  8386 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:51.115260  5866 rpcz_store.cc:276] 0709 12:25:49.861858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.081346 (+219488us) service_pool.cc:224] Handling call
0709 12:25:51.114551 (+1033205us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b540018a954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:51.114676 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476104576}
I20260709 12:25:51.116693  5866 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:51.117245  8386 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:51.105470  8570 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:51.108073  8426 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:51.110296  8585 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:25:51.111720  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:51.117462  8556 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:51.104704  8563 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.761s	user 0.010s	sys 0.000s
I20260709 12:25:51.121222  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:51.135303  6775 raft_consensus.cc:2410] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:25:51.066126  8512 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.008s	sys 0.008s
I20260709 12:25:51.122289  5871 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:51.128008  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:51.140285  8419 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:51.141010  8419 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.141430  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:51.141561  8419 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:51.143782  8525 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:51.132848  5872 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.133603  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:51.144552  8525 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:51.146047  8525 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:51.135944  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:51.109046  8566 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:51.147344  8588 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:51.149181  8426 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.151947  8566 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:51.146456  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 657) took 1014 ms. Trace:
I20260709 12:25:51.153214  8565 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:51.157698  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.162838  6793 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:51.166646  5915 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:51.167896  5871 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:51.132300  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 67635.1us
I20260709 12:25:51.168984  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:51.169488  5871 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:51.164187  5990 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:51.166990  6775 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:51.171312  8404 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:51.165091  8539 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:51.173415  8539 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:51.175750  8539 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:51.165445  6773 rpcz_store.cc:276] 0709 12:25:50.132012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.132177 (+   165us) service_pool.cc:224] Handling call
0709 12:25:50.132587 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:50.210714 (+ 78127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.210757 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:50.211610 (+   853us) log.cc:844] Serialized 26 byte log entry
0709 12:25:50.382069 (+170459us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:50.382113 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.382213 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.104004 (+721791us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:51.145994 (+ 41990us) raft_consensus.cc:1685] finished
0709 12:25:51.146027 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.146365 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111129,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":47,"spinlock_wait_cycles":10859776,"thread_start_us":216406,"threads_started":2,"wal-append.queue_time_us":101312}
I20260709 12:25:51.177933  8386 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:51.178689  8386 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:51.179126  8147 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:51.179708  8386 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:51.170325  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:51.184455  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:51.183058  5872 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:51.185052  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:51.188360  6775 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:51.189061  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:51.189950  8566 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:51.190470  8147 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:51.186153  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:51.192276  8515 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:51.139042  8512 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.009s	sys 0.008s
I20260709 12:25:51.187120  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:51.190501  8588 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:51.195999  8381 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:51.196403  8364 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:51.198932  8364 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:51.199889  8526 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:51.188584  5913 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:51.075440  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:51.189602  5872 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:51.203833  5872 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:25:51.207926  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.210067  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:51.210994  6773 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.212426  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:51.216665  5872 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:51.185192  5871 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 12:25:51.133517  5865 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:51.223299  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:51.226866  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:51.228993  5866 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:25:51.228428  8412 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:25:51.231758  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:51.228643  8016 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:25:51.139349  8556 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:51.234149  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:51.238849  6757 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:51.236411  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:51.239948  5871 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
W20260709 12:25:51.237162  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.240442  5871 pending_rounds.cc:85] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 3
I20260709 12:25:51.240765  5871 pending_rounds.cc:107] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:25:51.222062  8515 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:51.220610  8588 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:51.244109  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:51.247385  6785 raft_consensus.cc:1217] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Deduplicated request from leader. Original: 6.4->[7.5-7.5]   Dedup: 7.5->[]
I20260709 12:25:51.223071  8544 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.247951  6785 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:25:51.241071  5871 raft_consensus.cc:2889] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:51.246356  5987 leader_election.cc:400] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7.
I20260709 12:25:51.223453  8147 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:51.248741  5987 leader_election.cc:403] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:51.208500  8404 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:25:51.227620  8364 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:51.221022  8570 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:25:51.254551  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:51.252745  6773 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:51.268783  6785 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:51.234225  8389 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:51.259657  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:51.240176  8342 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:51.257707  8514 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:51.269191  5871 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:51.243883  8384 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:51.270885  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:51.250087  8147 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:51.228670  8014 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:25:51.244143  8383 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:51.254751  8515 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:51.237603  7750 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:25:51.281607  6793 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:25:51.267789  8364 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:51.279350  8532 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.275828  8588 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7.
I20260709 12:25:51.255038  8544 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:51.285082  8147 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:51.305910  5871 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:51.298514  8364 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:51.283864  8384 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.282078  8342 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:25:51.282234  8381 consensus_queue.cc:1059] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
W20260709 12:25:51.327901  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 88042.1us
I20260709 12:25:51.329382  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:51.331027  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:51.332631  8419 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:51.328047  8384 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.335350  8419 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:51.335430  8511 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:51.306610  8532 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:51.304957  8526 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:51.339169  5947 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:25:51.357968  5947 pending_rounds.cc:85] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 4
I20260709 12:25:51.358300  5947 pending_rounds.cc:107] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:25:51.358547  5947 raft_consensus.cc:2889] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:51.359112  5987 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:51.337383  8560 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.837s	user 0.005s	sys 0.000s
W20260709 12:25:51.340309  8557 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.831s	user 0.000s	sys 0.003s
W20260709 12:25:51.338829  8592 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:25:51.363874  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 88883us
I20260709 12:25:51.369678  8602 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:51.438591  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 70762.8us
I20260709 12:25:51.440023  8603 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:25:51.441345  5867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 803) took 1577 ms (client timeout 1770 ms). Trace:
W20260709 12:25:51.445144  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:51.452620  5867 rpcz_store.cc:269] 0709 12:25:49.863609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.938910 (+ 75301us) service_pool.cc:224] Handling call
0709 12:25:51.441076 (+1502166us) spinlock_profiling.cc:243] Waited 61 ms on lock 0x7b5400193054. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:51.441273 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140194176}
I20260709 12:25:51.456856  5874 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:51.441095  8419 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.469141  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 800) took 1606 ms. Trace:
I20260709 12:25:51.469681  5869 rpcz_store.cc:276] 0709 12:25:49.862566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.524142 (+661576us) service_pool.cc:224] Handling call
0709 12:25:51.468879 (+944737us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b5400118ad4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:51.469062 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":730572672}
I20260709 12:25:51.455612  8602 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.443621  8558 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.021s
W20260709 12:25:51.472558  8601 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:25:51.363850  8515 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:51.458144  8603 raft_consensus.cc:697] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:51.461874  8386 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:51.485361  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 126321us
I20260709 12:25:51.483480  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 765) took 1449 ms. Trace:
I20260709 12:25:51.490602  6022 rpcz_store.cc:276] 0709 12:25:50.034452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.034648 (+   196us) service_pool.cc:224] Handling call
0709 12:25:50.035075 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:50.035483 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.035512 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:50.051890 (+ 16378us) log.cc:844] Serialized 26 byte log entry
0709 12:25:50.201028 (+149138us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:50.201105 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.201347 (+   242us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.339617 (+1138270us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:51.483101 (+143484us) raft_consensus.cc:1685] finished
0709 12:25:51.483137 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.483407 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6666,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":12395,"spinlock_wait_cycles":35659648,"thread_start_us":116114,"threads_started":1,"wal-append.queue_time_us":275740}
W20260709 12:25:51.487484  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:51.490041  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.453121  7880 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:51.465056  8419 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:51.483091  8602 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:51.524082  8588 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:51.482331  8556 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:51.525743  8419 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:51.502295  8386 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:25:51.501583  7880 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:51.511909  8364 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:51.517717  8603 consensus_queue.cc:237] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:51.531899  8117 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:51.491955  8558 consensus_queue.cc:1243] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:25:51.522751  8532 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:51.533926  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:51.543081  5868 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:51.544417  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:51.545084  5868 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:51.545717  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:51.525763  8588 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:51.501575  8598 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:25:51.536048  8386 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:51.525786  8602 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:51.559461  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 66599.5us
I20260709 12:25:51.534525  8384 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:51.552142  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:51.536317  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 972) took 1858 ms. Trace:
I20260709 12:25:51.560726  5873 rpcz_store.cc:276] 0709 12:25:49.677574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.678479 (+   905us) service_pool.cc:224] Handling call
0709 12:25:49.696501 (+ 18022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.839382 (+1142881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.535999 (+696617us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400027e54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:51.536236 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2619349248}
I20260709 12:25:51.561564  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:51.562191  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:51.553750  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:51.562958  8608 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:51.563673  5945 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.563861  8608 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:51.564493  8608 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:51.566387  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.563030  5873 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:25:51.568310  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:51.571635  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:51.572672  6022 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:51.574967  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:51.576893  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:51.576833  5942 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:51.577172  6015 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:51.579596  5945 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:51.582190  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:51.580441  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:51.581192  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:51.581308  6022 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:51.580556  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:51.584082  5868 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:51.554780  8386 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:51.583323  6793 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:25:51.556156  8364 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:51.586342  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:51.588728  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:51.586709  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:51.589311  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:51.582037  8117 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:51.589907  6022 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:51.590656  8117 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:51.590953  6016 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:51.591581  6024 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:51.592218  8492 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:51.592443  8386 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:51.591192  6017 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 9. Ops: []
I20260709 12:25:51.592753  8492 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:51.593919  8386 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:51.594722  8492 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:51.595855  8492 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:51.598323  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:51.599345  6775 raft_consensus.cc:2410] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:25:51.603982  8147 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:51.606114  8611 consensus_queue.cc:1059] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:51.608865  5941 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.610600  5838 leader_election.cc:400] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 7. Current term is 8.
I20260709 12:25:51.611073  5838 leader_election.cc:403] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:25:51.614414  6757 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:51.614518  5868 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
I20260709 12:25:51.614518  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:51.619233  8364 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:51.621058  5873 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:51.621498  5915 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:51.618156  5990 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:51.622503  8147 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:51.625164  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 769) took 1921 ms. Trace:
I20260709 12:25:51.625599  6778 rpcz_store.cc:276] 0709 12:25:49.703595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.703756 (+   161us) service_pool.cc:224] Handling call
0709 12:25:49.704022 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.716784 (+1012762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.624871 (+908087us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400007154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:51.625092 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2326452608}
I20260709 12:25:51.626490  8377 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:51.630461  8377 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:51.631021  8544 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:51.631428  8377 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:51.632432  6773 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:51.632840  8377 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:51.626801  8516 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:51.627729  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:51.580294  8609 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:51.634943  6785 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.635179  8609 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:51.628012  8551 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:51.636145  8551 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:51.637051  8551 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:51.638162  8551 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:51.634395  8568 raft_consensus.cc:3055] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:51.628095  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:51.555758  8588 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:51.661366  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 52272.4us
W20260709 12:25:51.661805  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 43757.6us
I20260709 12:25:51.662474  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:51.664258  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:51.642403  6775 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:25:51.646665  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 59563.4us
I20260709 12:25:51.667557  6775 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:51.668068  5987 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:51.591205  8117 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.670729  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:51.674199  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:51.638197  8554 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.639101  8558 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:51.679224  6775 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:51.646509  8342 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:51.712373  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:51.647132  8568 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:51.684137  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 650) took 1599 ms. Trace:
I20260709 12:25:51.714942  6756 rpcz_store.cc:276] 0709 12:25:50.086099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.086298 (+   199us) service_pool.cc:224] Handling call
0709 12:25:51.683883 (+1597585us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b54002c1ad4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:51.684067 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1974725888}
I20260709 12:25:51.684727  6768 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:51.686592  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.688549  6785 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:25:51.689687  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:51.689699  5865 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 968) took 2014 ms (client timeout 2096 ms). Trace:
I20260709 12:25:51.692024  6775 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.693120  5941 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:51.668226  7880 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:51.681591  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:51.719683  6793 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:51.722096  6020 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:51.717895  5865 rpcz_store.cc:269] 0709 12:25:49.675362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.675505 (+   143us) service_pool.cc:224] Handling call
0709 12:25:51.689501 (+2013996us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400186d54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:51.689613 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3052104576}
I20260709 12:25:51.669862  8609 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:51.730868  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:51.731936  6024 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.678184  8434 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:51.736158  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 54638.2us
I20260709 12:25:51.681125  8610 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:51.688071  8014 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:51.669503  8516 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 12:25:51.688791  8532 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:51.712515  8565 raft_consensus.cc:3055] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:51.691378  8558 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:25:51.742942  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 80151.5us
I20260709 12:25:51.751904  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:51.714985  8383 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.752413  6775 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:25:51.764875  6775 pending_rounds.cc:85] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 4
I20260709 12:25:51.765326  6775 pending_rounds.cc:107] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:25:51.765817  6775 raft_consensus.cc:2889] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:51.734730  8603 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:51.759487  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:51.773919  6018 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:51.760579  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:51.691812  8554 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:51.750168  8014 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.777237  8014 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:51.753022  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:51.782505  6778 raft_consensus.cc:2368] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8.
W20260709 12:25:51.748771  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 74625.6us
I20260709 12:25:51.768083  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:51.713377  8602 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:25:51.783749  6021 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.768308  5946 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 5.
I20260709 12:25:51.748955  8610 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.789683  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:51.791451  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:51.789341  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:51.792227  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:51.782728  8603 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:51.794847  8562 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:51.796713  8532 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:51.797255  8562 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:51.798521  8570 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:51.799134  8570 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:51.800351  8570 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:51.803299  8384 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:51.803892  8389 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:51.805900  8412 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:51.807906  8554 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:51.764037  8517 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:51.809270  8517 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:51.810354  8517 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:51.811902  8562 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:51.795589  8603 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:51.813395  8562 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:51.766557  8558 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:51.773386  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:51.821826  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:51.822767  6766 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:25:51.797360  8532 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:51.754143  8565 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:51.826141  6019 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:51.825687  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:51.835453  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:51.840385  5915 leader_election.cc:400] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8.
I20260709 12:25:51.803983  8610 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:51.843341  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:51.800736  8608 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:51.843312  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:51.852586  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.854547  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:51.854748  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:51.840957  5915 leader_election.cc:403] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:25:51.855754  6766 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:25:51.844061  6021 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:25:51.846740  6015 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:51.855737  5947 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:51.836833  8386 raft_consensus.cc:3055] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:51.774971  8383 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:51.865816  6766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 767) took 2317 ms (client timeout 1554 ms). Trace:
I20260709 12:25:51.838558  8384 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:51.789283  8511 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
W20260709 12:25:51.868067  6766 rpcz_store.cc:269] 0709 12:25:49.539637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.540508 (+   871us) service_pool.cc:224] Handling call
0709 12:25:51.822366 (+2281858us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b58005d1ad0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bdf0aa 00007f39d2c61447 00007f39d2c79fe7 00007f39d2c7b7de 00007f39d2c8727c 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c
0709 12:25:51.857123 (+ 34757us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2508635648}
I20260709 12:25:51.867910  6021 raft_consensus.cc:1217] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 6.4->[7.5-7.5]   Dedup: 7.5->[]
I20260709 12:25:51.873960  6021 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:25:51.864079  8147 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8.
I20260709 12:25:51.856963  8539 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.878638  8364 raft_consensus.cc:3055] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:51.879293  8364 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:51.879583  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 383) took 1196 ms. Trace:
I20260709 12:25:51.843789  8602 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:51.880369  8364 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:51.882027  8381 raft_consensus.cc:3055] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:51.882597  8381 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:51.882846  6018 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:51.883201  8289 raft_consensus.cc:3055] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:51.883544  8381 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:51.884092  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:51.884796  6018 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.886135  8619 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:51.859054  8516 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:51.886768  8619 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:51.880097  5874 rpcz_store.cc:276] 0709 12:25:50.683055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.762106 (+ 79051us) service_pool.cc:224] Handling call
0709 12:25:50.762496 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.457488 (+694992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.879313 (+421825us) spinlock_profiling.cc:243] Waited 694 ms on lock 0x7b5400119754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:51.879512 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1596240640}
I20260709 12:25:51.893942  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:51.900009  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:51.906953  6018 raft_consensus.cc:2410] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 12:25:51.907327  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 142457us
W20260709 12:25:51.928396  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.879441  8383 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:51.959254  5873 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:51.948479  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:51.855486  8554 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:25:51.864626  8386 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:51.867724  8609 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:51.852764  8434 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:25:51.962291  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:51.963172  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.964184  6018 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:51.986788  5868 raft_consensus.cc:2376] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5. Re-sending same reply.
I20260709 12:25:51.883563  8289 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:51.895795  8516 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.878829  8539 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:51.909876  8114 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:51.977775  5946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 377) took 1246 ms (client timeout 1579 ms). Trace:
W20260709 12:25:52.011567  5946 rpcz_store.cc:269] 0709 12:25:50.731653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.731853 (+   200us) service_pool.cc:224] Handling call
0709 12:25:51.977446 (+1245593us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540081b854. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:51.977673 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2370323840}
W20260709 12:25:52.013103  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.978931  6016 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:51.980178  6015 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:51.783524  8568 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:52.001220  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:52.022051  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.007005  8516 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:52.036229  6015 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:25:52.046424  6018 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:25:51.954236  8612 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:25:51.908864  8565 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:51.974944  8511 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.057s
W20260709 12:25:52.133682  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 98910.1us
I20260709 12:25:52.033721  8533 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 7. Current term is 8.
I20260709 12:25:52.136641  5913 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:52.146497  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:52.139290  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.015805  8426 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.111s
I20260709 12:25:51.963419  8532 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:52.152387  8016 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:52.019204  8114 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:52.158953  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:52.168228  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 557665us
I20260709 12:25:52.228502  6024 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:52.229605  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:52.127266  8568 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:52.182178  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 315733us
I20260709 12:25:52.117187  7750 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
W20260709 12:25:52.238672  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.239123  5872 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:25:52.229638  5838 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:52.130955  8619 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:52.299655  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:52.236052  8551 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.597s	user 0.012s	sys 0.008s
W20260709 12:25:52.326538  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 53862us
I20260709 12:25:52.334978  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:52.335335  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.117362  8386 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:52.235880  7880 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:52.350507  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:52.247905  8016 raft_consensus.cc:697] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:52.331120  8340 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:52.371335  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:52.166780  8289 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:52.378708  5874 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:52.345961  8510 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
W20260709 12:25:52.407414  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.392056  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1002) took 1066 ms. Trace:
I20260709 12:25:52.413424  5869 rpcz_store.cc:276] 0709 12:25:51.325984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.470565 (+144581us) service_pool.cc:224] Handling call
0709 12:25:51.470945 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.954205 (+483260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.954257 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.954860 (+   603us) log.cc:844] Serialized 26 byte log entry
0709 12:25:52.014068 (+ 59208us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:52.014116 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.014190 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:52.391804 (+377614us) raft_consensus.cc:1685] finished
0709 12:25:52.391827 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.391972 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":592285,"raft.queue_time_us":528619,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":8480,"thread_start_us":58819,"threads_started":1,"wal-append.queue_time_us":435074}
I20260709 12:25:52.392191  5873 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:52.391311  5868 raft_consensus.cc:1217] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Deduplicated request from leader. Original: 2.2->[4.3-6.4]   Dedup: 4.3->[6.4-6.4]
I20260709 12:25:52.419425  5868 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:25:52.368624  8608 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:52.377125  8377 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.738s	user 0.005s	sys 0.009s
W20260709 12:25:52.429519  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:52.405553  8117 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.723s	user 0.007s	sys 0.003s
I20260709 12:25:52.444440  6785 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 12:25:52.444677  5987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.022s	sys 0.067s
W20260709 12:25:52.446219  5987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.022s	sys 0.067s
W20260709 12:25:52.449010  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 594453us
I20260709 12:25:52.383370  8386 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:52.393617  8623 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 12:25:52.375214  8340 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.385156  8629 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:52.382180  8554 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:52.341809  8556 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
W20260709 12:25:52.423066  8603 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.608s	user 0.010s	sys 0.002s
W20260709 12:25:52.458396  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.454123  8274 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:52.453547  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:52.411005  8588 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:52.401216  8364 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:52.455127  8630 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.482s	user 0.000s	sys 0.002s
I20260709 12:25:52.422096  8544 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.420256  8486 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:52.481717  8340 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.487656  8554 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.458146  8117 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:52.509361  8274 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:52.546942  6778 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:52.529430  8554 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:52.514503  8603 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:52.531989  8377 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.524511  8544 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:52.527448  8486 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:52.518632  8016 consensus_queue.cc:237] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:52.737917  6018 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
W20260709 12:25:52.746655  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 321861us
I20260709 12:25:52.632977  8274 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:52.750164  6762 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.008s	sys 0.006s
I20260709 12:25:52.750721  8114 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:52.751899  8274 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:52.722927  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1000) took 1407 ms. Trace:
W20260709 12:25:52.735827  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.754096  5870 rpcz_store.cc:276] 0709 12:25:51.315240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.460526 (+145286us) service_pool.cc:224] Handling call
0709 12:25:51.461068 (+   542us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.953408 (+492340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.953461 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.962786 (+  9325us) log.cc:844] Serialized 26 byte log entry
0709 12:25:52.420009 (+457223us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:52.420077 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.420202 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:52.721907 (+301705us) raft_consensus.cc:1685] finished
0709 12:25:52.721968 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.722495 (+   527us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":621029,"raft.queue_time_us":496336,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":11803,"thread_start_us":400274,"threads_started":1,"wal-append.queue_time_us":573915}
W20260709 12:25:52.750861  6762 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.008s	sys 0.006s
I20260709 12:25:52.761219  7898 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.774420  7898 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:52.764585  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:52.537663  8342 consensus_queue.cc:1059] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:25:52.746876  8117 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:52.584164  8377 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:52.789923  8377 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:52.791296  8513 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.013s	sys 0.012s
W20260709 12:25:52.792083  8513 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.013s	sys 0.012s
I20260709 12:25:52.790279  8381 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.749140  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:52.799693  8419 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:52.800621  8419 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:25:52.802995  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 54334.5us
I20260709 12:25:52.805071  8514 raft_consensus.cc:3055] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:52.805979  8514 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:52.808255  8514 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:52.810164  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.810716  8419 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:52.816825  8514 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:52.820042  8486 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:52.808909  8510 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:52.821223  8510 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:52.820725  7898 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:52.822854  8510 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:52.830610  8510 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:52.827730  8516 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.700s	user 0.011s	sys 0.015s
I20260709 12:25:52.831760  8516 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:52.837896  8516 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:52.821417  6611 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:52.842473  7898 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:52.843221  8516 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:52.845026  8516 consensus_queue.cc:1059] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:52.847033  8516 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:52.849103  8289 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:52.583369  8558 consensus_meta.cc:287] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.765s	user 0.015s	sys 0.006s
I20260709 12:25:52.851327  8663 raft_consensus.cc:3055] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:25:52.647696  8636 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 12:25:52.644810  8646 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
W20260709 12:25:52.663983  8610 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.802s	user 0.011s	sys 0.003s
I20260709 12:25:52.646903  8629 raft_consensus.cc:697] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:25:52.675606  8640 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.196s	user 0.004s	sys 0.000s
W20260709 12:25:52.663475  8642 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:25:52.873735  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:25:53.059512  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 186039us
I20260709 12:25:53.059696  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 665) took 1202 ms. Trace:
I20260709 12:25:53.060608  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:53.061156  6608 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.002s	sys 0.012s
I20260709 12:25:52.882046  8625 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:53.062680  8632 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:53.063221  8632 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:53.063621  8632 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:52.883631  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.065495  8676 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:52.889819  8517 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.023s	user 0.016s	sys 0.036s
I20260709 12:25:53.066349  8676 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:53.067039  8517 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.228s	user 0.017s	sys 0.036s
W20260709 12:25:52.892056  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:53.068275  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 578434us
I20260709 12:25:52.905138  8663 raft_consensus.cc:740] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:52.893355  8377 consensus_queue.cc:1059] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 12:25:53.039487  8664 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.055902  5868 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:25:53.056418  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 502946us
I20260709 12:25:53.056860  8659 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
W20260709 12:25:53.061672  6608 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.002s	sys 0.012s
W20260709 12:25:53.082289  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.083494  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 660) took 1356 ms. Trace:
I20260709 12:25:53.081902  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 778) took 1205 ms. Trace:
I20260709 12:25:53.132450  8676 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:53.082846  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1001) took 1762 ms. Trace:
W20260709 12:25:53.078874  8654 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:25:53.077980  8643 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:25:53.069571  8434 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.083417  8517 raft_consensus.cc:3055] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:53.079273  8556 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:53.156608  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 90389.1us
I20260709 12:25:53.088646  8539 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:53.060567  6762 rpcz_store.cc:276] 0709 12:25:51.857615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.857852 (+   237us) service_pool.cc:224] Handling call
0709 12:25:51.870922 (+ 13070us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.871144 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:52.758705 (+887561us) log.cc:844] Serialized 14 byte log entry
0709 12:25:53.043730 (+285025us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:53.052208 (+  8478us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:53.055526 (+  3318us) spinlock_profiling.cc:243] Waited 4.98 ms on lock 0x7b18000fe140. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:25:53.059027 (+  3501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.059233 (+   206us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.059576 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":85681,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":33983,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":216,"spinlock_wait_cycles":11455232,"thread_start_us":1050831,"threads_started":2}
I20260709 12:25:53.084252  8603 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:53.070362  8612 consensus_queue.cc:1243] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:25:53.183331  8543 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:53.162997  8510 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:53.074393  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:53.134449  6793 rpcz_store.cc:276] 0709 12:25:51.726968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.743145 (+ 16177us) service_pool.cc:224] Handling call
0709 12:25:51.743568 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.897217 (+153649us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:51.897269 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.897989 (+   720us) log.cc:844] Serialized 26 byte log entry
0709 12:25:52.124614 (+226625us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:52.124653 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.124722 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.082952 (+958230us) raft_consensus.cc:1685] finished
0709 12:25:53.083008 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.083283 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21118,"raft.queue_time_us":27104,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":14876,"thread_start_us":263548,"threads_started":2,"wal-append.queue_time_us":503466}
I20260709 12:25:53.189599  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:53.189791  8434 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:53.191921  8637 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.192232  8602 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:53.193156  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:53.192966  8637 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:53.192482  8565 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.196063  8637 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:53.196976  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.197830  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:53.199654  8556 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:53.202090  8610 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:53.202889  8556 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:53.143934  5867 rpcz_store.cc:276] 0709 12:25:51.319084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.457368 (+138284us) service_pool.cc:224] Handling call
0709 12:25:51.457833 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.955592 (+497759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.955646 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.962697 (+  7051us) log.cc:844] Serialized 26 byte log entry
0709 12:25:52.228795 (+266098us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:52.228922 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.229074 (+   152us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:52.674694 (+445620us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:53.061258 (+386564us) raft_consensus.cc:1685] finished
0709 12:25:53.061329 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.081407 (+ 20078us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":596542,"raft.queue_time_us":527885,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":9888,"spinlock_wait_cycles":6235008,"thread_start_us":208412,"threads_started":1,"wal-append.queue_time_us":479468}
W20260709 12:25:53.184088  5944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.014s	sys 0.018s
W20260709 12:25:53.204672  5944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.014s	sys 0.018s
I20260709 12:25:53.169270  8517 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:53.206278  6020 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 9: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:53.206777  8517 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:53.208076  8517 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:53.210296  6022 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:25:53.210922  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 233647us
W20260709 12:25:53.211851  8364 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.665s	user 0.013s	sys 0.009s
I20260709 12:25:53.212554  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 789) took 1053 ms. Trace:
I20260709 12:25:53.134819  6023 rpcz_store.cc:276] 0709 12:25:51.876343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.885741 (+  9398us) service_pool.cc:224] Handling call
0709 12:25:51.886081 (+   340us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:52.128611 (+242530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:52.128750 (+   139us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:52.136921 (+  8171us) log.cc:844] Serialized 52 byte log entry
0709 12:25:52.422896 (+285975us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:52.422935 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.423012 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.081200 (+658188us) raft_consensus.cc:1685] finished
0709 12:25:53.081245 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.081556 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":889797,"raft.queue_time_us":9086,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":50754,"thread_start_us":519457,"threads_started":2,"wal-append.queue_time_us":389450}
I20260709 12:25:53.216996  8324 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:52.879256  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:53.221951  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.222585  8615 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:53.223788  8641 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.007s	sys 0.027s
I20260709 12:25:53.073261  8663 consensus_queue.cc:260] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:53.227746  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 790) took 1068 ms. Trace:
W20260709 12:25:53.228936  8554 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.003s
I20260709 12:25:53.229198  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:53.231665  8638 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:25:53.233350  8624 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:53.233961  8117 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.234516  8543 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:53.183581  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:53.212965  5946 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:53.282723  5865 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:25:53.283766  8543 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:53.285408  8544 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:53.209489  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 507) took 1375 ms. Trace:
I20260709 12:25:53.240682  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:25:53.296851  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 58634.9us
I20260709 12:25:53.237013  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:53.298571  5868 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:53.299960  8340 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.714s	user 0.022s	sys 0.000s
I20260709 12:25:53.300669  8340 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:53.205276  8610 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:53.238525  6016 rpcz_store.cc:276] 0709 12:25:52.158868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.159843 (+   975us) service_pool.cc:224] Handling call
0709 12:25:52.160299 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:52.891382 (+731083us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:52.891458 (+    76us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:52.892395 (+   937us) log.cc:844] Serialized 26 byte log entry
0709 12:25:52.998012 (+105617us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:52.998061 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.010583 (+ 12522us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.212115 (+201532us) raft_consensus.cc:1685] finished
0709 12:25:53.212138 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.212333 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":730071,"raft.queue_time_us":901202,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":275,"thread_start_us":105029,"threads_started":1,"wal-append.queue_time_us":149998}
I20260709 12:25:53.192708  8434 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:53.250154  6022 rpcz_store.cc:276] 0709 12:25:52.159282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.159431 (+   149us) service_pool.cc:224] Handling call
0709 12:25:53.209345 (+1049914us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.210988 (+  1643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.227458 (+ 16470us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54003642d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:53.227663 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2412827904}
W20260709 12:25:53.257169  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:53.304932  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:53.258332  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.252526  8404 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:53.266089  6021 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:53.192836  8602 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:53.267181  5942 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:25:53.268663  8612 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.269788  8609 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.082741  8629 consensus_queue.cc:237] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:53.278230  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:53.270812  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:53.281261  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:53.295205  8603 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:53.298039  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:53.296576  6020 rpcz_store.cc:276] 0709 12:25:51.834635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.835011 (+   376us) service_pool.cc:224] Handling call
0709 12:25:53.208838 (+1373827us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54003642d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:53.209369 (+   531us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3149061504}
W20260709 12:25:53.279176  8641 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.007s	sys 0.028s
I20260709 12:25:53.322938  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:25:53.323307  8014 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.545s	user 0.009s	sys 0.016s
I20260709 12:25:53.324091  8014 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:53.324424  8404 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:53.325018  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 664) took 1504 ms. Trace:
I20260709 12:25:53.302135  8638 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:53.326184  8624 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:53.325588  6775 rpcz_store.cc:276] 0709 12:25:51.820590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.820733 (+   143us) service_pool.cc:224] Handling call
0709 12:25:51.821078 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.830150 (+  9072us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:51.853222 (+ 23072us) log.cc:844] Serialized 14 byte log entry
0709 12:25:52.744283 (+891061us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b18000acf60. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:25:53.190233 (+445950us) spinlock_profiling.cc:243] Waited 6.29 ms on lock 0x7b480052c158. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d7bacaa1 00007f39d7ba3b37 00007f39d7ba39ca 00007f39d7c9b26f 00007f39d7c9af37 00007f39d7caf55c 00007f39d7caf508 00007f39d7caf488 00007f39d7caf443 00007f39d7cae6ec 00007f39d6bd8a96
0709 12:25:53.214508 (+ 24275us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.215454 (+   946us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.215731 (+   277us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:53.215780 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.215897 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.320293 (+104396us) raft_consensus.cc:1685] finished
0709 12:25:53.320317 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.324805 (+  4488us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16663,"prepare.queue_time_us":23057,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":17757,"raft.run_cpu_time_us":33297,"raft.run_wall_time_us":1352796,"spinlock_wait_cycles":44219136,"thread_start_us":399605,"threads_started":1,"wal-append.queue_time_us":1364337}
I20260709 12:25:53.327143  8404 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:53.328855  8624 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:53.330487  8634 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.120s	user 0.000s	sys 0.003s
I20260709 12:25:53.333154  8014 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:53.334316  8014 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:53.335075  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:53.335624  8340 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:53.337206  8289 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:53.338094  8289 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:53.341027  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:53.353819  8340 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.354775  8666 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:53.354645  8340 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:53.355549  8666 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:53.356781  8666 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:53.357810  8386 consensus_meta.cc:287] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.891s	user 0.011s	sys 0.012s
I20260709 12:25:53.358955  6017 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:53.358748  8609 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:53.359553  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:53.341897  8404 consensus_queue.cc:1059] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:53.342892  8603 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:53.344144  5874 raft_consensus.cc:2410] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 9 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:25:53.345865  8612 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:53.360409  5947 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:53.347319  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:53.347851  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 50009.6us
I20260709 12:25:53.348004  8114 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:53.348367  8602 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:53.297367  8665 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.067s	user 0.002s	sys 0.000s
I20260709 12:25:53.352453  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:53.381017  5941 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.342006  8389 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:53.362919  5868 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
W20260709 12:25:53.366070  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.338182  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:53.371804  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 52468.1us
W20260709 12:25:53.362232  8274 consensus_meta.cc:287] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.573s	user 0.006s	sys 0.013s
I20260709 12:25:53.176156  8664 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:53.388154  8532 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:53.389441  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 781) took 1512 ms. Trace:
I20260709 12:25:53.390116  8384 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.421468  8384 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:53.391728  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:53.391777  5987 connection.cc:256] RPC call timeout handler was delayed by -2.0342031499999393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17700000000000002s for a response
W20260709 12:25:53.423344  5987 connection.cc:256] RPC call timeout handler was delayed by -1.7192031499998848s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21200000000000002s for a response
W20260709 12:25:53.423799  5987 connection.cc:256] RPC call timeout handler was delayed by -1.4933031499999743s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2371s for a response
I20260709 12:25:53.425554  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:53.425925  8274 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:53.398095  8612 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:53.317513  8554 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:53.400779  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:53.404917  8514 raft_consensus.cc:3055] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:53.360867  8340 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:53.406852  8602 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:53.407341  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 999) took 2174 ms. Trace:
I20260709 12:25:53.372294  8117 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:53.410416  8609 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:53.410630  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 508) took 1570 ms. Trace:
I20260709 12:25:53.425190  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:53.418215  6021 rpcz_store.cc:276] 0709 12:25:51.877252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.899802 (+ 22550us) service_pool.cc:224] Handling call
0709 12:25:53.389182 (+1489380us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400171bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:53.389359 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3104530816}
I20260709 12:25:53.079120  8558 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:53.439568  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.450965  8364 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:53.451689  8364 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:53.452368  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:53.453320  6778 raft_consensus.cc:2393] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:53.454106  8340 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:53.454872  8340 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:53.455415  8340 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.459743  8608 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:53.451020  8645 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:53.461095  8611 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.443929  6017 rpcz_store.cc:276] 0709 12:25:51.840677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.840830 (+   153us) service_pool.cc:224] Handling call
0709 12:25:51.856551 (+ 15721us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.359315 (+1502764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.375322 (+ 16007us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54001711d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:53.410548 (+ 35226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3453596544}
I20260709 12:25:53.461771  8645 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:53.461884  8611 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:53.453060  5840 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:53.467835  5838 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:53.475100  6762 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:53.435642  8386 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:53.435427  8514 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:25:53.480293  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 139422us
I20260709 12:25:53.441357  8532 consensus_queue.cc:1243] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:25:53.442685  8274 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:53.488548  6775 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:53.373045  8615 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:53.454139  8364 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:53.489283  8114 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:53.460624  8608 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:53.492674  5696 catalog_manager.cc:5697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 5 to 6, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 6 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:25:53.495180  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:53.441366  5871 rpcz_store.cc:276] 0709 12:25:51.233137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.438833 (+205696us) service_pool.cc:224] Handling call
0709 12:25:51.451495 (+ 12662us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:51.956094 (+504599us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:52.001959 (+ 45865us) log.cc:844] Serialized 14 byte log entry
0709 12:25:52.467765 (+465806us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:52.723351 (+255586us) log.cc:844] Serialized 52 byte log entry
0709 12:25:52.723887 (+   536us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:52.724717 (+   830us) log.cc:844] Serialized 14 byte log entry
0709 12:25:52.725225 (+   508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.725403 (+   178us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.406836 (+681433us) raft_consensus.cc:1685] finished
0709 12:25:53.406867 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.407078 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":693874,"prepare.queue_time_us":169871,"prepare.run_cpu_time_us":171,"prepare.run_wall_time_us":7512,"raft.queue_time_us":537757,"raft.run_cpu_time_us":482,"raft.run_wall_time_us":32963,"thread_start_us":342416,"threads_started":1,"wal-append.queue_time_us":573775}
I20260709 12:25:53.446636  8612 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:53.436937  8539 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 12:25:53.447968  8434 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:25:53.490495  8274 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.516357  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:53.479676  8543 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:53.448046  8624 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 12:25:53.501259  8114 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.535763  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1012) took 1128 ms. Trace:
W20260709 12:25:53.541021  5913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.022s	sys 0.031s
I20260709 12:25:53.533442  5696 catalog_manager.cc:5697] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 3 to 8, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 8 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:53.545492  5913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.023s	sys 0.032s
W20260709 12:25:53.546694  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 786247us
W20260709 12:25:53.549558  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 71626.3us
W20260709 12:25:53.552802  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 100025us
I20260709 12:25:53.555718  6773 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:53.557415  8510 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:53.555759  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:53.527819  8274 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:53.567788  8664 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:53.532292  8544 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:25:53.576370  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 60175.4us
I20260709 12:25:53.541698  8114 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:53.519541  8612 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:53.539762  5873 rpcz_store.cc:276] 0709 12:25:52.407229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.458067 (+ 50838us) service_pool.cc:224] Handling call
0709 12:25:52.721183 (+263116us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:52.722445 (+  1262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:52.722495 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:52.723874 (+  1379us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.016565 (+292691us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:53.121581 (+105016us) log.cc:844] Serialized 14 byte log entry
0709 12:25:53.149008 (+ 27427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.149117 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.296827 (+147710us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b240043b188. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfa1c9c4 00007f39cfa1aacb 00007f39cfbd5dc9 00007f39cfbd5a55 00007f39cfbd509d 00007f39d280710e 00007f39d281e88c 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221
0709 12:25:53.522472 (+225645us) raft_consensus.cc:1685] finished
0709 12:25:53.535269 (+ 12797us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.535524 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13358,"prepare.queue_time_us":117629,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":4918,"raft.queue_time_us":44858,"raft.run_cpu_time_us":583,"raft.run_wall_time_us":15742,"spinlock_wait_cycles":58293120,"thread_start_us":287692,"threads_started":1,"wal-append.queue_time_us":450313}
I20260709 12:25:53.575829  8539 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:25:53.609259  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.608258  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:53.621855  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:53.619856  6773 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
W20260709 12:25:53.622602  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 59245.6us
I20260709 12:25:53.586542  8645 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:53.626307  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:53.592172  8544 raft_consensus.cc:697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:53.574962  8543 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:53.598271  8324 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:53.635848  5868 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:53.577734  8514 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:53.641914  5865 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
W20260709 12:25:53.645430  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 89739.6us
I20260709 12:25:53.640131  6611 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:53.655113  6778 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:53.657673  5946 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:53.669512  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:53.671183  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 143736us
I20260709 12:25:53.672760  5696 catalog_manager.cc:5697] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:53.676326  5695 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40830 (request call id 148) took 23258 ms (client timeout 15000 ms). Trace:
I20260709 12:25:53.676491  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:25:53.680776  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.680711  8544 consensus_queue.cc:237] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:53.671484  8637 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:25:53.682209  6757 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:53.705854  5868 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:25:53.681247  8632 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.617s	user 0.011s	sys 0.004s
I20260709 12:25:53.691936  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:53.713166  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:53.713778  8340 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:53.717038  8615 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:53.717753  8632 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:53.724546  6778 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:25:53.724643  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 92420.2us
I20260709 12:25:53.665776  8117 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:53.728195  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:53.676875  8624 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:53.685258  8543 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:53.727851  8664 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.750733  8664 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:53.747478  8532 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:25:53.752444  8661 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:53.753100  8661 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:53.755373  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:53.755378  8661 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:53.758836  8663 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:53.726691  8632 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:53.762423  8554 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:53.677028  5695 rpcz_store.cc:269] 0709 12:25:30.431910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.434826 (+  2916us) service_pool.cc:224] Handling call
0709 12:25:53.645156 (+23210330us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:49.701081 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:49.701733 (+   652us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:49.701790 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:25:49.701812 (+    22us) tablet.cc:662] Decoding operations
0709 12:25:49.707407 (+  5595us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:49.707542 (+   135us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:49.707586 (+    44us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:25:49.709521 (+  1935us) tablet.cc:701] Row locks acquired
0709 12:25:49.709599 (+    78us) write_op.cc:260] PREPARE: finished
0709 12:25:49.742165 (+ 32566us) write_op.cc:270] Start()
0709 12:25:49.784878 (+ 42713us) write_op.cc:276] Timestamp: P: 1783599949735294 usec, L: 0
0709 12:25:49.784909 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:25:49.785821 (+   912us) log.cc:844] Serialized 6643 byte log entry
0709 12:25:50.816641 (+1030820us) op_driver.cc:464] REPLICATION: finished
0709 12:25:50.872564 (+ 55923us) write_op.cc:301] APPLY: starting
0709 12:25:50.872674 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:50.900006 (+ 27332us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:50.912540 (+ 12534us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:50.967924 (+ 55384us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:50.968254 (+   330us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 12:25:50.968302 (+    48us) write_op.cc:312] APPLY: finished
0709 12:25:50.968736 (+   434us) log.cc:844] Serialized 118 byte log entry
0709 12:25:50.968967 (+   231us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:50.969167 (+   200us) write_op.cc:454] Released schema lock
0709 12:25:50.969242 (+    75us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:50.969777 (+   535us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:50.969814 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3669015,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":55488,"apply.run_cpu_time_us":5258,"apply.run_wall_time_us":97153,"num_ops":17,"prepare.queue_time_us":284037,"prepare.run_cpu_time_us":17246,"prepare.run_wall_time_us":463452,"raft.queue_time_us":201010,"raft.run_cpu_time_us":15599,"raft.run_wall_time_us":56470,"replication_time_us":1031430,"spinlock_wait_cycles":21248,"thread_start_us":659950,"threads_started":4,"wal-append.queue_time_us":799691,"wal-append.run_cpu_time_us":14224,"wal-append.run_wall_time_us":1380642}]]}
W20260709 12:25:53.764897  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 90936.9us
I20260709 12:25:53.766103  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:53.708161  8667 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:25:53.762516  8340 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:53.768071  8667 raft_consensus.cc:3055] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:53.768175  8340 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:53.768602  8667 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:53.768858  8340 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:53.769867  8638 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:53.769743  8667 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:53.771407  8667 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:53.774799  8274 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:53.777119  8404 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:53.777715  8404 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:53.778673  8404 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:53.779628  8514 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.780004  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:53.782096  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:53.783032  6023 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
W20260709 12:25:53.785012  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.785461  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:53.785934  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:53.787451  8147 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:53.787936  8147 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.788326  8147 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:53.789258  5942 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 8: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:53.791375  8608 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:53.792554  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:53.792905  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:53.781311  8381 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:53.793994  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:53.769448  8340 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:53.735183  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 792) took 1530 ms. Trace:
I20260709 12:25:53.735386  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:53.795488  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:53.794019  8381 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:53.796391  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:53.797055  8381 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:53.797531  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:53.798108  8686 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:53.737052  8384 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:53.798355  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:53.803606  8686 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:53.795117  6024 rpcz_store.cc:276] 0709 12:25:52.159999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.230096 (+ 70097us) service_pool.cc:224] Handling call
0709 12:25:52.230418 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.250000 (+1019582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.257560 (+  7560us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.331172 (+ 73612us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.548745 (+217573us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:53.548805 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.548951 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.673585 (+124634us) raft_consensus.cc:1685] finished
0709 12:25:53.673600 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.690521 (+ 16921us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":779224,"raft.queue_time_us":837211,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":135,"thread_start_us":338745,"threads_started":2,"wal-append.queue_time_us":273891}
I20260709 12:25:53.805342  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:53.738641  8679 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.386s	user 0.004s	sys 0.001s
I20260709 12:25:53.739581  5990 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:53.739673  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:53.683562  8558 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:53.740690  8615 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:25:53.740705  8671 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.410s	user 0.003s	sys 0.000s
I20260709 12:25:53.743619  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:53.743582  5696 catalog_manager.cc:5697] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 5 to 6, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 6 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:25:53.746910  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:53.747483  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:53.748179  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:53.749397  8603 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:53.749562  6016 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:53.686949  8611 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:53.798358  5943 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 8: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:53.736222  8324 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:53.681402  8687 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:53.810679  5941 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:53.812386  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:53.813308  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:53.832975  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:53.798729  8381 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:53.815874  6017 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:25:53.833981  5946 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:53.823239  6611 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:53.780124  8404 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:53.842451  6775 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:25:53.832427  5940 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:53.834987  5947 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:53.842345  5945 raft_consensus.cc:2368] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
W20260709 12:25:53.843098  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 103589us
I20260709 12:25:53.815557  8554 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.852216  5838 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:53.854336  6023 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:53.835748  8289 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.829718  8324 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:53.842900  8615 consensus_queue.cc:1243] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:25:53.877408  6017 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:53.832973  8532 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:53.862286  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:53.886332  6021 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:53.872080  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:53.869983  6020 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:53.846462  8384 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:53.883033  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:53.883759  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:53.872196  8694 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:25:53.877907  6017 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:53.886184  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:53.907683  6016 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:53.889791  6757 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:25:53.893915  8515 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:53.898138  5696 catalog_manager.cc:5697] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 2 to 4, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 4 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
W20260709 12:25:53.900298  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.902748  8324 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:53.876013  8629 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:53.904222  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:53.909202  5871 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:53.873315  8666 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.882649  8645 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.903035  8114 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:53.909904  6757 pending_rounds.cc:85] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 3
I20260709 12:25:53.891469  8384 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.872813  8510 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:53.926679  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:53.926702  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:53.934706  5940 raft_consensus.cc:2410] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:53.939384  5946 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:53.916303  5870 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:53.942866  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:53.956164  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:53.951872  6018 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:25:53.952198  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:53.958240  5868 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:53.892184  8543 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:53.866734  8554 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:53.957221  5945 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:53.965973  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:25:53.882498  8289 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:53.943440  6793 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:53.971212  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:53.976063  8668 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:53.976259  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:53.979954  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:53.981257  6017 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:53.953527  8612 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:53.925338  8629 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.986378  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:53.994582  8695 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.110s	user 0.000s	sys 0.003s
I20260709 12:25:53.936604  8539 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.001600  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 391) took 1623 ms. Trace:
I20260709 12:25:54.003103  8686 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:53.940722  6757 pending_rounds.cc:107] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:25:53.981464  5696 catalog_manager.cc:5697] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 2 to 5, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 5 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:54.018180  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 52345.8us
I20260709 12:25:54.020185  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:54.021011  6018 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:54.014858  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:54.025813  8672 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:54.026458  6023 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:53.961787  8384 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:54.058444  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:54.059504  6015 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:54.061437  5946 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:53.962241  8324 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:54.064275  8114 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:54.006506  8645 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:54.066428  6756 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:25:54.050084  6757 raft_consensus.cc:2889] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:54.067582  8645 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:54.069043  5871 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:54.070807  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:54.027824  6773 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:53.931720  8274 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:53.919224  8517 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:54.026554  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:54.011406  6019 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
W20260709 12:25:54.032943  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.039211  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:53.916757  8532 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:53.941098  8666 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:54.063025  8638 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.063655  8539 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:54.026793  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:54.048846  6762 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:54.052156  8558 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:54.039951  8629 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:54.104095  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:54.123481  6015 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:54.081302  6609 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:54.115475  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:54.116865  6778 raft_consensus.cc:2410] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:54.108841  5696 catalog_manager.cc:5697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:25:54.036573  5944 rpcz_store.cc:276] 0709 12:25:52.377925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.378113 (+   188us) service_pool.cc:224] Handling call
0709 12:25:52.383272 (+  5159us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:52.383706 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:52.383736 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:52.384283 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.205294 (+821011us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:53.205376 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.205494 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.739886 (+534392us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:54.001145 (+261259us) raft_consensus.cc:1685] finished
0709 12:25:54.001172 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.001484 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":138,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":8863,"spinlock_wait_cycles":68682496,"thread_start_us":739043,"threads_started":1,"wal-append.queue_time_us":903546}
I20260709 12:25:54.061510  8565 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:25:54.127590  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 46437.5us
I20260709 12:25:54.094210  8611 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:54.134572  5947 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:25:54.133591  6017 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:54.138160  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:54.141669  5941 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:54.103374  8687 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.142822  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:54.038923  8543 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:54.008703  8515 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:54.073982  8637 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 12:25:54.108331  8117 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.114354  8668 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.144655  6611 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:54.127727  8664 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:54.125620  8645 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.094586  8686 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 12:25:54.139139  8516 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:54.160590  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:54.161581  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:54.168125  5696 catalog_manager.cc:5697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:54.180133  6611 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:54.114007  8638 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:54.114734  8517 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:54.171635  6756 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:54.185400  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:54.191725  6793 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:54.186481  5696 catalog_manager.cc:5697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:54.196059  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.197893  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:54.199873  6019 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:54.211169  6778 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:54.218598  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 243141us
I20260709 12:25:54.218093  5696 catalog_manager.cc:5697] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:25:54.217136  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:54.188072  8516 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:54.237895  6018 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:54.148689  8565 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:54.240573  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.242475  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:54.147545  8611 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:54.217408  8664 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.247202  6756 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:25:54.151715  8687 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:54.259848  6756 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:54.260382  8687 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:54.264194  8638 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:54.233026  8602 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.265007  8602 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:54.265736  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:54.267393  8602 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:54.199425  8645 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:54.268133  8703 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.268882  8703 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:54.171861  8668 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:54.237800  8389 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:25:54.244359  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:54.159852  8117 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:54.220494  8612 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.253481  5696 catalog_manager.cc:5697] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 1 to 4, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 4 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:25:54.184813  8340 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:54.261412  6756 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:25:54.277143  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.293859  6757 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:54.306070  5942 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:54.294126  5941 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:25:54.303462  6015 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:25:54.276237  6785 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:25:54.278879  8632 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:25:54.277964  8615 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:25:54.308897  5945 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:25:54.247908  8516 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:54.324560  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 70093.1us
W20260709 12:25:54.283017  8698 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 12:25:54.317404  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 161311us
I20260709 12:25:54.314662  6793 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:25:54.270160  8645 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:54.284016  8517 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:54.324752  5696 catalog_manager.cc:5697] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 1 to 3, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 3 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:25:54.278411  8539 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:54.273689  8554 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:54.269470  8289 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:54.284518  8114 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:54.384600  8114 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:54.384645  6015 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:54.391516  8624 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.588s	user 0.012s	sys 0.000s
I20260709 12:25:54.392191  8624 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:54.280743  8608 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 12:25:54.393941  8608 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:54.394361  8608 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:54.395463  8608 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:54.396853  8608 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:54.398138  8702 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:25:54.294799  8664 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:54.401315  8668 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:54.310479  8544 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:54.367751  6757 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:54.275594  8611 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:54.282940  8515 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:54.273883  8565 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:54.374493  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.283327  8532 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:54.309579  8632 consensus_queue.cc:1243] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:54.412794  5696 catalog_manager.cc:5697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 3 to 4, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 4 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:25:54.310338  8637 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.314564  8661 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:54.315006  8602 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:25:54.379841  5838 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:54.365785  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 228762us
I20260709 12:25:54.313477  8612 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:54.316672  8638 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:54.311444  8615 raft_consensus.cc:3055] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:54.417999  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:54.439136  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.439205  5871 raft_consensus.cc:2410] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:54.303085  8147 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:54.425485  6778 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:54.431191  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:54.432782  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 53054.5us
I20260709 12:25:54.440227  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:54.424273  8661 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.442576  5696 catalog_manager.cc:5697] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:25:54.452791  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:54.453760  5945 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:54.455865  5874 raft_consensus.cc:2368] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 5. Current term is 6.
W20260709 12:25:54.434434  8663 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.675s	user 0.009s	sys 0.004s
I20260709 12:25:54.460669  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:54.463586  5940 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:54.458588  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:54.462047  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:54.463811  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 283755us
W20260709 12:25:54.464761  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.467929  5943 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 8. Ops: []
I20260709 12:25:54.482707  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:54.484920  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:54.438133  8615 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:54.483711  5946 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:54.441835  8274 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:54.429793  8637 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:54.452407  8661 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:54.456030  8666 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:54.498736  5945 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:54.507620  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:54.494410  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:54.511106  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:54.508518  5866 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:54.501940  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:54.440757  8638 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:54.521323  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56356us
I20260709 12:25:54.521525  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:54.522603  6611 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:54.514992  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:54.531505  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.499370  8543 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:54.536381  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:54.512266  5865 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
W20260709 12:25:54.530469  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 251522us
I20260709 12:25:54.532670  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:54.541777  8517 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:54.542296  8517 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:25:54.542699  8517 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:54.542891  8340 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:54.543823  8340 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:54.544332  8340 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:54.512763  8289 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:25:54.501561  8624 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.532301  8539 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:25:54.547168  8624 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:54.549093  8667 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.777s	user 0.013s	sys 0.001s
I20260709 12:25:54.511106  8684 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:54.426883  8544 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:54.506438  8510 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.003s	sys 0.022s
I20260709 12:25:54.546049  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:54.550585  5949 raft_consensus.cc:2410] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:25:54.550390  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:54.533949  5945 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:54.559231  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:54.564944  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:54.566076  5865 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:54.550401  8645 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:25:54.561703  8381 consensus_meta.cc:287] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.711s	user 0.014s	sys 0.000s
I20260709 12:25:54.587021  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:54.556264  8510 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.003s	sys 0.023s
I20260709 12:25:54.574820  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:54.550912  8384 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:54.551769  8624 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:54.597805  5987 leader_election.cc:336] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Timed out: RequestConsensusVote RPC to 127.2.4.195:43329 timed out after 1.503s (SENT)
W20260709 12:25:54.601495  5987 leader_election.cc:336] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.503s (SENT)
I20260709 12:25:54.601979  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:54.567306  8558 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:54.545257  8340 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:54.600253  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.556279  8515 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:54.573004  8615 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:54.610924  8712 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.100s
I20260709 12:25:54.613500  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:54.583364  8539 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:25:54.614780  5866 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:54.616787  6019 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:25:54.616441  8706 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.119s
I20260709 12:25:54.584611  8611 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:54.621129  5987 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.550s (SENT) [suppressed 16 similar messages]
W20260709 12:25:54.624670  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 49936.1us
I20260709 12:25:54.650090  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:54.650854  5867 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:54.597028  8510 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:54.627528  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:54.585302  8565 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:25:54.603996  8117 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:54.654680  6017 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
W20260709 12:25:54.647353  8700 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 12:25:54.683828  6017 pending_rounds.cc:85] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 4
I20260709 12:25:54.707360  6017 pending_rounds.cc:107] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:25:54.707790  6017 raft_consensus.cc:2889] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:54.616492  8384 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:54.683192  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 266022us
W20260709 12:25:54.701551  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 261429us
I20260709 12:25:54.625336  8515 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:54.628470  8602 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.726400  5865 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:54.725396  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:54.728304  5865 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:54.728389  5869 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
W20260709 12:25:54.701886  8404 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.851s	user 0.015s	sys 0.000s
I20260709 12:25:54.724864  5915 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:54.743582  5948 raft_consensus.cc:2410] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 12:25:54.721185  8699 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
W20260709 12:25:54.729831  8603 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:25:54.744922  8147 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:54.734707  8602 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:54.821892  6766 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:54.700765  8611 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:54.826010  8638 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:54.828428  8713 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 12:25:54.832072  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:54.831823  8558 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:54.730257  8612 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:54.798635  8404 raft_consensus.cc:2764] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 12:25:54.851325  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 201325us
W20260709 12:25:54.841307  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 321936us
I20260709 12:25:54.757480  8384 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.108s
I20260709 12:25:54.853219  5869 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:54.859081  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:54.859834  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 298180us
I20260709 12:25:54.850374  8686 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:54.873788  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 517) took 1097 ms. Trace:
I20260709 12:25:54.897998  5990 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:54.900090  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:54.899792  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:54.901321  5945 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:25:54.857335  8637 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:54.902686  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 335013us
I20260709 12:25:54.862754  8645 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:54.912981  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:54.874432  6022 rpcz_store.cc:276] 0709 12:25:53.776338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.777266 (+   928us) service_pool.cc:224] Handling call
0709 12:25:53.777676 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.778115 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.778148 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.778751 (+   603us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.140072 (+361321us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:54.140119 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.140197 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.873456 (+733259us) raft_consensus.cc:1685] finished
0709 12:25:54.873488 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.873706 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1284,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":360818,"threads_started":1,"wal-append.queue_time_us":469689}
W20260709 12:25:54.915808  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 294577us
I20260709 12:25:54.855409  8274 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:54.922369  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:54.922447  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 184785us
I20260709 12:25:54.888943  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:54.912213  5940 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:54.861717  8703 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:54.933323  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:54.934623  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 422) took 1307 ms. Trace:
I20260709 12:25:54.860730  8706 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:25:54.960330  5873 rpcz_store.cc:276] 0709 12:25:53.627093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.744497 (+117404us) service_pool.cc:224] Handling call
0709 12:25:53.744924 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.777878 (+ 32954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.777911 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.778518 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.218158 (+439640us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:54.218221 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.218299 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.934295 (+715996us) raft_consensus.cc:1685] finished
0709 12:25:54.934326 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.934540 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32407,"raft.queue_time_us":33753,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":88,"thread_start_us":409601,"threads_started":1,"wal-append.queue_time_us":516159}
I20260709 12:25:54.923836  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:54.852764  8727 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 12:25:54.899751  8638 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.915416  8539 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:54.920686  8544 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:54.860077  8510 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:54.981289  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:54.989048  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 526) took 1008 ms. Trace:
I20260709 12:25:54.996676  6020 rpcz_store.cc:276] 0709 12:25:53.980921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.029208 (+ 48287us) service_pool.cc:224] Handling call
0709 12:25:54.097833 (+ 68625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:54.098212 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.098251 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:54.105372 (+  7121us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.317133 (+211761us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:54.317173 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.317252 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.988729 (+671477us) raft_consensus.cc:1685] finished
0709 12:25:54.988765 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.988956 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":163608,"threads_started":1,"wal-append.queue_time_us":398421}
I20260709 12:25:55.004014  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:54.966614  8663 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:54.966953  8645 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:54.975526  8539 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.008888  5873 raft_consensus.cc:2410] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:25:55.010083  6609 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:55.034444  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.035306  8716 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.228s	user 0.000s	sys 0.000s
W20260709 12:25:55.036051  8720 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
W20260709 12:25:55.037563  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.038021  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.041813  5940 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:25:55.051998  8724 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:25:55.045496  8381 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.986230  8638 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:55.052116  8684 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:25:55.056792  5949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 856) took 1263 ms (client timeout 1649 ms). Trace:
W20260709 12:25:55.058482  8726 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:54.971517  8722 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:55.061528  5874 raft_consensus.cc:1217] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 4.3->[6.4-6.4]   Dedup: 6.4->[]
W20260709 12:25:55.083144  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 75180us
W20260709 12:25:55.094095  8723 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.236s	user 0.001s	sys 0.001s
I20260709 12:25:55.052942  8715 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:55.080355  8539 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:55.095729  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:55.097767  8554 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:55.099510  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:55.102495  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:55.101429  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:55.110400  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:55.105993  5949 rpcz_store.cc:269] 0709 12:25:53.793477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.795249 (+  1772us) service_pool.cc:224] Handling call
0709 12:25:55.056543 (+1261294us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b5400672d54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:55.056702 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1734436736}
I20260709 12:25:55.113375  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:55.135303  6778 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:25:55.113865  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:55.083992  5867 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:25:55.136802  6793 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:55.116973  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:55.115924  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:55.120947  5874 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:25:55.066636  8721 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:55.139143  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:55.139894  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.132794  6022 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:55.133525  6024 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:55.111233  8684 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:55.148177  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:55.124078  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 226133us
I20260709 12:25:55.136191  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 801) took 1004 ms. Trace:
I20260709 12:25:55.143421  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:55.147383  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:55.153751  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:55.162810  6756 raft_consensus.cc:2393] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 8: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:55.163024  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:55.163913  5867 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:55.096050  8663 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:55.168463  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:55.171594  8733 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:25:55.172533  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 443124us
I20260709 12:25:55.173185  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 798) took 1081 ms. Trace:
I20260709 12:25:55.174010  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:55.174448  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 683) took 1195 ms. Trace:
I20260709 12:25:55.174537  8738 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:55.115235  8117 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:25:55.177922  8710 raft_consensus.cc:3055] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:55.180274  8672 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:55.181119  8612 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:55.003362  8274 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:55.134487  8554 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.193897  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:55.208781  8734 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 12:25:55.150974  6021 rpcz_store.cc:276] 0709 12:25:54.131630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.131846 (+   216us) service_pool.cc:224] Handling call
0709 12:25:54.132273 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.146877 (+ 14604us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:54.534909 (+388032us) log.cc:844] Serialized 14 byte log entry
0709 12:25:54.925552 (+390643us) log.cc:844] Serialized 14 byte log entry
0709 12:25:55.130035 (+204483us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:55.130098 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:55.130149 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.130900 (+   751us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.136084 (+  5184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41731,"prepare.queue_time_us":462571,"prepare.run_cpu_time_us":173,"prepare.run_wall_time_us":2752,"raft.queue_time_us":350887,"raft.run_cpu_time_us":459,"raft.run_wall_time_us":27332,"thread_start_us":862130,"threads_started":3}
W20260709 12:25:55.175827  8517 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.632s	user 0.007s	sys 0.004s
I20260709 12:25:55.200924  6017 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
W20260709 12:25:55.201121  5866 consensus_meta.cc:287] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.548s	user 0.013s	sys 0.000s
I20260709 12:25:55.197016  6778 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:25:55.206278  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:55.205825  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 679) took 1205 ms. Trace:
I20260709 12:25:55.124622  8722 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:55.154196  8735 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:55.229730  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:55.205981  8738 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.262903  8735 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:55.213313  8544 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:55.211984  8710 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:55.103572  8381 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:55.197093  6023 rpcz_store.cc:276] 0709 12:25:54.091945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.092122 (+   177us) service_pool.cc:224] Handling call
0709 12:25:54.114162 (+ 22040us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:54.152457 (+ 38295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.152510 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:54.186618 (+ 34108us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.538691 (+352073us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:54.912099 (+373408us) log.cc:844] Serialized 14 byte log entry
0709 12:25:55.065541 (+153442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.065655 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:55.172937 (+107282us) raft_consensus.cc:1685] finished
0709 12:25:55.172965 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.173123 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17188,"prepare.queue_time_us":168153,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":213489,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":13305,"thread_start_us":713391,"threads_started":3,"wal-append.queue_time_us":395349}
I20260709 12:25:55.193414  8384 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:55.246994  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 117116us
I20260709 12:25:55.228596  8612 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:55.201098  8611 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:55.197558  6773 rpcz_store.cc:276] 0709 12:25:53.979119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.217708 (+238589us) service_pool.cc:224] Handling call
0709 12:25:54.232628 (+ 14920us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:54.285186 (+ 52558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.285223 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:54.285844 (+   621us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.591407 (+305563us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:54.591475 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.591586 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:55.174233 (+582647us) raft_consensus.cc:1685] finished
0709 12:25:55.174253 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.174386 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1759,"raft.queue_time_us":58855,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"thread_start_us":299153,"threads_started":1,"wal-append.queue_time_us":474566}
I20260709 12:25:55.248973  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:55.231986  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 115136us
I20260709 12:25:55.288360  5865 raft_consensus.cc:2410] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:25:55.288647  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:55.257750  5866 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
W20260709 12:25:55.230301  8661 consensus_meta.cc:287] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.001s
I20260709 12:25:55.217307  8703 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:55.235487  6793 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:25:55.228189  8672 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:55.271618  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:55.274544  5869 raft_consensus.cc:2410] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 12:25:55.243414  8516 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.871s	user 0.015s	sys 0.000s
W20260709 12:25:55.275738  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46129.9us
I20260709 12:25:55.309931  5840 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:55.312249  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:55.154533  6775 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:25:55.313982  6775 raft_consensus.cc:606] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Received request to transfer leadership to TS 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:55.255038  6768 rpcz_store.cc:276] 0709 12:25:53.977539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.978271 (+   732us) service_pool.cc:224] Handling call
0709 12:25:53.978806 (+   535us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:54.046964 (+ 68158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.047005 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:54.103313 (+ 56308us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.506938 (+403625us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:54.506990 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.507079 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:55.181615 (+674536us) raft_consensus.cc:1685] finished
0709 12:25:55.182705 (+  1090us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.182875 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50383,"raft.queue_time_us":81917,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":123,"thread_start_us":328631,"threads_started":1,"wal-append.queue_time_us":563379}
I20260709 12:25:55.243026  8554 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:55.316510  5873 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 7: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:55.243124  8731 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.283284  6017 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:55.289979  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:55.276640  8340 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:55.294610  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:55.260263  8517 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:55.266582  8738 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:55.244421  8637 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:25:55.321555  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:55.345495  5865 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:55.206923  8645 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:55.282527  8735 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:55.374014  8735 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:55.374651  8735 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:55.377053  8735 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:55.378391  8735 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:55.379171  8735 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:55.379865  8735 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:55.318969  8565 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:55.388540  5873 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:55.391455  5873 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:55.392983  5873 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:25:55.393374  5873 pending_rounds.cc:85] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 3
I20260709 12:25:55.393651  5873 pending_rounds.cc:107] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:25:55.395491  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:55.400357  5873 raft_consensus.cc:2889] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:55.317318  8672 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:55.287319  8544 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:55.403998  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:55.345213  8340 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.286410  8384 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:55.299558  8661 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:55.346041  8703 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:55.408345  8715 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.319293  8516 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:55.491230  8629 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.345s	user 0.016s	sys 0.000s
I20260709 12:25:55.427518  8340 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:55.502126  5870 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:25:55.551033  8663 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:55.462082  8114 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:55.394743  8731 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:55.560619  8608 consensus_meta.cc:287] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.163s	user 0.017s	sys 0.001s
I20260709 12:25:55.605057  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1158) took 1226 ms. Trace:
I20260709 12:25:55.605700  5942 rpcz_store.cc:276] 0709 12:25:54.379102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.379449 (+   347us) service_pool.cc:224] Handling call
0709 12:25:54.380900 (+  1451us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:54.411774 (+ 30874us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:54.412145 (+   371us) log.cc:844] Serialized 14 byte log entry
0709 12:25:55.236098 (+823953us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:55.255477 (+ 19379us) log.cc:844] Serialized 52 byte log entry
0709 12:25:55.255838 (+   361us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:55.255894 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.256005 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:55.604719 (+348714us) raft_consensus.cc:1685] finished
0709 12:25:55.604746 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.604959 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41997,"prepare.queue_time_us":795,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":48,"raft.queue_time_us":188361,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":29894,"spinlock_wait_cycles":22422144,"thread_start_us":812548,"threads_started":2,"wal-append.queue_time_us":523683}
I20260709 12:25:55.567152  8117 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.619287  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:55.568359  8638 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:55.410008  8612 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:55.620386  6023 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
W20260709 12:25:55.629807  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 317601us
I20260709 12:25:55.414347  8710 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:55.640411  8515 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:55.641789  8515 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:55.642127  6023 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19"
dest_uuid: "6add331326ca47038308a447bf798fa7"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:48972
I20260709 12:25:55.645504  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 873) took 1486 ms. Trace:
I20260709 12:25:55.646093  6019 rpcz_store.cc:276] 0709 12:25:54.162261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.220124 (+ 57863us) service_pool.cc:224] Handling call
0709 12:25:54.242308 (+ 22184us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:54.653517 (+411209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.645128 (+991611us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b5400641754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:55.645425 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":860478208}
I20260709 12:25:55.589026  8114 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:55.639292  5840 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:55.573069  8629 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:55.632747  8730 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 12:25:55.664147  8381 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:55.654297  8709 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
W20260709 12:25:55.716542  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.674599  8544 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:55.666779  8745 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 12:25:55.575861  8715 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:55.681780  8763 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 12:25:55.640501  8710 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:55.656164  8608 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:55.731199  8608 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:55.686312  8753 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.734397  8753 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:55.686889  8661 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:55.687309  8722 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:55.688190  8772 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:55.737569  8772 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:55.738210  8772 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:55.743057  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 546214us
W20260709 12:25:55.737835  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 814029us
I20260709 12:25:55.744694  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:55.654703  8719 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:55.654043  8737 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.242s	user 0.000s	sys 0.000s
I20260709 12:25:55.626009  8117 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:55.692605  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.695708  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.696005  8664 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:55.666663  8765 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
W20260709 12:25:55.699594  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.698945  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.665387  8516 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:55.698375  8629 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:55.738349  8147 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:55.704365  8686 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:25:55.664176  5948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 850) took 1872 ms (client timeout 1550 ms). Trace:
I20260709 12:25:55.695741  8384 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:55.791654  8731 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:55.792273  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:55.792716  8715 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:55.717792  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 47419.6us
I20260709 12:25:55.794061  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:55.672559  8114 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:55.795286  8389 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:55.795940  8389 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:55.796777  8114 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:55.748349  6611 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:55.754534  6023 raft_consensus.cc:606] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:55.797771  6023 raft_consensus.cc:614] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:55.798717  8703 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:55.773731  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:55.802060  6021 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:55.743279  8668 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:25:55.774986  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:55.810485  8340 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:55.811084  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:55.811776  5869 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:55.812837  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:55.762487  8612 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:55.815279  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 678) took 1838 ms. Trace:
I20260709 12:25:55.815755  6766 rpcz_store.cc:276] 0709 12:25:53.977105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.977361 (+   256us) service_pool.cc:224] Handling call
0709 12:25:53.977794 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.849105 (+871311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.814986 (+965881us) spinlock_profiling.cc:243] Waited 622 ms on lock 0x7b54002a0dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:55.815205 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1430332288}
I20260709 12:25:55.817760  8686 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:55.818409  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 810) took 1374 ms. Trace:
I20260709 12:25:55.762413  8638 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.819741  8638 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:55.818867  5868 rpcz_store.cc:276] 0709 12:25:54.443570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.443786 (+   216us) service_pool.cc:224] Handling call
0709 12:25:54.464209 (+ 20423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:54.486741 (+ 22532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.486786 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:54.501398 (+ 14612us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.877516 (+376118us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:55.267740 (+390224us) log.cc:844] Serialized 14 byte log entry
0709 12:25:55.362030 (+ 94290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.362161 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:55.818144 (+455983us) raft_consensus.cc:1685] finished
0709 12:25:55.818170 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.818341 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":581585,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":29424,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":16976,"thread_start_us":762558,"threads_started":3,"wal-append.queue_time_us":728170}
I20260709 12:25:55.782955  8147 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.822170  8147 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:55.798780  8667 raft_consensus.cc:3055] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:55.823428  8667 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:55.824383  8667 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:55.788422  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 43870.5us
I20260709 12:25:55.798802  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:55.826463  6020 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:55.790015  8608 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:55.769400  8767 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.827601  8767 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:55.828138  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.828959  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.829362  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.783531  5948 rpcz_store.cc:269] 0709 12:25:53.791773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.792552 (+   779us) service_pool.cc:224] Handling call
0709 12:25:55.660680 (+1868128us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b5400633154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:55.664063 (+  3383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2183191424}
W20260709 12:25:55.829771  8756 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:25:55.830022  8707 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:55.778368  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:55.787710  8117 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:55.821350  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:55.833981  5869 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:55.844859  5868 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:55.796993  8389 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:55.799098  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:55.842329  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 94139.6us
I20260709 12:25:55.842612  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:55.851915  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:55.839015  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 44231.8us
I20260709 12:25:55.850224  6019 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:55.855839  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:55.858299  8735 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:55.858740  6756 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:55.853727  8289 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.860569  8289 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:55.862013  8684 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.693s	user 0.012s	sys 0.002s
I20260709 12:25:55.862293  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:55.862531  8684 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:55.862937  8289 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:55.864477  8554 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:55.865139  8684 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:55.860852  8735 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:55.880822  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:55.874032  8554 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:55.840664  8667 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:55.863010  5868 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:55.879436  8638 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:55.878155  8753 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:55.788467  8515 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:55.929977  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:55.916268  8539 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.745s	user 0.014s	sys 0.004s
I20260709 12:25:55.884869  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:55.887849  8738 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:55.948671  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.855423  8516 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:55.891158  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:55.906648  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 54810.9us
I20260709 12:25:55.956702  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:55.928125  8736 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.869s	user 0.007s	sys 0.007s
I20260709 12:25:55.910889  6775 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:55.910876  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 433) took 1009 ms. Trace:
I20260709 12:25:55.907954  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:55.873307  8772 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:55.919201  6021 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:25:55.854810  8672 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:55.935822  5869 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:55.940090  8711 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:55.941879  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 811) took 1474 ms. Trace:
I20260709 12:25:55.942701  8114 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.944406  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:55.945175  8615 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:55.947252  8517 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:55.932377  6019 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:55.950371  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:55.965776  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:55.978042  5838 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:56.008253  5874 rpcz_store.cc:276] 0709 12:25:54.467042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.489079 (+ 22037us) service_pool.cc:224] Handling call
0709 12:25:55.061369 (+572290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:55.137929 (+ 76560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.941540 (+803611us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b540005dcd0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:55.941785 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1274380544}
I20260709 12:25:55.978652  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:56.007297  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:55.988967  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:56.014842  5940 raft_consensus.cc:2368] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:25:56.015038  5942 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:56.006877  6611 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:56.013163  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 62912.6us
I20260709 12:25:56.003842  5868 raft_consensus.cc:2410] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:56.018276  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:56.005971  6773 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:55.955924  8516 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.022593  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:56.024230  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:55.958315  5865 rpcz_store.cc:276] 0709 12:25:54.901322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.371119 (+469797us) service_pool.cc:224] Handling call
0709 12:25:55.371526 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:55.371769 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:55.603441 (+231672us) log.cc:844] Serialized 14 byte log entry
0709 12:25:55.906751 (+303310us) log.cc:844] Serialized 14 byte log entry
0709 12:25:55.910385 (+  3634us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:55.910476 (+    91us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:55.910544 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.910618 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.910801 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38530,"prepare.queue_time_us":91473,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":4285,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":1877,"spinlock_wait_cycles":19437056,"thread_start_us":457890,"threads_started":2}
I20260709 12:25:56.028321  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:56.020987  6020 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:25:56.018836  5868 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:55.955551  8707 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.972177  8629 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:55.972086  8767 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:56.040925  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 62977.3us
I20260709 12:25:56.040956  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:56.042071  6017 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:56.035497  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:56.036361  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 57832.5us
I20260709 12:25:56.029069  5873 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:55.974493  8340 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:56.045248  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:56.021852  8711 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:55.953557  8539 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:55.950839  8515 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:56.005328  8602 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:56.127398  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 427) took 1670 ms. Trace:
I20260709 12:25:56.087108  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:56.085418  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:56.128507  5865 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:25:56.045499  5913 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:56.129798  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 84398.4us
W20260709 12:25:56.131247  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.090734  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 45995.3us
I20260709 12:25:56.091246  5868 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
W20260709 12:25:56.095399  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 51111.4us
I20260709 12:25:56.045784  8707 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:56.027758  8516 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:56.005291  8644 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.041905  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:56.045398  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:56.101253  6017 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:56.106922  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:56.040540  8629 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.047444  8663 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:56.037984  8706 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:56.043183  8664 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:56.114279  8711 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:56.119029  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:56.124707  8747 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.011965  8114 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:56.088775  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 47652.2us
I20260709 12:25:56.160744  6611 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:56.128610  5946 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:25:56.128159  5871 rpcz_store.cc:276] 0709 12:25:54.457144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.457301 (+   157us) service_pool.cc:224] Handling call
0709 12:25:54.477454 (+ 20153us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:54.492765 (+ 15311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.492821 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:54.520376 (+ 27555us) log.cc:844] Serialized 52 byte log entry
0709 12:25:54.618390 (+ 98014us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b480000cfd8. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d7bacaa1 00007f39d7ba3b37 00007f39d7ba39ca 00007f39d7c9b26f 00007f39d7c9af37 00007f39d7caf55c 00007f39d7caf508 00007f39d7caf488 00007f39d7caf443 00007f39d7cae6ec 00007f39d6bd8a96
0709 12:25:54.853711 (+235321us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:54.853778 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.853867 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:55.887631 (+1033764us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:25:56.126974 (+239343us) raft_consensus.cc:1685] finished
0709 12:25:56.127021 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.127295 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10004,"raft.queue_time_us":198,"raft.run_cpu_time_us":2646,"raft.run_wall_time_us":129310,"spinlock_wait_cycles":32551552,"thread_start_us":303564,"threads_started":1,"wal-append.queue_time_us":426006}
I20260709 12:25:56.143034  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:56.087720  8117 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:56.163158  6611 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:56.144414  6019 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:56.173851  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:56.102979  8638 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:56.152536  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:56.158134  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 116300us
I20260709 12:25:56.178391  6017 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:56.178651  5838 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:56.154467  8014 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:56.159243  6609 leader_election.cc:400] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:25:56.208724  5871 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:56.151556  8664 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.217682  6609 leader_election.cc:403] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:25:56.148865  8629 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:56.220120  5871 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:56.220602  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:56.221613  6017 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
W20260709 12:25:56.224260  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 45689.2us
W20260709 12:25:56.225100  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.229086  8735 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:56.185006  8014 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.180437  8727 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:56.230118  8014 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:56.230310  8727 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:56.220211  8664 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:56.235872  6019 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:56.169445  8708 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:56.242800  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.150408  8554 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:25:56.169503  8117 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.209770  8514 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:56.208524  8710 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.216207  8663 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:56.261459  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 124145us
W20260709 12:25:56.268210  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 58746.6us
W20260709 12:25:56.276355  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.270110  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:56.279757  6019 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:56.247632  8708 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.169914  8666 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 12:25:56.284790  6017 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:56.287184  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:56.151446  8644 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:56.290347  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:56.170403  8747 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:56.299615  6017 raft_consensus.cc:2368] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:25:56.263718  8117 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:56.242574  8404 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:25:56.272835  8738 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:56.255995  8729 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 12:25:56.159513  8706 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:56.318897  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 1.38895e+06us
I20260709 12:25:56.289673  8708 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:56.334441  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.335546  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.371629  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:56.339650  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:56.374460  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:56.376981  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:56.379070  5987 connection.cc:256] RPC call timeout handler was delayed by -1.3955486579998251s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
W20260709 12:25:56.379632  5987 connection.cc:256] RPC call timeout handler was delayed by -1.3934984600000462s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2082s for a response
W20260709 12:25:56.380059  5987 connection.cc:256] RPC call timeout handler was delayed by -1.3680945040000552s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
W20260709 12:25:56.380416  5987 connection.cc:256] RPC call timeout handler was delayed by -1.351894504000029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
I20260709 12:25:56.386698  5867 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:56.390116  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:56.393538  6766 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:56.396584  8661 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.397436  6611 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:56.398375  6611 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:56.399179  6611 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:56.400727  6611 leader_election.cc:400] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:25:56.402010  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:25:56.403617  8762 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:56.404163  8762 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:56.404404  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:56.409703  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:56.413416  6756 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 12:25:56.397246  8661 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:56.421559  5871 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:25:56.425148  5871 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:25:56.428022  6766 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:25:56.433055  8661 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:56.433763  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:56.434162  8661 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:56.435335  8661 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.435971  8661 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:56.438386  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:56.455731  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.461855  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 76561.6us
I20260709 12:25:56.463550  5987 leader_election.cc:400] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
I20260709 12:25:56.463990  5987 leader_election.cc:403] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:25:56.464818  8738 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
I20260709 12:25:56.480583  5944 raft_consensus.cc:2410] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:25:56.481156  6019 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 11: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:56.490439  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:56.312546  8672 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:56.286998  8710 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:56.501765  8710 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:56.502597  8710 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:56.503062  8710 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:56.503509  8710 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:56.489611  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:56.505986  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:56.508595  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:56.491434  5949 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:25:56.325513  8772 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:56.529381  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.405186  8762 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:56.539898  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:56.553323  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 277785us
I20260709 12:25:56.563058  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:56.564972  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:56.326009  8722 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.185s
I20260709 12:25:56.506987  8738 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:56.569274  6016 raft_consensus.cc:2410] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:25:56.606294  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:56.607772  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:56.608390  6766 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:56.594849  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.621431  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:56.622277  6778 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:25:56.551908  8602 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:56.533586  8515 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:56.640348  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.570415  8715 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:56.600421  8778 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.314s	user 0.001s	sys 0.001s
W20260709 12:25:56.652575  6773 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.632s	user 0.014s	sys 0.000s
I20260709 12:25:56.563347  8539 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:56.659485  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:56.653915  8738 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:56.601403  8672 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.599674  8554 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.085s
I20260709 12:25:56.627327  8707 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:56.664121  8517 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.648010  8565 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:56.600791  8666 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:56.675328  8706 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:56.686517  8786 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.385s	user 0.001s	sys 0.000s
W20260709 12:25:56.687027  8780 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.536s	user 0.000s	sys 0.002s
W20260709 12:25:56.688021  8791 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.191s	user 0.000s	sys 0.000s
I20260709 12:25:56.702927  8644 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:56.708278  8661 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:56.714314  8516 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.562s	user 0.012s	sys 0.002s
I20260709 12:25:56.717005  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:56.718173  8727 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:25:56.719584  8792 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:25:56.617048  8703 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:56.724782  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.758029  6773 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:56.533679  8721 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.116s
I20260709 12:25:56.795480  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:56.798723  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 440) took 1018 ms. Trace:
I20260709 12:25:56.890545  6766 raft_consensus.cc:2410] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:25:56.766642  8672 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:56.764644  8602 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:25:56.802964  5873 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.047s	sys 0.116s
W20260709 12:25:56.901836  5873 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.047s	sys 0.117s
I20260709 12:25:56.713476  8710 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:56.802237  8612 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.989s	user 0.015s	sys 0.000s
W20260709 12:25:56.838012  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.851078  6757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.023s	sys 0.051s
W20260709 12:25:56.903255  6757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.024s	sys 0.051s
W20260709 12:25:56.810184  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 494167us
I20260709 12:25:56.767694  8738 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:56.891659  5867 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.009s	sys 0.030s
I20260709 12:25:56.811005  8747 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:56.890800  5869 rpcz_store.cc:276] 0709 12:25:55.780664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.812535 (+ 31871us) service_pool.cc:224] Handling call
0709 12:25:56.798435 (+985900us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b5400118ad4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:56.798631 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46999296}
I20260709 12:25:56.898819  6756 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:25:56.904868  5867 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.009s	sys 0.030s
I20260709 12:25:56.795109  8517 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:56.800848  8565 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:56.923326  8517 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:56.924880  5872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 426) took 2479 ms (client timeout 1691 ms). Trace:
I20260709 12:25:56.909161  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 441) took 1065 ms. Trace:
I20260709 12:25:56.909864  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:56.906602  5915 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:56.922111  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1026) took 1169 ms. Trace:
I20260709 12:25:56.924642  8565 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:56.811827  8666 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:56.937518  5838 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:56.932420  6766 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:56.948287  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:56.943485  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:56.978878  5941 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:25:56.903122  5874 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.001s	sys 0.013s
I20260709 12:25:56.975893  5867 rpcz_store.cc:276] 0709 12:25:55.844109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.387710 (+543601us) service_pool.cc:224] Handling call
0709 12:25:56.388034 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.388282 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:56.388538 (+   256us) log.cc:844] Serialized 14 byte log entry
0709 12:25:56.908658 (+520120us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:56.908738 (+    80us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:56.908803 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.908905 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.909083 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4193,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":3372,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":119,"thread_start_us":519701,"threads_started":1}
W20260709 12:25:56.979369  5874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.002s	sys 0.013s
I20260709 12:25:56.955163  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:56.970914  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:56.949415  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:56.974949  5866 rpcz_store.cc:276] 0709 12:25:55.752305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.752521 (+   216us) service_pool.cc:224] Handling call
0709 12:25:55.773398 (+ 20877us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:55.838945 (+ 65547us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:55.838976 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:55.865983 (+ 27007us) log.cc:844] Serialized 26 byte log entry
0709 12:25:56.140858 (+274875us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:56.141101 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.141209 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:56.921845 (+780636us) raft_consensus.cc:1685] finished
0709 12:25:56.921869 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.922037 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8315,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":13346,"thread_start_us":210732,"threads_started":1,"wal-append.queue_time_us":940227}
W20260709 12:25:56.987962  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 50588.8us
W20260709 12:25:56.990415  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.925513  5872 rpcz_store.cc:269] 0709 12:25:54.445690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.446117 (+   427us) service_pool.cc:224] Handling call
0709 12:25:56.924629 (+2478512us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54004a0454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:56.924777 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2989180800}
W20260709 12:25:56.997100  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 90564us
I20260709 12:25:56.954701  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:56.998533  5869 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:56.998520  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:56.889372  8532 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
W20260709 12:25:57.043233  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44795.9us
I20260709 12:25:57.044494  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:56.834604  8516 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:56.927258  8629 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.672s	user 0.011s	sys 0.004s
W20260709 12:25:56.946209  8781 log.cc:927] Time spent T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Append to log took a long time: real 0.738s	user 0.000s	sys 0.003s
I20260709 12:25:57.100452  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:57.085754  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 686) took 1140 ms. Trace:
I20260709 12:25:57.086032  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:57.093022  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:56.959735  8710 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:57.097312  6020 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.013s	sys 0.010s
I20260709 12:25:57.097577  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:57.110037  6020 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.014s	sys 0.011s
I20260709 12:25:56.889750  8703 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:56.934478  8782 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.542s	user 0.000s	sys 0.002s
I20260709 12:25:57.045792  8515 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:25:57.049772  8645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.672s	user 0.018s	sys 0.049s
I20260709 12:25:56.949312  8715 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:57.117776  6775 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:57.045156  8743 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.002s	sys 0.013s
I20260709 12:25:56.901610  8602 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:25:56.937386  8783 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
W20260709 12:25:57.117537  8645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.740s	user 0.018s	sys 0.049s
I20260709 12:25:57.106940  8629 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:57.132565  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.132181  8602 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:57.134060  8629 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:57.135578  8790 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.862s	user 0.000s	sys 0.003s
I20260709 12:25:57.124574  8715 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:56.959606  8641 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.140848  8641 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:57.141464  8738 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:57.142299  8795 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.333s	user 0.001s	sys 0.002s
I20260709 12:25:57.085150  8707 consensus_queue.cc:1059] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:25:57.143978  8641 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:57.144520  8672 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:57.145350  8014 consensus_meta.cc:287] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.914s	user 0.009s	sys 0.003s
I20260709 12:25:57.146618  5946 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 12:25:57.093880  8615 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.072s	user 0.014s	sys 0.000s
W20260709 12:25:57.093528  8667 consensus_meta.cc:287] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.170s	user 0.013s	sys 0.002s
I20260709 12:25:57.154765  5868 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:56.950358  8612 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:57.046815  8637 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:57.045589  8706 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:25:57.115332  8515 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.167519  6015 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 12:25:57.168133  6015 pending_rounds.cc:85] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 4
I20260709 12:25:57.168412  6015 pending_rounds.cc:107] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 12:25:57.169862  6015 raft_consensus.cc:2889] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:57.149251  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:57.021876  8663 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.005s	sys 0.011s
I20260709 12:25:57.101508  6757 rpcz_store.cc:276] 0709 12:25:55.945687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.945884 (+   197us) service_pool.cc:224] Handling call
0709 12:25:55.946409 (+   525us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:55.946990 (+   581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:55.947019 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:55.947653 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 12:25:56.138278 (+190625us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:56.304870 (+166592us) log.cc:844] Serialized 14 byte log entry
0709 12:25:56.903637 (+598767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.903718 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.085480 (+181762us) raft_consensus.cc:1685] finished
0709 12:25:57.085501 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.085687 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":184520,"prepare.queue_time_us":150759,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":4298,"raft.queue_time_us":69569,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":162,"thread_start_us":930111,"threads_started":3,"wal-append.queue_time_us":291043}
I20260709 12:25:57.143110  8661 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:57.157760  8721 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:57.184763  5870 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.805s	user 0.000s	sys 0.031s
I20260709 12:25:57.156558  8738 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:25:57.147697  8289 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.000s	sys 0.027s
W20260709 12:25:57.191567  5945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.105s	user 0.006s	sys 0.033s
I20260709 12:25:57.190527  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:57.189826  5870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.000s	sys 0.031s
I20260709 12:25:57.160485  8637 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:25:57.182713  8117 consensus_meta.cc:287] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.863s	user 0.012s	sys 0.000s
W20260709 12:25:57.181232  8686 consensus_meta.cc:287] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.363s	user 0.010s	sys 0.006s
I20260709 12:25:57.171564  8515 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:57.183243  8667 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:57.197322  5945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.111s	user 0.006s	sys 0.033s
I20260709 12:25:57.193524  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1037) took 1376 ms. Trace:
W20260709 12:25:57.195470  8289 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.000s	sys 0.028s
I20260709 12:25:57.157763  8014 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:57.208418  8602 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:57.177306  8641 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:57.181474  8752 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:57.185475  8663 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.883s	user 0.005s	sys 0.014s
W20260709 12:25:57.197304  6611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.006s	sys 0.018s
W20260709 12:25:57.249882  6611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.006s	sys 0.018s
W20260709 12:25:57.250306  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 703334us
I20260709 12:25:57.252907  8602 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:57.254344  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:57.265498  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1034) took 1449 ms. Trace:
I20260709 12:25:57.265736  8752 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:57.266454  5874 rpcz_store.cc:276] 0709 12:25:55.815825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.219907 (+404082us) service_pool.cc:224] Handling call
0709 12:25:56.256995 (+ 37088us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.263179 (+  6184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.263225 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.263824 (+   599us) log.cc:844] Serialized 26 byte log entry
0709 12:25:56.979879 (+716055us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:56.979928 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.980003 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.265033 (+285030us) raft_consensus.cc:1685] finished
0709 12:25:57.265070 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.265390 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3211,"raft.queue_time_us":4193,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":10213,"thread_start_us":662876,"threads_started":1,"wal-append.queue_time_us":963912}
W20260709 12:25:57.225775  5990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.020s	sys 0.028s
I20260709 12:25:57.164393  8612 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:57.239178  6609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.019s	sys 0.031s
I20260709 12:25:57.236776  5870 rpcz_store.cc:276] 0709 12:25:55.816930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.378258 (+561328us) service_pool.cc:224] Handling call
0709 12:25:56.378633 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.378934 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:56.379245 (+   311us) log.cc:844] Serialized 14 byte log entry
0709 12:25:57.192946 (+813701us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:57.193030 (+    84us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:57.193106 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.193208 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.193432 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13312,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":2613,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":8872,"thread_start_us":813162,"threads_started":1}
I20260709 12:25:57.171758  8801 consensus_queue.cc:1059] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:25:57.211328  8637 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:57.273259  8612 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:57.196280  8707 raft_consensus.cc:3055] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:57.269073  8663 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:57.196485  8721 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:57.285833  8707 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:57.286654  8663 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:57.287035  8707 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:57.287701  8721 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:57.288425  8641 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:57.288616  8707 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:57.219058  8686 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:57.213728  8738 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:57.218684  8117 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:57.241402  8747 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:57.235716  8667 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.275790  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:57.341706  6019 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:57.276747  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:57.343086  6021 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
W20260709 12:25:57.281258  6609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.020s	sys 0.031s
I20260709 12:25:57.269361  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:57.344631  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 787332us
I20260709 12:25:57.345043  6016 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:57.273373  5990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.021s	sys 0.028s
I20260709 12:25:57.290429  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:57.303236  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 548) took 1523 ms. Trace:
I20260709 12:25:57.308975  6768 raft_consensus.cc:2393] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:57.274766  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:57.348601  6015 rpcz_store.cc:276] 0709 12:25:55.779695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.854086 (+ 74391us) service_pool.cc:224] Handling call
0709 12:25:55.880352 (+ 26266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.179953 (+1299601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.302930 (+122977us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400364cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:57.303157 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2950347520}
I20260709 12:25:57.303602  6766 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:57.379220  6022 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:57.319559  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:57.382520  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:57.357156  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.302007  6775 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:57.330296  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:57.281267  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:57.282356  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:57.399266  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:25:57.284896  8514 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.009s	user 0.007s	sys 0.003s
I20260709 12:25:57.403584  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:57.395114  5942 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:25:57.404349  6762 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:57.319120  5944 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:25:57.304782  8711 consensus_meta.cc:287] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.125s	user 0.008s	sys 0.004s
I20260709 12:25:57.413760  8711 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:57.416440  8711 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:57.417358  6785 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:57.401667  8721 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:57.418395  6024 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:57.418238  8721 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:57.419811  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:57.323318  8747 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:57.420564  6024 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:25:57.310797  8014 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:57.330996  8801 raft_consensus.cc:3055] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:57.346454  8767 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 12:25:57.421823  8801 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:57.289407  8641 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:57.422901  8801 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:57.317610  8664 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.085s	user 0.006s	sys 0.006s
I20260709 12:25:57.424115  8801 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:57.424237  8664 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:57.324312  8807 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:25:57.323592  8667 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:57.426518  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:57.401589  6757 raft_consensus.cc:2368] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9.
I20260709 12:25:57.419816  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 688) took 1461 ms. Trace:
I20260709 12:25:57.433493  6773 raft_consensus.cc:2393] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 8: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:57.305848  8516 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:57.425940  6023 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 10. Ops: [8.7-8.7]
I20260709 12:25:57.468171  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:57.289561  8752 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:57.468986  6023 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:57.470899  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:57.471560  6023 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:57.315909  8147 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.493s	user 0.010s	sys 0.005s
I20260709 12:25:57.489477  6019 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:57.430065  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:57.306455  8666 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:57.431599  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 679) took 1580 ms. Trace:
I20260709 12:25:57.506583  6768 rpcz_store.cc:276] 0709 12:25:55.846989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.847137 (+   148us) service_pool.cc:224] Handling call
0709 12:25:57.427551 (+1580414us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540081b5d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:57.427755 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3110354176}
I20260709 12:25:57.322896  8703 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:57.438265  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:57.246866  8769 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.010s	sys 0.021s
I20260709 12:25:57.439319  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:57.428730  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:57.440455  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:57.431818  8711 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.433959  6785 rpcz_store.cc:276] 0709 12:25:55.958303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.958484 (+   181us) service_pool.cc:224] Handling call
0709 12:25:56.011739 (+ 53255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.417933 (+1406194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.419566 (+  1633us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54002a21d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:25:57.419749 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3231759744}
I20260709 12:25:57.443681  8747 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:57.448259  8515 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:57.420676  8721 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:57.443689  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:57.450140  8664 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:57.309216  8738 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:57.283708  8114 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:57.458799  6762 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:57.311627  8117 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:57.462006  8602 consensus_queue.cc:1059] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:57.463716  5990 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:57.465214  8708 consensus_meta.cc:287] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.092s	user 0.006s	sys 0.006s
I20260709 12:25:57.311300  8686 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:57.486833  6016 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
W20260709 12:25:57.511653  8684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.885s	user 0.024s	sys 0.039s
I20260709 12:25:57.489291  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:57.528494  6773 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:57.529320  5949 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:57.529969  8602 consensus_queue.cc:1243] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:57.530624  8602 raft_consensus.cc:3055] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:57.516499  8711 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:57.429733  5948 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:25:57.531817  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68158.3us
I20260709 12:25:57.496662  8667 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:57.426414  8514 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:57.500229  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:57.317786  8715 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:57.509994  5944 raft_consensus.cc:2410] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:57.509871  6015 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:25:57.511605  6021 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:25:57.511775  6024 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:25:57.514513  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:57.431916  8612 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
W20260709 12:25:57.512218  8769 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.010s	sys 0.022s
I20260709 12:25:57.522038  6757 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:57.487819  6022 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:25:57.488158  8615 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:25:57.555318  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:57.531000  8602 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:57.532544  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:57.497843  8637 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:57.555323  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:57.580025  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:57.581100  5948 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
W20260709 12:25:57.581450  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.536849  8721 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:57.534929  5942 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:57.534891  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:57.575829  6016 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:25:57.540813  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:57.526536  8747 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:57.500623  8147 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:57.554220  6773 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
W20260709 12:25:57.534071  8404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.162s	user 0.015s	sys 0.033s
I20260709 12:25:57.574013  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:57.534549  8117 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:25:57.536934  8684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.025s	sys 0.040s
I20260709 12:25:57.540287  6019 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:57.585547  6022 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:57.527285  8708 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:57.538120  8686 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:57.608121  5949 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:57.557453  8738 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:57.592764  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:57.533463  5987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.986s	user 0.023s	sys 0.029s
W20260709 12:25:57.601548  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.589779  6611 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:57.627281  6015 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:57.625537  6023 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:57.615911  6022 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:25:57.619971  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:57.635196  6024 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:57.627822  5987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.080s	user 0.023s	sys 0.029s
W20260709 12:25:57.636765  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 1.09684e+06us
I20260709 12:25:57.649014  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:57.770833  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 181101us
I20260709 12:25:57.704025  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:57.588835  8721 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.708235  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:57.685460  6015 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:57.748536  6024 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:57.690008  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:57.695634  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.811987  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:57.812588  6022 raft_consensus.cc:2368] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9.
I20260709 12:25:57.698755  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:57.813333  6021 raft_consensus.cc:2410] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 12:25:57.813386  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.587884  8803 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:57.830720  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 133193us
W20260709 12:25:57.734193  8808 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:25:57.826084  8721 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:57.640600  5946 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 10: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:25:57.768793  5840 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.005s	user 0.008s	sys 0.026s
I20260709 12:25:57.814675  6015 pending_rounds.cc:85] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 4
W20260709 12:25:57.850515  5840 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.087s	user 0.008s	sys 0.026s
I20260709 12:25:57.850593  6015 pending_rounds.cc:107] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:25:57.850977  8715 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:57.657488  5913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.008s	sys 0.003s
W20260709 12:25:57.851800  6608 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.088s	user 0.000s	sys 0.036s
W20260709 12:25:57.604671  8389 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.987s	user 0.025s	sys 0.051s
W20260709 12:25:57.851917  5913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.009s	sys 0.012s
I20260709 12:25:57.851084  6015 raft_consensus.cc:2889] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:57.852953  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 732288us
W20260709 12:25:57.852448  6608 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.088s	user 0.000s	sys 0.036s
W20260709 12:25:57.590770  8404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.247s	user 0.015s	sys 0.033s
I20260709 12:25:57.831933  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:57.843848  8708 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:57.844352  8711 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:57.807994  6611 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:57.851469  8612 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:25:57.781359  8762 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.012s	sys 0.020s
I20260709 12:25:57.782251  8802 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:57.807711  6610 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.055s	user 0.030s	sys 0.047s
W20260709 12:25:57.867985  6610 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.115s	user 0.031s	sys 0.047s
I20260709 12:25:57.854136  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:57.834589  8811 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:57.856841  6024 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:57.734009  8637 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:57.727111  8667 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:57.627044  8738 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:25:57.853681  8389 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.255s	user 0.026s	sys 0.051s
I20260709 12:25:57.863600  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:57.827028  8515 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:57.637073  8686 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:57.851059  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 1.22571e+06us
W20260709 12:25:57.906193  8816 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 12:25:57.910977  8668 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.015s	sys 0.006s
I20260709 12:25:57.870878  5913 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:57.917609  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:57.919020  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 48223.8us
W20260709 12:25:57.867550  5873 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.951s	user 0.028s	sys 0.037s
W20260709 12:25:57.920302  5873 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.029s	sys 0.038s
I20260709 12:25:57.921411  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 438) took 2149 ms. Trace:
W20260709 12:25:57.921537  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 58089.2us
I20260709 12:25:57.922501  5987 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:57.921880  5873 rpcz_store.cc:276] 0709 12:25:55.772059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.155738 (+383679us) service_pool.cc:224] Handling call
0709 12:25:56.156090 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.156311 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:56.902330 (+746019us) log.cc:844] Serialized 14 byte log entry
0709 12:25:57.920847 (+1018517us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:57.920938 (+    91us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:57.921012 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.921139 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.921328 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10622,"prepare.queue_time_us":87182,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":59,"thread_start_us":1710535,"threads_started":2}
I20260709 12:25:57.873023  8711 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.924608  8389 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:57.925205  8389 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.924787  8711 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:57.926044  8114 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.926587  8114 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:57.927325  8813 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:57.927536  8711 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:57.818193  8684 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:25:57.947230  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:57.947956  5868 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:57.928617  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:57.943571  5873 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 12:25:57.876972  8818 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:25:57.879758  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:57.879472  6020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.007s	sys 0.020s
W20260709 12:25:57.964136  6020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.008s	sys 0.021s
I20260709 12:25:57.859680  8752 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:57.691957  8602 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:57.607053  8117 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:57.869493  8802 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.865195  8612 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:57.896638  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:57.900060  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:25:57.867188  8762 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.013s	sys 0.022s
I20260709 12:25:57.890898  8812 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:25:57.883586  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 75730.5us
W20260709 12:25:57.917495  6018 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.008s	sys 0.015s
I20260709 12:25:57.876276  8147 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:57.948288  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:57.953284  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:57.954531  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:57.877301  8810 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:58.005926  6773 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:25:57.950080  8663 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.010s	sys 0.014s
W20260709 12:25:57.972312  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:58.006891  6018 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.008s	sys 0.020s
I20260709 12:25:57.919562  8729 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:57.970161  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:57.970820  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:58.034600  5872 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:58.035307  6793 raft_consensus.cc:2410] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:25:57.969825  5990 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:58.036742  8814 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 12:25:57.930621  8389 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:57.949939  8668 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.015s	sys 0.006s
I20260709 12:25:58.006273  6611 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:58.043594  6609 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.014s	sys 0.023s
W20260709 12:25:58.044584  6609 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.014s	sys 0.023s
W20260709 12:25:58.044958  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 617710us
I20260709 12:25:57.981999  6785 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:57.852218  8803 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:57.929728  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.928680  8114 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:58.055119  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 48931.1us
I20260709 12:25:57.962527  8703 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
W20260709 12:25:58.025807  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.909s	user 0.021s	sys 0.021s
W20260709 12:25:57.911278  8781 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.000s	sys 0.021s
W20260709 12:25:58.037389  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 67644.9us
I20260709 12:25:58.068189  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:57.983156  5874 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:57.928582  8637 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:25:58.025198  8810 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:58.008561  8707 consensus_meta.cc:287] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.719s	user 0.016s	sys 0.000s
W20260709 12:25:58.069752  6778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.950s	user 0.017s	sys 0.044s
W20260709 12:25:58.042665  5943 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.022s	sys 0.015s
I20260709 12:25:58.008697  8743 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:25:58.028319  8663 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.011s	sys 0.015s
W20260709 12:25:58.051834  5941 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.022s	sys 0.025s
W20260709 12:25:58.053788  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.024261  8802 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:58.037806  8818 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:25:58.067811  8711 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:58.055944  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:57.968184  5868 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
W20260709 12:25:58.063549  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.021s	sys 0.022s
W20260709 12:25:58.103734  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 47883us
W20260709 12:25:58.103876  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 1.01138e+06us
I20260709 12:25:58.104518  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:57.965816  8738 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:58.112287  8738 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:58.066953  5838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.014s	sys 0.025s
I20260709 12:25:58.042750  8812 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:25:58.113880  5838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.014s	sys 0.025s
W20260709 12:25:58.114426  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 777655us
I20260709 12:25:58.033898  8729 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:25:58.072003  6778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.956s	user 0.018s	sys 0.044s
I20260709 12:25:58.116076  8817 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:58.116782  8817 raft_consensus.cc:697] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:58.107527  6611 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:25:58.083133  5943 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.022s	sys 0.015s
I20260709 12:25:58.077993  8615 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:58.068039  8781 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.000s	sys 0.025s
W20260709 12:25:58.094795  5941 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.022s	sys 0.026s
I20260709 12:25:58.091053  6609 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:25:58.055893  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 417) took 2072 ms. Trace:
I20260709 12:25:58.102615  5872 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:25:58.068813  8822 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:58.106648  5913 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:58.074254  8810 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:58.138803  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:58.109697  6793 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:25:58.110356  8644 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:58.147145  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:58.148105  5942 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:58.093060  8769 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:58.149307  5947 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:25:58.120756  5944 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:58.051540  8819 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.153493  6773 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:58.152902  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:58.155961  6024 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:25:58.121946  6611 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:58.123417  5945 rpcz_store.cc:276] 0709 12:25:55.983135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.983785 (+   650us) service_pool.cc:224] Handling call
0709 12:25:56.023705 (+ 39920us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.030901 (+  7196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.030935 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.080964 (+ 50029us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.236152 (+1155188us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:57.236225 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.236355 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:58.055667 (+819312us) raft_consensus.cc:1685] finished
0709 12:25:58.055686 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.055831 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12757,"raft.queue_time_us":16767,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":42509,"thread_start_us":1149758,"threads_started":1,"wal-append.queue_time_us":1676799}
I20260709 12:25:58.127844  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:58.130155  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.130227  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:58.108992  8823 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:58.134625  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:58.161792  6768 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:58.096516  8684 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:58.087596  8114 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:58.136435  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:58.202867  6016 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:58.152902  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:58.204427  5872 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:58.159500  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:58.132526  8663 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:25:58.207340  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.169780  5873 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 6: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:25:58.162618  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.117702  8708 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:25:58.211280  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 89437.2us
I20260709 12:25:58.213737  6022 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 12:25:58.096808  8743 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:58.130731  8711 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:58.241545  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:58.242708  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:58.216638  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:58.218302  5865 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 6: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:25:58.218971  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 71321.5us
I20260709 12:25:58.221946  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:58.225530  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:58.250216  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:58.250746  6022 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:58.251197  5870 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:58.156800  8667 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:58.232489  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:58.260386  5873 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:25:58.157169  8801 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:58.252720  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:58.254216  6024 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:25:58.153618  8539 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:58.082161  8612 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:58.277577  8602 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:58.278427  8612 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:58.257831  6023 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:58.258076  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:58.282521  6019 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:25:58.282603  8706 raft_consensus.cc:3055] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:58.283208  8706 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:58.284132  8706 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:58.284775  6768 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:58.291749  8743 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:58.291348  8729 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:58.170432  8769 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:58.293499  5868 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:58.294798  8769 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:58.295457  8769 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:58.297092  8769 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:25:58.299624  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.300155  8117 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:58.300869  8667 raft_consensus.cc:697] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:58.302209  8667 consensus_queue.cc:237] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:58.303184  8721 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:58.259011  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:58.306571  8721 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:58.229022  8516 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:58.310210  8516 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:58.311354  8516 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:58.312970  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:58.314190  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:58.205585  8823 raft_consensus.cc:697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:58.264511  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:58.269050  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:58.330945  6757 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:58.272751  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:25:58.275661  8821 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 12:25:58.275177  8812 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:58.276098  8539 raft_consensus.cc:697] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:58.369999  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 696) took 1842 ms. Trace:
I20260709 12:25:58.370100  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:58.219810  8684 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:58.328677  6785 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 10: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:58.335649  5868 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:58.233121  5838 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:58.342320  5942 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:25:58.117722  8817 consensus_queue.cc:237] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:58.150367  8822 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:58.357858  8747 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.767s	user 0.008s	sys 0.006s
I20260709 12:25:58.358516  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:58.383356  6768 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:58.383823  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:58.369092  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:58.384747  6019 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:58.230561  8114 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:58.293363  8729 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:58.373101  5948 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:25:58.370594  6778 rpcz_store.cc:276] 0709 12:25:56.527691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.929903 (+402212us) service_pool.cc:224] Handling call
0709 12:25:56.930301 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.930697 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.930729 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.976377 (+ 45648us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.115559 (+1139182us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:58.115609 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.115701 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:58.369721 (+254020us) raft_consensus.cc:1685] finished
0709 12:25:58.369748 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.369928 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":181,"raft.run_cpu_time_us":58,"raft.run_wall_time_us":59,"thread_start_us":998944,"threads_started":1,"wal-append.queue_time_us":1266311}
I20260709 12:25:58.266979  8615 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.248512  8663 raft_consensus.cc:697] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:25:58.377530  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.381121  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:25:58.381160  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 148117us
I20260709 12:25:58.366117  5872 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:58.379917  8684 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:58.387020  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:58.421188  6023 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:25:58.422744  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:58.390105  5915 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:58.379293  8801 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:58.263103  8708 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:58.168016  8819 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:58.413266  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.385288  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:58.445827  6022 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:58.305087  8824 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 12:25:58.315555  8686 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:58.425530  6611 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:58.302681  8711 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:58.402539  8641 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.977s	user 0.029s	sys 0.054s
I20260709 12:25:58.411037  8707 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.389321  8706 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:58.388432  8637 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:58.405987  6017 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.142s	user 0.016s	sys 0.022s
I20260709 12:25:58.431033  5915 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:58.455266  6017 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.192s	user 0.016s	sys 0.022s
I20260709 12:25:58.456337  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:58.315536  8769 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:58.387776  8721 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:58.433962  5840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.007s	sys 0.026s
I20260709 12:25:58.428856  5838 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:58.464864  5840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.007s	sys 0.026s
W20260709 12:25:58.466118  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 539518us
I20260709 12:25:58.443377  6019 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:25:58.437826  8747 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:58.451573  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:25:58.453723  8641 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.028s	user 0.030s	sys 0.054s
I20260709 12:25:58.478284  6022 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:25:58.457459  8637 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.485303  8637 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:58.488287  8703 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:58.419044  8615 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:58.454263  8799 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:58.465013  8404 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:58.434319  8835 consensus_queue.cc:1059] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 12:25:58.464788  8828 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:58.506047  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:58.519702  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:58.522951  6019 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:58.535148  6020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.014s	sys 0.042s
I20260709 12:25:58.535715  6023 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:25:58.457054  8707 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:58.539142  6020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.015s	sys 0.042s
W20260709 12:25:58.537808  5987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.008s	sys 0.039s
W20260709 12:25:58.546350  5987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.008s	sys 0.039s
W20260709 12:25:58.547748  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 599485us
I20260709 12:25:58.503360  8799 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.560823  6019 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:58.545871  8813 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:58.562337  6611 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:58.562752  8799 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:58.565989  8715 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:58.566488  8715 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.566939  8715 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:58.505684  8803 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:58.570869  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.483512  8721 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:58.526279  8828 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.577777  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 556) took 2213 ms. Trace:
I20260709 12:25:58.578537  6020 rpcz_store.cc:276] 0709 12:25:56.334971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.380631 (+ 45660us) service_pool.cc:224] Handling call
0709 12:25:56.381028 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.381271 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:57.110714 (+729443us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.545455 (+1434741us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.545552 (+    97us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:58.545664 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.548364 (+  2700us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.548599 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4584,"prepare.queue_time_us":1202700,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":4804,"raft.queue_time_us":62548,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":9554,"thread_start_us":2158431,"threads_started":3,"wal-append.queue_time_us":965215}
I20260709 12:25:58.584547  6768 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:25:58.586863  5990 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:58.504341  8767 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 12:25:58.545362  8835 consensus_queue.cc:1243] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:58.594497  8721 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:58.550455  8801 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:58.489319  8810 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:58.562474  8813 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:58.616951  8835 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:58.617851  8835 raft_consensus.cc:697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:58.619786  8615 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:58.455873  8823 consensus_queue.cc:237] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:58.621910  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:58.618816  8835 consensus_queue.cc:237] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:58.624617  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:58.620541  8813 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:58.598278  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:58.628064  8703 raft_consensus.cc:3055] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:58.628458  6768 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:25:58.628973  8703 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:58.630641  8822 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:58.632035  8114 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:58.632292  5990 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:58.482805  8539 consensus_queue.cc:237] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:58.571871  8117 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:58.635746  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.638283  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:58.654485  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 80138.4us
I20260709 12:25:58.663409  5987 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:58.656172  5873 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
W20260709 12:25:58.659257  6015 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.015s	sys 0.022s
W20260709 12:25:58.667219  6015 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.015s	sys 0.022s
I20260709 12:25:58.671700  5870 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
W20260709 12:25:58.680437  5941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.002s	sys 0.022s
W20260709 12:25:58.681142  5941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.002s	sys 0.022s
I20260709 12:25:58.681192  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:25:58.681720  5943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.013s	sys 0.011s
W20260709 12:25:58.685379  5943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.013s	sys 0.011s
I20260709 12:25:58.501431  8663 consensus_queue.cc:237] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:25:58.674026  8668 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.012s	sys 0.030s
I20260709 12:25:58.688032  6020 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:58.682430  5871 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:58.639356  8602 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:58.590494  8828 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:58.629097  8810 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:58.611732  8707 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:58.689883  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:58.679978  8813 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:58.740092  5872 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:58.695606  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:58.707584  6609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.016s	sys 0.033s
W20260709 12:25:58.741641  6609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.017s	sys 0.034s
W20260709 12:25:58.742479  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 651463us
I20260709 12:25:58.743530  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:58.717361  6018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.009s	sys 0.043s
W20260709 12:25:58.744297  6018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.010s	sys 0.043s
I20260709 12:25:58.736352  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 426) took 1361 ms. Trace:
I20260709 12:25:58.678464  8615 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:58.693435  8668 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.012s	sys 0.030s
I20260709 12:25:58.755200  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:58.746316  5943 rpcz_store.cc:276] 0709 12:25:57.375582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.375759 (+   177us) service_pool.cc:224] Handling call
0709 12:25:57.421414 (+ 45655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.421650 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:58.123428 (+701778us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.734855 (+611427us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.734947 (+    92us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:58.735018 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.736048 (+  1030us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.736259 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":871374,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":28294,"thread_start_us":1265512,"threads_started":2}
W20260709 12:25:58.762215  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.761819  8637 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:58.696195  8114 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:58.757486  6609 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:25:58.768970  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 73575us
I20260709 12:25:58.770429  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:58.772027  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:58.778640  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1180) took 1460 ms. Trace:
I20260709 12:25:58.684172  8117 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:58.779301  5941 rpcz_store.cc:276] 0709 12:25:57.318536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.324607 (+  6071us) service_pool.cc:224] Handling call
0709 12:25:57.325183 (+   576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.325524 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:58.122700 (+797176us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.748543 (+625843us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.778145 (+ 29602us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.778220 (+    75us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:58.778276 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.778370 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.778569 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71917,"prepare.queue_time_us":974462,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":86,"raft.queue_time_us":51872,"spinlock_wait_cycles":23868288,"thread_start_us":1355356,"threads_started":2}
I20260709 12:25:58.742323  8602 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:25:58.799811  8810 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:58.800377  8810 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.800719  8602 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:58.745751  8813 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.800751  8810 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:58.766633  8803 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:58.833091  5872 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:58.774224  8114 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:25:58.830086  6610 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.020s	sys 0.011s
I20260709 12:25:58.786825  8721 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:58.827376  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 55442.9us
I20260709 12:25:58.840646  8769 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:58.783066  8819 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:58.846498  6610 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.021s	sys 0.012s
I20260709 12:25:58.805053  8708 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:25:58.847620  6021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.039s	sys 0.042s
W20260709 12:25:58.854022  6021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.040s	sys 0.042s
I20260709 12:25:58.836077  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:58.826119  8752 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:58.777661  8637 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:58.811364  8813 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:58.864120  5946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.026s	sys 0.043s
W20260709 12:25:58.867585  5946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.026s	sys 0.043s
I20260709 12:25:58.839175  8615 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:58.851215  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:58.886937  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:58.841429  8664 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.006s	sys 0.028s
W20260709 12:25:58.897825  8664 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.007s	sys 0.028s
W20260709 12:25:58.900563  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.749260  8703 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:58.869380  8743 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:25:58.912037  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:58.918192  6024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.013s	sys 0.013s
W20260709 12:25:58.918951  6024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.013s	sys 0.013s
I20260709 12:25:58.875376  8686 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:25:58.920519  5865 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.012s	sys 0.013s
W20260709 12:25:58.856200  5874 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.002s	sys 0.032s
I20260709 12:25:58.869716  8842 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:58.925814  5874 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.002s	sys 0.032s
W20260709 12:25:58.926129  5865 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.013s	sys 0.013s
W20260709 12:25:58.936627  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.864008  8803 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.050s
W20260709 12:25:58.964130  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 535227us
I20260709 12:25:58.965492  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:58.966873  8769 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:58.916321  8721 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:58.915571  8843 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:58.959797  8743 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:58.958457  8703 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:58.967062  5838 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:59.012701  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.000s	sys 0.044s
W20260709 12:25:58.985734  6017 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.012s	sys 0.020s
W20260709 12:25:59.040645  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 73713.9us
W20260709 12:25:58.991711  5840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.007s	sys 0.014s
W20260709 12:25:59.040679  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.000s	sys 0.047s
W20260709 12:25:59.044433  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 557088us
W20260709 12:25:59.046511  6016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.011s	sys 0.015s
W20260709 12:25:59.047070  6016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.012s	sys 0.015s
W20260709 12:25:59.041505  6017 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.014s	sys 0.021s
W20260709 12:25:59.032320  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.053069  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:25:59.021010  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 263591us
I20260709 12:25:59.058451  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 561) took 1796 ms. Trace:
I20260709 12:25:59.067963  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:59.069232  5942 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
W20260709 12:25:59.065590  5840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.011s	sys 0.014s
W20260709 12:25:59.070372  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 603149us
I20260709 12:25:59.070990  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:59.078424  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:59.078760  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 44877.4us
I20260709 12:25:59.073263  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 812) took 1120 ms. Trace:
I20260709 12:25:59.080286  6762 rpcz_store.cc:276] 0709 12:25:57.952740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.952924 (+   184us) service_pool.cc:224] Handling call
0709 12:25:59.072942 (+1120018us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b54002a0dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:59.073152 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2207627648}
I20260709 12:25:59.082757  5915 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:59.066890  6017 rpcz_store.cc:276] 0709 12:25:57.262416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.262599 (+   183us) service_pool.cc:224] Handling call
0709 12:25:57.262938 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.263149 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:58.456110 (+1192961us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.057970 (+601860us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:59.058046 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:59.058113 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.058213 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.058374 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9904,"prepare.queue_time_us":377978,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":10049,"thread_start_us":1783692,"threads_started":2}
W20260709 12:25:58.989097  8667 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.019s	sys 0.017s
I20260709 12:25:59.090804  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:59.001777  8668 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.019379  8721 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.096922  5915 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:59.055477  8637 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:58.998813  8769 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 9 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:59.001662  8843 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:58.988696  8686 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:25:59.102975  5990 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:59.103910  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 277578us
W20260709 12:25:59.090142  8847 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.179s	user 0.002s	sys 0.000s
I20260709 12:25:59.120371  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:25:59.068326  8811 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:59.092097  8612 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.807s	user 0.015s	sys 0.000s
I20260709 12:25:58.962102  8114 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:25:59.097286  8667 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.021s	sys 0.020s
I20260709 12:25:59.095139  8664 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.123862  5915 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:59.119369  8711 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.019s	sys 0.070s
W20260709 12:25:59.129285  8853 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:25:59.131629  8840 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:25:59.116007  8843 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.135215  8686 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:59.093427  8641 consensus_queue.cc:1059] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
W20260709 12:25:59.137519  8844 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:25:59.139212  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 562) took 1874 ms. Trace:
I20260709 12:25:59.146427  6022 raft_consensus.cc:2410] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:25:59.132916  8848 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:25:59.163705  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 276863us
I20260709 12:25:59.102662  8707 consensus_queue.cc:1059] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:25:59.121001  8147 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:59.103672  8835 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:25:59.126086  8842 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:59.224987  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:59.110239  8708 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 12:25:59.108845  8668 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:25:59.144850  8823 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.010s	sys 0.013s
W20260709 12:25:59.167191  8711 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.019s	sys 0.071s
I20260709 12:25:59.189021  8114 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:59.112390  8721 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:59.200425  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:59.142205  8819 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:25:59.202358  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:25:59.231231  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:59.171073  8801 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:59.169591  8860 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:59.122510  8743 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:59.247543  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:59.174971  8117 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:59.250198  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:59.289436  6019 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.148346  8644 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:59.265432  6024 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:59.183682  6018 rpcz_store.cc:276] 0709 12:25:57.264739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.264896 (+   157us) service_pool.cc:224] Handling call
0709 12:25:57.265300 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.265740 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:58.029724 (+763984us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.767751 (+738027us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:58.806964 (+ 39213us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.807191 (+   227us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:58.807242 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.807350 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.138884 (+331534us) raft_consensus.cc:1685] finished
0709 12:25:59.138917 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.139119 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22872,"prepare.queue_time_us":459725,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":54121,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":21327,"thread_start_us":1432728,"threads_started":2,"wal-append.queue_time_us":937920}
I20260709 12:25:59.280581  6775 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:25:59.244688  6785 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
W20260709 12:25:59.279505  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 48440.4us
W20260709 12:25:59.281322  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 202970us
I20260709 12:25:59.300432  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1185) took 1182 ms. Trace:
I20260709 12:25:59.302296  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:59.305775  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 570) took 1781 ms. Trace:
I20260709 12:25:59.306270  6015 rpcz_store.cc:276] 0709 12:25:57.524446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.813312 (+288866us) service_pool.cc:224] Handling call
0709 12:25:57.814429 (+  1117us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.863476 (+ 49047us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.863528 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:57.864177 (+   649us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.667893 (+803716us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:58.667966 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.668079 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.305484 (+637405us) raft_consensus.cc:1685] finished
0709 12:25:59.305512 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.305698 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":791825,"threads_started":1,"wal-append.queue_time_us":1050864}
W20260709 12:25:59.315156  8846 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:25:59.315364  8752 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:59.315968  8752 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:59.251690  8637 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:25:59.320703  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.321499  5987 connection.cc:256] RPC call timeout handler was delayed by -1.5779786550001518s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15500000000000003s for a response
W20260709 12:25:59.322638  5987 connection.cc:256] RPC call timeout handler was delayed by -1.187378654999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19840000000000002s for a response
W20260709 12:25:59.323055  5987 connection.cc:256] RPC call timeout handler was delayed by -1.0370786550001867s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
W20260709 12:25:59.323482  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 256659us
I20260709 12:25:59.280079  8860 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.328183  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:59.304105  8802 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.180s	user 0.015s	sys 0.006s
I20260709 12:25:59.301138  5946 rpcz_store.cc:276] 0709 12:25:58.118541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.118713 (+   172us) service_pool.cc:224] Handling call
0709 12:25:58.145795 (+ 27082us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:58.146339 (+   544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.146365 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:58.234816 (+ 88451us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.877336 (+642520us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:58.877386 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.877465 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.300121 (+422656us) raft_consensus.cc:1685] finished
0709 12:25:59.300152 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.300345 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":13785,"thread_start_us":575285,"threads_started":1,"wal-append.queue_time_us":752512}
I20260709 12:25:59.279873  8801 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:25:59.334574  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:59.291539  8844 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
W20260709 12:25:59.251151  8823 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.011s	sys 0.013s
I20260709 12:25:59.304847  8842 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:59.371825  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 298500us
I20260709 12:25:59.375342  5838 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:25:59.375656  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 47548.8us
I20260709 12:25:59.378476  8663 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:59.379197  8663 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.379650  8663 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:59.390369  8866 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:25:59.371611  8706 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.911s	user 0.008s	sys 0.009s
W20260709 12:25:59.396616  8828 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.649s	user 0.010s	sys 0.000s
I20260709 12:25:59.396934  8747 raft_consensus.cc:3055] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:59.399753  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 829) took 1030 ms. Trace:
I20260709 12:25:59.421295  6022 rpcz_store.cc:276] 0709 12:25:58.369420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.482254 (+112834us) service_pool.cc:224] Handling call
0709 12:25:59.399482 (+917228us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b5400363dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:25:59.399680 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1433706240}
W20260709 12:25:59.401017  8715 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.834s	user 0.007s	sys 0.006s
I20260709 12:25:59.401844  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 579) took 1026 ms. Trace:
I20260709 12:25:59.402568  8668 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:59.402668  8667 raft_consensus.cc:3055] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:59.402946  8721 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:59.355373  8801 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:59.333326  8860 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:59.404065  8664 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:59.405200  8708 raft_consensus.cc:3055] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:59.407065  8859 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:59.411365  5873 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:59.479681  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:59.416213  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:59.480702  5942 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:25:59.416603  6021 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:25:59.424309  8872 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:25:59.424731  8389 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.359s	user 0.017s	sys 0.005s
I20260709 12:25:59.393587  8637 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:59.426469  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:25:59.432262  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:59.433561  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:59.435348  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:59.509003  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:59.438402  6775 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:59.436264  6016 rpcz_store.cc:276] 0709 12:25:58.375528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.487771 (+112243us) service_pool.cc:224] Handling call
0709 12:25:58.488063 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:58.488406 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.488429 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:58.488935 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.047616 (+558681us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:59.047662 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.047743 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.401502 (+353759us) raft_consensus.cc:1685] finished
0709 12:25:59.401530 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.401776 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4684,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":8904,"thread_start_us":558210,"threads_started":1,"wal-append.queue_time_us":869789}
W20260709 12:25:59.441897  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 66555.3us
I20260709 12:25:59.358956  8842 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:59.444684  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:59.445199  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:59.518142  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:59.517565  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:25:59.506469  5871 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:59.521831  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:59.452325  6019 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:59.454125  8539 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:25:59.465109  5987 leader_election.cc:336] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.550s (SENT)
I20260709 12:25:59.249225  8769 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:59.477424  5948 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 12:25:59.482991  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 188876us
I20260709 12:25:59.443147  8667 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:25:59.424760  8867 log.cc:927] Time spent T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:59.498293  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:59.360752  8802 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:59.482082  8708 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:25:59.479523  8859 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:59.421077  8747 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:25:59.514055  5947 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:59.527258  5944 raft_consensus.cc:2410] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:25:59.543358  8539 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:59.571614  8539 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:59.485536  8819 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 12:25:59.529436  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:59.531054  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:59.536329  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:59.529736  8842 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:59.540099  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:59.482725  8841 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:59.545086  5945 raft_consensus.cc:2368] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 8. Current term is 9.
I20260709 12:25:59.505738  6022 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:59.548735  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:25:59.584043  5941 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:25:59.550535  8875 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:25:59.556178  8864 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.241s	user 0.000s	sys 0.004s
W20260709 12:25:59.527881  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.479056  8715 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:59.435822  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 57055.3us
I20260709 12:25:59.581679  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:25:59.434437  8828 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:59.586364  6766 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 10: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:25:59.588449  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 48465.4us
I20260709 12:25:59.549886  5949 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:59.563616  8859 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:59.583276  8810 consensus_meta.cc:287] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.768s	user 0.014s	sys 0.000s
I20260709 12:25:59.591373  8841 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:59.594151  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:25:59.596148  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 60434.5us
W20260709 12:25:59.579851  8729 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.166s	user 0.012s	sys 0.004s
I20260709 12:25:59.590724  8861 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:59.536604  8516 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.048s
W20260709 12:25:59.601250  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 136167us
I20260709 12:25:59.590374  8747 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:59.686069  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 48004.4us
I20260709 12:25:59.516286  8389 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:59.692114  5943 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:25:59.673470  5947 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
I20260709 12:25:59.678747  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:59.687556  5987 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.984s (SENT) [suppressed 22 similar messages]
I20260709 12:25:59.688107  6778 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:25:59.601436  8404 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:59.684501  8738 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.530s	user 0.019s	sys 0.000s
I20260709 12:25:59.686350  8841 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:59.690590  8827 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.685292  8861 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.687829  8878 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:59.732044  5940 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:25:59.742964  8612 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.749189  8612 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:25:59.750634  8615 consensus_meta.cc:287] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.855s	user 0.017s	sys 0.000s
I20260709 12:25:59.751209  8615 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:59.753394  8828 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:59.753831  6023 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:25:59.690604  8708 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:59.686035  8667 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:59.685619  8810 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:59.757169  8810 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:59.755040  8708 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:59.730252  8721 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.758992  8389 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:59.760040  8827 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:59.732409  8747 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:59.724051  8404 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:59.761155  8861 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:59.762133  8862 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:25:59.766309  8813 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.866s	user 0.014s	sys 0.001s
I20260709 12:25:59.755115  8802 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:25:59.768033  8799 consensus_meta.cc:287] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.205s	user 0.015s	sys 0.001s
W20260709 12:25:59.768486  8684 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.333s	user 0.016s	sys 0.002s
I20260709 12:25:59.687700  8890 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:59.826946  8684 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:59.824470  8799 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:59.826040  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:59.830212  8875 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.831054  6757 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:59.831037  8875 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:59.832505  6022 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:59.834196  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:25:59.835014  6016 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:59.826790  8813 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:59.827105  8890 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:59.838820  8890 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.840180  8813 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:59.829329  8799 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:25:59.841944  6020 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:59.843070  8875 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:59.756903  8663 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:25:59.829345  8684 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:25:59.861685  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 174143us
I20260709 12:25:59.774907  8612 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:25:59.766357  8703 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:59.863306  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:25:59.774648  8843 consensus_meta.cc:287] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.585s	user 0.008s	sys 0.005s
I20260709 12:25:59.879160  5942 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:25:59.774086  8114 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
W20260709 12:25:59.881310  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 202686us
I20260709 12:25:59.817909  8404 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:59.884348  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:59.884881  6016 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:25:59.886107  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:59.856086  6022 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:25:59.862813  8875 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:25:59.882225  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 286145us
I20260709 12:25:59.890832  8842 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:59.887588  8875 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:25:59.778134  8721 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:25:59.898914  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:59.901878  5840 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:25:59.917528  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 323384us
W20260709 12:25:59.713304  8707 consensus_queue.cc:1175] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6add331326ca47038308a447bf798fa7" responder_term: 10 status { last_received { term: 5 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: [7.4-9.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:59.880402  8612 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:59.922902  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:59.932725  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:59.936852  5872 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.871641  8663 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:59.939400  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:59.939847  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:25:59.940443  6775 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:25:59.940788  5870 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 9: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:25:59.948194  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:25:59.958190  5865 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.019s	sys 0.018s
I20260709 12:25:59.914485  8637 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:59.958810  5865 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.019s	sys 0.018s
I20260709 12:25:59.958464  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:59.928478  8819 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:59.959693  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:25:59.969303  5915 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:25:59.961170  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 59414.3us
I20260709 12:25:59.963608  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:25:59.928833  8612 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:25:59.959491  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:25:59.883486  8843 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:25:59.971457  6762 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:59.964887  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:25:59.971601  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:59.944331  8539 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:59.946398  8715 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:25:59.922336  8835 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:59.921988  8389 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:25:59.980409  5987 leader_election.cc:400] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 8. Current term is 9.
I20260709 12:25:59.975451  5872 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:25:59.965848  8743 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:59.978844  8612 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.988858  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:25:59.962653  8637 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:25:59.990707  5840 leader_election.cc:400] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9.
I20260709 12:26:00.039958  5840 leader_election.cc:403] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:59.992009  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:25:59.969678  8819 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.044852  8819 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:00.047880  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 63785.5us
I20260709 12:25:59.982846  8615 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:00.029954  5987 leader_election.cc:403] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:00.048374  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56593.8us
I20260709 12:25:59.980124  8878 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:59.970080  8668 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:00.029335  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:00.039884  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 51106.9us
I20260709 12:25:59.985795  8889 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:59.979328  8859 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:00.064011  8810 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.064666  8810 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:00.039734  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:00.058218  8703 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9.
I20260709 12:26:00.061357  8878 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.982087  8828 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:25:59.977876  8752 consensus_meta.cc:287] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.661s	user 0.007s	sys 0.006s
I20260709 12:26:00.042604  8729 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.043102  8637 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:00.036316  8835 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:00.049463  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:00.056490  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:00.058089  6766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 695) took 1836 ms (client timeout 1817 ms). Trace:
I20260709 12:25:59.986644  8539 raft_consensus.cc:697] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:00.065606  8859 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:00.054359  8147 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:00.055141  8615 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.067874  8729 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:00.096096  8615 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:59.978688  8904 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:00.066219  5874 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:00.066535  8878 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.067319  8828 raft_consensus.cc:697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:00.094885  8663 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:00.067608  8752 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:00.129606  6018 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:00.108168  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:00.074662  6766 rpcz_store.cc:269] 0709 12:25:58.221287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.221571 (+   284us) service_pool.cc:224] Handling call
0709 12:26:00.057718 (+1836147us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400005854. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:00.057977 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3071845248}
I20260709 12:26:00.110400  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:00.075129  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:00.160379  6019 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:00.077425  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 97152.2us
I20260709 12:26:00.046746  8817 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:00.041203  8860 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.542s	user 0.013s	sys 0.000s
I20260709 12:26:00.163184  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:00.070325  6020 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:26:00.163134  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 113728us
I20260709 12:26:00.068961  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:26:00.162331  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 94654.4us
W20260709 12:26:00.167479  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.061005  6762 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:00.079867  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:00.166410  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 91378.7us
I20260709 12:26:00.168615  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.169487  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.169850  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.170511  5867 raft_consensus.cc:2368] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 8. Current term is 9.
I20260709 12:26:00.171697  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.173870  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.174561  5865 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
W20260709 12:26:00.098284  8901 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.162s	user 0.001s	sys 0.001s
I20260709 12:26:00.182672  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.179461  5840 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:00.078161  8897 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 12:26:00.178241  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 98475.3us
I20260709 12:26:00.180434  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.186522  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.187253  5869 raft_consensus.cc:2368] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
I20260709 12:26:00.187253  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:00.180644  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.187425  5873 raft_consensus.cc:2368] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8.
I20260709 12:26:00.188488  5872 raft_consensus.cc:2368] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 8. Current term is 9.
I20260709 12:26:00.188868  6016 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:00.077320  8811 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.191228  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.191447  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.191955  5870 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:26:00.192504  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.074360  8835 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:00.198743  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.066913  8810 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:00.097908  8729 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:00.070868  8843 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:25:59.986948  8801 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:00.189024  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.168917  8886 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:00.104120  8875 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:00.180601  8147 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:00.177310  8870 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:00.196202  8747 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:00.202816  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:00.192615  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.163393  8904 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:00.253543  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.254528  5866 raft_consensus.cc:2368] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
I20260709 12:26:00.192871  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.255424  5870 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:26:00.177881  8664 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.205793  8769 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:00.084127  8637 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:00.219285  8801 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:00.218726  8828 consensus_queue.cc:237] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:00.219864  5949 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:00.262063  5866 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:26:00.078873  8743 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:00.223702  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.224807  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:00.097522  8404 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:00.197033  8813 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:26:00.256088  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 48630.5us
W20260709 12:26:00.252835  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 62306.7us
W20260709 12:26:00.256778  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 54081.3us
I20260709 12:26:00.111261  8861 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:00.088433  8841 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:00.213898  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.269161  6611 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:00.205835  8668 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:26:00.220585  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.187980  8823 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:26:00.193544  8848 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:26:00.075642  8539 consensus_queue.cc:237] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:00.266757  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:00.178326  8860 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:00.118937  8615 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:00.259910  8810 consensus_queue.cc:1059] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:26:00.272842  8859 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:00.273455  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:00.258452  8729 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:00.273816  8870 consensus_queue.cc:1059] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:00.275504  8752 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:00.257229  8904 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:00.278239  8842 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:00.270179  8612 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:00.278991  8404 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:00.279201  8813 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:00.279699  8769 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:00.188267  8892 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:26:00.272486  6019 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:00.290550  8516 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:00.294934  8637 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:00.295487  8743 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:00.297013  8668 raft_consensus.cc:3055] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:00.319850  8147 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:26:00.269966  8849 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:00.413676  8910 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:26:00.261936  8664 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:00.256579  8827 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:00.416744  8841 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:00.417678  5947 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:00.318754  5866 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:00.322043  5869 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 4.
W20260709 12:26:00.326192  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 303641us
W20260709 12:26:00.328599  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 61922.3us
I20260709 12:26:00.422646  8743 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:00.329334  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 46526.2us
I20260709 12:26:00.410187  8703 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:00.426180  8703 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:00.426740  5873 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:00.427084  8703 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:00.428040  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.428278  8703 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.428684  5873 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.430051  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.431337  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:00.311084  8708 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.000s
I20260709 12:26:00.432152  6019 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:26:00.342722  5867 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:26:00.350157  5940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1195) took 1283 ms (client timeout 1570 ms). Trace:
I20260709 12:26:00.432713  8811 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 8. Current term is 9.
W20260709 12:26:00.432973  5940 rpcz_store.cc:269] 0709 12:25:59.066367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.066544 (+   177us) service_pool.cc:224] Handling call
0709 12:26:00.349883 (+1283339us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b5400633154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:00.350074 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1466764544}
I20260709 12:26:00.379871  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:00.309100  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:00.438122  6022 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:26:00.438426  8615 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:00.416777  8664 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:00.441802  6785 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:00.442663  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.443109  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:00.444495  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:00.445101  6785 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:26:00.446002  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:00.446767  6775 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:26:00.446985  8612 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:00.448397  6793 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:00.448660  8721 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:00.449717  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:00.450457  6756 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:00.452703  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:00.340409  8871 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:00.390882  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 833) took 1189 ms. Trace:
I20260709 12:26:00.394732  5990 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:00.369773  8888 raft_consensus.cc:3055] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:00.454519  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:00.454844  8888 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:00.455019  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 60379.8us
I20260709 12:26:00.454555  8515 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:00.456168  8515 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:00.455825  8888 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:00.404808  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:00.426180  5872 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:00.457504  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 705) took 1257 ms. Trace:
I20260709 12:26:00.454252  6023 rpcz_store.cc:276] 0709 12:25:59.201254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.202066 (+   812us) service_pool.cc:224] Handling call
0709 12:26:00.390711 (+1188645us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b5400763754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:00.390820 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1108821888}
W20260709 12:26:00.375550  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 107205us
I20260709 12:26:00.458034  6757 rpcz_store.cc:276] 0709 12:25:59.199659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.199820 (+   161us) service_pool.cc:224] Handling call
0709 12:25:59.200106 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.831796 (+631690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.457232 (+625436us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b5400004454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:00.457438 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1450512640}
I20260709 12:26:00.387154  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1191) took 1336 ms. Trace:
I20260709 12:26:00.408268  6017 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:00.459467  5840 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:00.365360  8817 raft_consensus.cc:3055] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:00.258222  8886 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:00.413340  8813 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:00.256654  8747 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:00.434612  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:00.435352  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.327561  8861 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:00.352038  8729 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:00.453428  5945 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
W20260709 12:26:00.386231  6778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 817) took 1765 ms (client timeout 2211 ms). Trace:
I20260709 12:26:00.403228  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:00.364238  8859 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:00.408368  8668 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:00.459939  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:00.459651  5943 rpcz_store.cc:276] 0709 12:25:59.051104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.053484 (+  2380us) service_pool.cc:224] Handling call
0709 12:25:59.053998 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.692784 (+638786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.386885 (+694101us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b5400602f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:00.387085 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1466601344}
I20260709 12:26:00.461723  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:00.461937  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:00.459384  8114 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:26:00.463140  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:00.463589  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:00.499955  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 77882.7us
I20260709 12:26:00.501271  5944 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:00.500169  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 75264.1us
I20260709 12:26:00.485337  6016 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:00.503767  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:00.504724  5946 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:00.468550  5866 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
W20260709 12:26:00.504243  6778 rpcz_store.cc:269] 0709 12:25:58.620846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.621529 (+   683us) service_pool.cc:224] Handling call
0709 12:26:00.385998 (+1764469us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54002c1fd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:00.386152 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2444835072}
I20260709 12:26:00.507647  6762 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 10: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:00.479842  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:00.508778  5948 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:00.457049  8888 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.511379  8114 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:00.514683  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:00.511538  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 49680.2us
I20260709 12:26:00.514114  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:00.520529  6785 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:00.520635  5840 leader_election.cc:400] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9.
I20260709 12:26:00.522058  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:00.514755  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:00.524407  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:00.524518  8114 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:00.525687  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:00.527323  5873 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:00.470772  8747 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:00.465322  8817 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:00.532898  6773 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:00.533685  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 263449us
I20260709 12:26:00.479902  8721 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:00.534015  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:00.536459  6778 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.479138  8860 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:00.500839  8859 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:00.499195  8516 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:00.536917  5913 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:00.571652  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.509965  8615 raft_consensus.cc:2764] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:26:00.573428  5948 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:00.518141  8842 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.543102  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:00.579728  6793 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.580410  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:00.581236  6023 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:00.578447  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:00.510793  8811 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.510561  8612 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:00.530476  8911 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 12:26:00.584249  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:00.586752  6778 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
W20260709 12:26:00.588845  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 86126.2us
I20260709 12:26:00.575744  8721 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:00.595310  8721 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:00.595788  6762 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:00.597903  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:00.598683  6762 raft_consensus.cc:2368] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9.
I20260709 12:26:00.599588  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:00.602336  8819 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.557s	user 0.012s	sys 0.000s
I20260709 12:26:00.601539  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:00.602901  8819 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:00.604174  5940 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 10: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:00.605080  8819 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:00.606937  8890 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.767s	user 0.007s	sys 0.008s
I20260709 12:26:00.606498  8878 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:00.607744  8667 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:00.609817  8612 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:00.613299  8612 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:00.615056  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:00.617980  6024 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 9: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:00.618238  8859 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:00.619184  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:00.619400  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:00.619668  8859 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.620097  6019 raft_consensus.cc:2410] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:00.620528  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:00.620966  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:00.621891  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:00.621691  8516 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:00.622681  6775 raft_consensus.cc:2393] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:00.623533  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:00.624033  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:00.624485  6756 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:00.624995  6775 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:00.621332  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:00.627151  5943 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:26:00.607430  8890 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:00.527220  8664 consensus_queue.cc:1059] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 12:26:00.457047  8515 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:00.629065  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.572589  8835 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:00.620790  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:00.604094  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:00.635360  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.698163  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:00.698832  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.528280  8708 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:00.637537  5987 leader_election.cc:400] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 8. Current term is 9.
I20260709 12:26:00.638445  6756 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:00.605110  6778 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:00.642268  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:00.642894  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.716792  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.714687  5987 leader_election.cc:403] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:26:00.644429  8114 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:00.646130  6022 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:00.671912  5865 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:00.722076  5865 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:00.587775  6017 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:00.679543  5874 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:26:00.722810  5874 pending_rounds.cc:85] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 6
I20260709 12:26:00.723109  5874 pending_rounds.cc:107] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:26:00.609303  8878 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:00.628545  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.726806  5867 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 7: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:00.730822  8114 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:26:00.691898  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 155040us
I20260709 12:26:00.694646  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.731689  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:00.732518  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.733249  5872 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:00.693411  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.706148  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.734877  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:00.542963  8860 consensus_queue.cc:1059] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.100s
W20260709 12:26:00.723347  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.723402  5874 raft_consensus.cc:2889] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:00.760579  8114 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:00.761055  5866 raft_consensus.cc:2368] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9.
I20260709 12:26:00.761111  8114 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:00.762229  8114 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:00.763671  8114 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.731129  5840 leader_election.cc:400] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9.
I20260709 12:26:00.715866  6778 raft_consensus.cc:606] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:00.781275  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.766647  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:00.736218  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:00.736479  8515 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:00.684514  8668 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:00.786136  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.579023  8912 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.208s
I20260709 12:26:00.737653  5865 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:00.670027  8811 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:00.727937  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.747015  5871 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:00.747426  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:00.749053  5944 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:00.749706  5873 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 12:26:00.782912  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.737192  6762 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 12:26:00.714568  8810 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:00.630280  8842 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:00.815836  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46575.7us
I20260709 12:26:00.673835  8886 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
I20260709 12:26:00.726172  8889 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.725651  8389 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.802533  5867 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:26:00.858265  6019 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
W20260709 12:26:00.808698  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 61366.3us
I20260709 12:26:00.852048  6793 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:00.810869  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:00.818535  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:00.745980  8875 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.821015  5865 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.863808  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:00.807093  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:00.824327  6611 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:00.787101  8801 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:00.832852  5840 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:00.750391  8644 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:00.833036  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:00.880836  6021 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 10: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:00.880144  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:00.842068  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:00.759835  8835 raft_consensus.cc:697] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:00.893605  5838 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:00.895018  8114 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:00.895777  8114 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:00.830832  8871 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:00.864035  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:00.864810  6775 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 10: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:00.690583  5946 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:00.905388  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:00.906932  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:00.907667  5946 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.911057  8114 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:00.925038  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.934885  6775 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:26:00.868317  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:00.872051  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:00.751552  8641 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 12:26:00.888109  8637 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:00.800310  8878 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 8. Current term is 9.
I20260709 12:26:00.876680  6020 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:00.815197  8810 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:00.881844  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 49142.3us
I20260709 12:26:00.968623  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:00.853566  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:00.861526  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 50742.8us
W20260709 12:26:00.897001  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 259587us
I20260709 12:26:00.921846  8762 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:00.971658  5987 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:00.972411  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:00.873064  8872 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:26:00.957549  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:00.978993  5943 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.729142  8817 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:00.848697  8848 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 12:26:00.899405  8644 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:00.912931  8893 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:26:00.873416  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:00.848073  8915 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.264s
I20260709 12:26:00.977074  8813 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.872205  8889 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:01.028268  6785 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.001s
I20260709 12:26:01.031396  6785 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:00.975666  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:00.878348  8875 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:00.883018  8389 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:00.990917  5948 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
W20260709 12:26:00.966835  8921 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.152s	user 0.001s	sys 0.001s
I20260709 12:26:01.008538  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:01.104847  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:00.972831  8860 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.960999  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:00.995157  8843 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:01.106050  8860 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:01.106101  5940 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:00.988373  8743 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:00.999004  8762 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:00.994866  8884 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 12:26:01.062058  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 86518.5us
I20260709 12:26:01.119578  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:01.019774  8841 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
W20260709 12:26:01.050552  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.009249  8823 consensus_queue.cc:1175] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6add331326ca47038308a447bf798fa7" responder_term: 9 status { last_received { term: 8 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:01.054071  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:01.129820  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 75840us
I20260709 12:26:01.088904  8664 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:01.051843  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:01.134492  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:01.069060  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:01.053143  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:01.136013  6785 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:26:01.072489  8703 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.644s	user 0.010s	sys 0.006s
I20260709 12:26:00.974674  8810 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:01.106005  5948 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 8: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:01.108537  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1199) took 1339 ms. Trace:
W20260709 12:26:01.108990  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.140496  5949 rpcz_store.cc:276] 0709 12:25:59.787778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.787937 (+   159us) service_pool.cc:224] Handling call
0709 12:25:59.788358 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.265195 (+476837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.108156 (+842961us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b5400633654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:01.108446 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":991777536}
I20260709 12:26:01.034278  8813 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:01.068665  8612 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:01.016579  8835 consensus_queue.cc:237] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:01.124619  6024 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:26:01.127533  5946 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:01.130754  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:01.133438  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:01.148738  6019 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:01.115535  8892 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:01.052697  8915 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:01.131839  8615 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:01.132308  8918 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:26:01.122530  8842 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:01.161055  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:01.115329  8811 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:01.138568  8516 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.024s	sys 0.049s
I20260709 12:26:01.179097  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:01.186857  5948 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:26:01.188526  5946 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:01.188467  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:01.189822  5949 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 10: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:26:01.150622  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 178278us
I20260709 12:26:01.191234  8612 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:01.173674  8615 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:01.192304  8615 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:01.192907  8615 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:01.193070  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:01.192118  8612 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:01.193902  6019 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:01.194936  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:01.195106  8644 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:01.196137  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:01.197515  8516 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.025s	sys 0.049s
I20260709 12:26:01.198585  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:01.199631  5948 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:01.200071  8612 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:01.200517  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:01.203138  5946 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:26:01.203730  5946 pending_rounds.cc:85] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 5
I20260709 12:26:01.204047  5946 pending_rounds.cc:107] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:26:01.204391  5946 raft_consensus.cc:2889] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:01.201088  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:01.197597  6019 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 10. Ops: [8.5-8.5]
I20260709 12:26:01.211016  8860 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:01.212420  6019 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 12:26:01.144811  8721 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:01.224120  8721 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:26:01.217760  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.192219  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.226485  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 242882us
I20260709 12:26:01.162647  8915 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:01.174770  8743 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:01.229939  6019 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:01.134863  8664 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:01.238514  5874 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:01.243602  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.247596  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 837) took 1069 ms. Trace:
I20260709 12:26:01.248328  5871 rpcz_store.cc:276] 0709 12:26:00.168517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.169564 (+  1047us) service_pool.cc:224] Handling call
0709 12:26:01.238051 (+1068487us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54007350d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:01.238353 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1264003968}
I20260709 12:26:01.252192  8801 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:01.254122  8801 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:01.255139  8938 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:01.261535  8938 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.263582  8938 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:01.267463  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 401311us
I20260709 12:26:01.209681  8389 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:01.261277  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 48748.6us
W20260709 12:26:01.273108  8928 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:26:01.274740  8813 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:01.274297  8915 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:01.275470  8923 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
W20260709 12:26:01.275820  8914 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
W20260709 12:26:01.276818  8925 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 12:26:01.278816  8762 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:01.258605  8940 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:01.283967  6016 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:01.225282  8721 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:01.285336  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 424598us
I20260709 12:26:01.286870  5940 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
W20260709 12:26:01.288266  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.295588  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.297171  8888 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.775s	user 0.008s	sys 0.008s
I20260709 12:26:01.338406  8888 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:01.340113  8888 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:01.340415  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 838) took 1257 ms. Trace:
I20260709 12:26:01.340633  8888 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:01.341596  8889 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:01.342123  5942 raft_consensus.cc:2410] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:01.344110  8729 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:01.345774  8875 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:01.347390  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:01.348766  6766 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:01.326184  5943 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:01.327731  8848 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:01.328373  6773 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:01.301802  8927 log.cc:927] Time spent T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 12:26:01.336083  8843 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:01.337175  8915 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:01.292374  8944 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.313719  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:01.310840  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:01.350764  6021 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:26:01.341094  6018 rpcz_store.cc:276] 0709 12:26:00.082822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.183962 (+101140us) service_pool.cc:224] Handling call
0709 12:26:00.209771 (+ 25809us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.210149 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.210178 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.210698 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.479836 (+269138us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:00.479881 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.479949 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.340165 (+860216us) raft_consensus.cc:1685] finished
0709 12:26:01.340191 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.340339 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":218171,"threads_started":1,"wal-append.queue_time_us":711572}
I20260709 12:26:01.293502  8389 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:26:01.352656  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:01.363003  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:01.372460  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 58881.1us
I20260709 12:26:01.379576  5915 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:01.372745  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 242021us
I20260709 12:26:01.381256  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:01.338411  8943 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:26:01.374330  5865 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:01.373034  5867 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:01.383385  5987 leader_election.cc:400] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
I20260709 12:26:01.388615  6017 raft_consensus.cc:2410] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:26:01.389145  5987 leader_election.cc:403] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 12:26:01.388126  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.362190  8747 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.788s	user 0.012s	sys 0.001s
I20260709 12:26:01.406869  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:01.416524  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.409521  5915 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:01.377354  8729 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:01.457592  6018 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e"
dest_uuid: "6add331326ca47038308a447bf798fa7"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:48972
I20260709 12:26:01.386052  8848 raft_consensus.cc:3055] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:01.399425  8875 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:01.361514  8615 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:01.403048  8843 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
I20260709 12:26:01.453719  8743 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:01.312427  8664 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:01.484472  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 101279us
I20260709 12:26:01.486131  5865 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:01.486601  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:01.489401  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 80018.9us
I20260709 12:26:01.520876  6018 raft_consensus.cc:606] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:01.521437  6018 raft_consensus.cc:614] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:01.520761  5987 leader_election.cc:400] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8.
I20260709 12:26:01.523765  5987 leader_election.cc:403] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:26:01.411343  8747 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:01.405642  8888 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:01.489838  8667 consensus_meta.cc:287] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.882s	user 0.010s	sys 0.007s
W20260709 12:26:01.480993  8904 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.109s	user 0.009s	sys 0.006s
I20260709 12:26:01.469098  8848 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:01.526544  5872 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:01.540373  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:01.521184  8729 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:01.522892  8940 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:01.549548  8940 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:01.522413  8843 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:01.524360  8931 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.525394  8664 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:01.546015  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:01.544369  8515 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:01.562248  5987 leader_election.cc:400] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
I20260709 12:26:01.560003  8888 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:01.571991  5987 leader_election.cc:403] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:26:01.573130  8843 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
I20260709 12:26:01.574836  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:26:01.545185  8859 consensus_meta.cc:287] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.925s	user 0.017s	sys 0.000s
I20260709 12:26:01.544034  8875 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:26:01.543776  8801 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.579329  5943 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:26:01.581486  5868 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
W20260709 12:26:01.583423  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.583467  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:01.565371  8752 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:01.555125  8950 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8.
I20260709 12:26:01.610021  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:01.613940  8875 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:01.619228  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:01.584045  8842 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:01.637156  5946 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:26:01.639052  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.610570  8612 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:01.387485  8944 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:01.542078  8904 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:01.631829  8842 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.566854  8931 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:01.647964  8932 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.537s	user 0.003s	sys 0.000s
W20260709 12:26:01.631204  8939 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 12:26:01.625367  8843 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:01.584738  8801 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:01.778312  8959 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.811834  6766 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:01.823660  5869 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:01.811710  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:01.653254  8721 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:26:01.815974  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.585016  8743 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:01.581521  8848 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:01.888814  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.879027  8667 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.794875  8842 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:01.816423  8615 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:01.574616  8940 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:01.901448  8721 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:01.827726  8747 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:01.927022  6775 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:26:01.942924  8956 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.333s	user 0.000s	sys 0.001s
I20260709 12:26:01.931602  8667 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:01.972342  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 353211us
I20260709 12:26:01.929121  8515 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:01.870281  8959 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:01.999428  5868 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 456) took 1372 ms (client timeout 1622 ms). Trace:
I20260709 12:26:01.999960  8515 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:02.001871  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:02.007097  8950 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:02.008858  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:01.999989  5868 rpcz_store.cc:269] 0709 12:26:00.627132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.628254 (+  1122us) service_pool.cc:224] Handling call
0709 12:26:01.999195 (+1370941us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b54002fa3d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:01.999344 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1965872384}
I20260709 12:26:02.016089  5987 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:26:02.019353  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.025820  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:02.028653  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.019301  8962 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.092s	user 0.001s	sys 0.001s
I20260709 12:26:01.945966  8721 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:02.043815  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 708) took 1608 ms. Trace:
I20260709 12:26:02.044389  6768 rpcz_store.cc:276] 0709 12:26:00.435552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.437735 (+  2183us) service_pool.cc:224] Handling call
0709 12:26:00.438104 (+   369us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:00.576432 (+138328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.576473 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:00.608714 (+ 32241us) log.cc:844] Serialized 52 byte log entry
0709 12:26:01.347466 (+738752us) spinlock_profiling.cc:243] Waited 36.7 ms on lock 0x7b18000f8ce0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:26:01.607254 (+259788us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:01.607326 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.607463 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.043387 (+435924us) raft_consensus.cc:1685] finished
0709 12:26:02.043414 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.043729 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95082,"raft.queue_time_us":105311,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":12500,"spinlock_wait_cycles":84436352,"thread_start_us":242229,"threads_started":1,"wal-append.queue_time_us":1313479}
I20260709 12:26:02.033615  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:02.036145  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:02.034933  8931 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:02.018795  8615 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:26:02.042172  8810 consensus_meta.cc:287] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.877s	user 0.007s	sys 0.009s
I20260709 12:26:02.062294  6775 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:02.023875  8875 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:02.219959  8912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.331s	user 0.002s	sys 0.005s
W20260709 12:26:02.429817  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.068595  8810 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:02.112357  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1081) took 1689 ms. Trace:
W20260709 12:26:02.430037  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.430070  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:02.428465  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1208) took 1254 ms. Trace:
I20260709 12:26:02.431182  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48972 (request call id 10) took 1032 ms. Trace:
I20260709 12:26:02.067081  8937 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.441795  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:02.442471  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.445776  5940 rpcz_store.cc:276] 0709 12:26:01.173976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.309438 (+135462us) service_pool.cc:224] Handling call
0709 12:26:01.309995 (+   557us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:01.442193 (+132198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.442231 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:01.448588 (+  6357us) log.cc:844] Serialized 52 byte log entry
0709 12:26:01.557785 (+109197us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:01.557873 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.557990 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.083381 (+525391us) raft_consensus.cc:1685] finished
0709 12:26:02.083420 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.428394 (+344974us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29438,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"thread_start_us":198422,"threads_started":2,"wal-append.queue_time_us":314201}
I20260709 12:26:02.447443  5869 rpcz_store.cc:276] 0709 12:26:00.422472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.433818 (+ 11346us) service_pool.cc:224] Handling call
0709 12:26:00.434133 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.824612 (+1390479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.111827 (+287215us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b540069ee54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:02.112230 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3194770304}
I20260709 12:26:02.445864  6018 rpcz_store.cc:276] 0709 12:26:01.399070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.412783 (+ 13713us) service_pool.cc:224] Handling call
0709 12:26:02.110820 (+698037us) spinlock_profiling.cc:243] Waited 62.3 ms on lock 0x7b540041da54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c7d853 00007f39d7c4b60a 00007f39d11d17d4 00007f39d11d1762 00007f39d11d1691 00007f39d11d160c 00007f39d11d08c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:02.431072 (+320252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143195904}
W20260709 12:26:02.075520  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.451524  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:02.452248  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 610) took 1858 ms. Trace:
I20260709 12:26:02.452823  6018 raft_consensus.cc:2410] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:02.452736  6022 rpcz_store.cc:276] 0709 12:26:00.593401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.721348 (+127947us) service_pool.cc:224] Handling call
0709 12:26:00.721725 (+   377us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:00.865545 (+143820us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.865574 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:00.878211 (+ 12637us) log.cc:844] Serialized 52 byte log entry
0709 12:26:01.311297 (+433086us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:01.311358 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.108927 (+797569us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540036f954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:02.431744 (+322817us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.431812 (+    68us) raft_consensus.cc:1685] finished
0709 12:26:02.431833 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.452176 (+ 20343us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7034,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":8326,"spinlock_wait_cycles":329732992,"thread_start_us":347450,"threads_started":1,"wal-append.queue_time_us":545260}
W20260709 12:26:02.457008  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.464453  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:02.475981  8904 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:02.476850  8904 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.477686  8980 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:02.478201  8944 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:02.478667  8980 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:02.480340  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:26:02.455026  8948 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:02.484323  8948 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:02.461871  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:02.487242  8948 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:02.488862  8959 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:02.434563  8404 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.489853  8404 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:02.492908  8404 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:02.485973  6018 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
W20260709 12:26:02.438920  8912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.555s	user 0.005s	sys 0.007s
W20260709 12:26:02.442754  8888 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.870s	user 0.018s	sys 0.008s
W20260709 12:26:02.487543  8938 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.222s	user 0.017s	sys 0.009s
W20260709 12:26:02.442353  8915 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.065s	user 0.027s	sys 0.002s
I20260709 12:26:02.507448  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:02.442811  8817 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:02.460458  8842 consensus_meta.cc:287] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.525s	user 0.015s	sys 0.002s
I20260709 12:26:02.523185  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:02.523487  8667 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:02.522439  8944 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:02.523675  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:02.524488  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 606) took 2364 ms. Trace:
I20260709 12:26:02.526688  6015 rpcz_store.cc:276] 0709 12:26:00.160408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.392022 (+231614us) service_pool.cc:224] Handling call
0709 12:26:00.392447 (+   425us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:00.392882 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.392909 (+    27us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:00.428316 (+ 35407us) log.cc:844] Serialized 78 byte log entry
0709 12:26:00.764058 (+335742us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:01.081937 (+317879us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.082662 (+   725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.082784 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.127651 (+1044867us) spinlock_profiling.cc:243] Waited 9.56 ms on lock 0x7b180025b460. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:26:02.524109 (+396458us) raft_consensus.cc:1685] finished
0709 12:26:02.524230 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.524418 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":66608,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":6471,"raft.queue_time_us":6503,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":4158,"spinlock_wait_cycles":21984384,"thread_start_us":593389,"threads_started":2,"wal-append.queue_time_us":474461}
I20260709 12:26:02.528848  8938 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:02.524632  8842 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:02.530699  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:02.531356  8938 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:02.523627  8951 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:26:02.535336  6762 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:26:02.540272  8944 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:02.542435  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:02.545557  8944 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:02.547281  8706 raft_consensus.cc:3055] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:02.548032  8706 raft_consensus.cc:740] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:02.548506  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:02.435390  8875 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:02.550019  8875 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:02.553853  8977 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:26:02.555771  8810 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:02.556130  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:02.557230  6762 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:02.567039  6609 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:02.570201  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:02.571427  6022 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:02.577327  8944 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:02.548638  8843 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.725s	user 0.018s	sys 0.000s
W20260709 12:26:02.565317  8957 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.012s	sys 0.028s
I20260709 12:26:02.514252  8980 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:02.576894  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.477404  8904 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:02.581103  8890 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:02.575661  6762 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:26:02.578521  8539 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:02.600651  8515 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.014s	sys 0.037s
I20260709 12:26:02.610520  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:02.620980  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.631906  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.459450  8937 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:02.637050  6768 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:02.633733  8904 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:02.638222  8904 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.638820  8904 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:02.633759  8915 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:02.643221  8915 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:02.652422  5948 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:02.654461  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 849) took 2071 ms. Trace:
W20260709 12:26:02.654780  5866 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 458) took 2027 ms (client timeout 2029 ms). Trace:
I20260709 12:26:02.654922  6024 rpcz_store.cc:276] 0709 12:26:00.582808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.621442 (+ 38634us) service_pool.cc:224] Handling call
0709 12:26:00.635917 (+ 14475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.147429 (+511512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.654190 (+1506761us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b5400171bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:02.654389 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1123084672}
I20260709 12:26:02.656350  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:02.607398  8843 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:02.649135  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 898) took 1921 ms. Trace:
I20260709 12:26:02.658813  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 468) took 1917 ms. Trace:
I20260709 12:26:02.657522  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:02.659448  5870 rpcz_store.cc:276] 0709 12:26:00.741712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.988970 (+247258us) service_pool.cc:224] Handling call
0709 12:26:01.014692 (+ 25722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.148234 (+133542us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:01.148283 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.148347 (+    64us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:01.148402 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.645607 (+1497205us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b5400193054. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:02.658514 (+ 12907us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.658733 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":63596,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"spinlock_wait_cycles":306107264}
I20260709 12:26:02.658420  6021 rpcz_store.cc:276] 0709 12:26:00.729256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.890741 (+161485us) service_pool.cc:224] Handling call
0709 12:26:01.350398 (+459657us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.372331 (+ 21933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.648810 (+1276479us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b540041e450. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:02.649042 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1055960320}
I20260709 12:26:02.662372  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:02.658888  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:02.662473  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 52108.6us
W20260709 12:26:02.655194  5866 rpcz_store.cc:269] 0709 12:26:00.627826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.628172 (+   346us) service_pool.cc:224] Handling call
0709 12:26:02.654589 (+2026417us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b54001be754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:02.654720 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61337344}
I20260709 12:26:02.615936  8849 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:02.671446  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 853) took 1365 ms. Trace:
W20260709 12:26:02.661401  8515 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.014s	sys 0.037s
I20260709 12:26:02.673136  6024 rpcz_store.cc:276] 0709 12:26:01.306369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.670527 (+1364158us) service_pool.cc:224] Handling call
0709 12:26:02.670886 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.671115 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.671141 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.671184 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:02.671220 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.671275 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.671385 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.628865  6022 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:26:02.671595  6021 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:26:02.661600  8987 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.677369  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 617) took 1336 ms. Trace:
I20260709 12:26:02.671537  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:02.579665  8706 consensus_queue.cc:260] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:02.664304  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 457) took 1774 ms. Trace:
I20260709 12:26:02.680445  6022 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:02.684574  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 619) took 1339 ms. Trace:
I20260709 12:26:02.685020  6022 rpcz_store.cc:276] 0709 12:26:01.344921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.678367 (+1333446us) service_pool.cc:224] Handling call
0709 12:26:02.679976 (+  1609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.684210 (+  4234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.684489 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.678158  6024 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:02.690697  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 618) took 1347 ms. Trace:
I20260709 12:26:02.681468  5948 rpcz_store.cc:276] 0709 12:26:00.879566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.651583 (+1772017us) service_pool.cc:224] Handling call
0709 12:26:02.652155 (+   572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.653274 (+  1119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.653570 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.681716  5915 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:02.696168  6024 rpcz_store.cc:276] 0709 12:26:01.343256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.675658 (+1332402us) service_pool.cc:224] Handling call
0709 12:26:02.677890 (+  2232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.690264 (+ 12374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.690614 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.672151  8871 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:02.675928  8890 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.723677  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 620) took 1241 ms. Trace:
I20260709 12:26:02.731052  6022 rpcz_store.cc:276] 0709 12:26:01.482458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.722417 (+1239959us) service_pool.cc:224] Handling call
0709 12:26:02.722927 (+   510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.723212 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.723258 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.723320 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:02.723365 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.723436 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.723598 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.674623  8810 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:02.733994  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:02.693418  8843 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.672748  8975 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:26:02.672585  8957 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:26:02.756145  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:02.680366  8849 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:02.702400  8706 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:02.702423  8539 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:02.681063  6021 rpcz_store.cc:276] 0709 12:26:01.340731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.670968 (+1330237us) service_pool.cc:224] Handling call
0709 12:26:02.671230 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.677000 (+  5770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.677298 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.707086  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:02.734540  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 622) took 1110 ms. Trace:
W20260709 12:26:02.803694  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 96757us
I20260709 12:26:02.803490  6022 rpcz_store.cc:276] 0709 12:26:01.623797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.732529 (+1108732us) service_pool.cc:224] Handling call
0709 12:26:02.733829 (+  1300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.734193 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.734222 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.734265 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:02.734294 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.734349 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.734465 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.757604  8878 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:02.757797  8811 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:02.705940  8871 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:26:02.771994  8967 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.753s	user 0.001s	sys 0.000s
I20260709 12:26:02.706655  8515 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:02.679299  8938 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:26:02.755437  8615 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:02.801262  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.695920  8915 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:02.735625  8980 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:02.815384  6022 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:02.691915  8987 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:02.818766  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 857) took 1194 ms. Trace:
I20260709 12:26:02.729394  8404 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 12:26:02.821571  6022 rpcz_store.cc:276] 0709 12:26:01.624321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.805198 (+1180877us) service_pool.cc:224] Handling call
0709 12:26:02.814909 (+  9711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.817960 (+  3051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.818693 (+   733us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.806697  8861 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.809801  8931 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:26:02.817101  8973 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.812s	user 0.000s	sys 0.003s
I20260709 12:26:02.737833  8890 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:02.813344  8871 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:02.814080  8515 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.799271  8843 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:02.816130  8811 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:02.818749  8904 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:02.824514  8539 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:02.853153  5942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 888) took 2388 ms (client timeout 1951 ms). Trace:
W20260709 12:26:02.856244  5942 rpcz_store.cc:269] 0709 12:26:00.458784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.459689 (+   905us) service_pool.cc:224] Handling call
0709 12:26:02.847243 (+2387554us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b5400188b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:02.847506 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920688384}
I20260709 12:26:02.843607  8515 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:02.820114  8615 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:02.838649  8931 raft_consensus.cc:697] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:02.824141  8980 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:02.842527  8915 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:02.876127  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.870456  8667 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.876120  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 616) took 1586 ms. Trace:
I20260709 12:26:02.855746  8849 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:02.832974  8861 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:02.862989  8904 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:02.879393  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:02.906100  6023 rpcz_store.cc:276] 0709 12:26:01.289984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.577889 (+287905us) service_pool.cc:224] Handling call
0709 12:26:01.578339 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.579035 (+   696us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.579072 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:01.579618 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 12:26:01.920057 (+340439us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:01.920108 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.920190 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.541585 (+621395us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b480050d258. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d7bacaa1 00007f39d7ba3b37 00007f39d7ba39ca 00007f39d7c9b26f 00007f39d7c9af37 00007f39d7caf55c 00007f39d7caf508 00007f39d7caf488 00007f39d7caf443 00007f39d7cae6ec 00007f39d6bd8a96
0709 12:26:02.773504 (+231919us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:02.875771 (+102267us) raft_consensus.cc:1685] finished
0709 12:26:02.875805 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.876032 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":734,"raft.run_cpu_time_us":3231,"raft.run_wall_time_us":960034,"spinlock_wait_cycles":35450112,"thread_start_us":339876,"threads_started":1,"wal-append.queue_time_us":417665}
W20260709 12:26:02.908169  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 236699us
W20260709 12:26:02.906752  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.882618  5943 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:02.885561  5947 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:02.887126  6016 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:26:02.887452  6020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 897) took 2187 ms (client timeout 1641 ms). Trace:
I20260709 12:26:02.911692  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 456) took 2041 ms. Trace:
W20260709 12:26:02.911814  6020 rpcz_store.cc:269] 0709 12:26:00.683513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.715486 (+ 31973us) service_pool.cc:224] Handling call
0709 12:26:02.870792 (+2155306us) spinlock_profiling.cc:243] Waited 92.9 ms on lock 0x7b540041e954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:02.870982 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":213756160}
I20260709 12:26:02.875058  8890 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:02.888756  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1091) took 2154 ms. Trace:
I20260709 12:26:02.918195  5874 rpcz_store.cc:276] 0709 12:26:00.734691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.974148 (+239457us) service_pool.cc:224] Handling call
0709 12:26:01.238204 (+264056us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.246331 (+  8127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.888379 (+1642048us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54007350d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:02.888636 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606225280}
I20260709 12:26:02.889262  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:02.879748  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:02.908099  5948 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 4.
I20260709 12:26:02.912261  5947 rpcz_store.cc:276] 0709 12:26:00.870394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.884827 (+2014433us) service_pool.cc:224] Handling call
0709 12:26:02.885293 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.911371 (+ 26078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.911615 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.886085  8769 raft_consensus.cc:3055] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:02.932479  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 623) took 1276 ms. Trace:
I20260709 12:26:02.892984  8843 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:02.891295  8987 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:02.887980  8721 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:02.913524  8988 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:02.937791  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.928369  8975 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:02.916298  8989 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.933688  6020 rpcz_store.cc:276] 0709 12:26:01.655723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.913198 (+1257475us) service_pool.cc:224] Handling call
0709 12:26:02.931650 (+ 18452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.931949 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.931995 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.932063 (+    68us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:02.932114 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.932197 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.932387 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.913770  8708 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.904516  8915 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:02.896620  8875 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:02.914508  8667 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:02.930387  8769 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:02.883176  8811 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:02.969173  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:02.877590  8931 consensus_queue.cc:237] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:02.971698  5874 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:26:02.971809  5943 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:02.942924  8993 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:02.967458  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 161991us
W20260709 12:26:02.964236  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 205049us
I20260709 12:26:02.921797  8904 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:02.966006  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:02.953619  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 860) took 1178 ms. Trace:
I20260709 12:26:02.978485  8615 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:02.977761  6020 rpcz_store.cc:276] 0709 12:26:01.775004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.952204 (+1177200us) service_pool.cc:224] Handling call
0709 12:26:02.952777 (+   573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.953051 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.953095 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.953168 (+    73us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:02.953219 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.953305 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.953525 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.974953  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:02.979136  5871 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:02.962282  8708 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:02.983242  8904 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:02.977963  8769 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:02.985140  8769 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:02.985503  8667 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:02.987035  8915 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:02.973981  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:02.992587  6020 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:02.993026  8861 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:02.996608  8708 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:03.000538  8980 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:02.949499  8988 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:03.002795  8991 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.003679  8988 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:03.003365  8991 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:03.001569  8987 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:03.015888  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:03.021867  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:03.025053  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:03.026042  6768 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:03.054140  5942 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:26:03.058406  5869 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
W20260709 12:26:03.062754  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.066440  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 621) took 1443 ms. Trace:
I20260709 12:26:03.068748  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:03.070794  6024 rpcz_store.cc:276] 0709 12:26:01.623521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.731368 (+1107847us) service_pool.cc:224] Handling call
0709 12:26:02.731856 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.732297 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:02.938370 (+206073us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.066000 (+127630us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:03.066064 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:03.066124 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.066197 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.066359 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":243867,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":116,"thread_start_us":305690,"threads_started":2}
I20260709 12:26:03.097383  6020 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:03.013504  8980 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:03.050374  8615 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:03.099736  6019 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:03.108919  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:03.061178  8827 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.103s	user 0.048s	sys 0.021s
I20260709 12:26:03.072058  8912 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:03.092147  8991 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:02.962494  8539 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:03.050719  8987 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:02.952282  8989 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:03.114071  8827 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.165s	user 0.049s	sys 0.022s
I20260709 12:26:03.012302  8993 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:03.152801  8871 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:03.155524  8871 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:03.156689  8875 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:03.167263  8801 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:03.149431  8993 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:03.116528  8980 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:03.124498  8615 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:03.175087  8817 consensus_meta.cc:287] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.598s	user 0.014s	sys 0.003s
I20260709 12:26:03.135203  8912 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:03.192381  8912 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:03.291370  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 380370us
I20260709 12:26:03.442160  8936 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.443439  8936 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:03.445204  8644 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.445785  9006 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:03.446559  8644 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:03.448709  8988 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:03.449252  8988 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:03.300877  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.087622  5941 raft_consensus.cc:2410] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:03.070596  5943 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:03.320650  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 859) took 1695 ms. Trace:
I20260709 12:26:03.104633  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:03.323065  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 856) took 1698 ms. Trace:
I20260709 12:26:03.343327  6609 leader_election.cc:400] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9.
I20260709 12:26:03.345557  8147 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:03.349851  8708 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:03.217006  8980 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:03.218550  8987 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:03.182159  8989 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:03.405663  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.497730  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.498157  6019 raft_consensus.cc:2410] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:26:03.498409  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.472223  8147 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:03.488204  8615 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:03.501583  9020 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.412457  8979 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:26:03.502866  9020 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:03.413151  9014 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:03.430884  8667 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:03.440267  8875 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:03.450233  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.440267  8904 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:03.514252  8904 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:03.517184  8904 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:03.491739  8389 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:03.518639  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:03.519270  8389 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:03.444591  8915 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
W20260709 12:26:03.297550  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.349036  8515 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:03.412567  9013 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:26:03.470942  6609 leader_election.cc:403] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:03.441309  8890 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:03.470703  6021 rpcz_store.cc:276] 0709 12:26:01.624072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.803017 (+1178945us) service_pool.cc:224] Handling call
0709 12:26:02.820620 (+ 17603us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.821049 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:02.964166 (+143117us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.067485 (+103319us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.069173 (+  1688us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.069346 (+   173us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:03.069588 (+   242us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.069834 (+   246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.069913 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.322341 (+252428us) raft_consensus.cc:1685] finished
0709 12:26:03.322402 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.322808 (+   406us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8576,"prepare.queue_time_us":191490,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":100,"raft.queue_time_us":5326,"raft.run_cpu_time_us":447,"raft.run_wall_time_us":13153,"thread_start_us":197917,"threads_started":2,"wal-append.queue_time_us":187557}
I20260709 12:26:03.484434  6775 raft_consensus.cc:2410] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:26:03.484840  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 66205.7us
I20260709 12:26:03.487192  5838 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:03.464834  6023 rpcz_store.cc:276] 0709 12:26:01.624868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.909000 (+1284132us) service_pool.cc:224] Handling call
0709 12:26:02.909454 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.926835 (+ 17381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.926885 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:02.934539 (+  7654us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.065403 (+130864us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:03.065461 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.065549 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.319669 (+254120us) raft_consensus.cc:1685] finished
0709 12:26:03.319731 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.320233 (+   502us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10706,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":8983,"thread_start_us":102068,"threads_started":1,"wal-append.queue_time_us":220997}
I20260709 12:26:03.437979  8721 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:03.491649  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.491253  8708 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:03.495087  9006 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:03.510494  9014 consensus_queue.cc:1243] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:03.510706  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:03.511374  8667 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:03.512322  5870 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:03.512622  8875 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:03.484263  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:03.527990  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.491282  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:03.501022  8147 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:03.537616  6021 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.538646  8996 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.539008  8708 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:03.539625  8996 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:03.542102  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 199552us
I20260709 12:26:03.543614  8515 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:03.546083  5871 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:03.547919  8890 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:03.550274  8950 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.549147  8960 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9.
I20260709 12:26:03.551213  8950 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:03.552580  8959 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.553521  8959 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:03.556278  8959 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:03.556806  8950 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:03.557322  5866 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:26:03.557976  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:03.558991  6757 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:03.559336  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:03.560168  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 730) took 1027 ms. Trace:
I20260709 12:26:03.559374  8936 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:03.551216  8890 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:03.553143  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:03.563488  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:03.564224  5866 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:03.564301  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 627) took 1060 ms. Trace:
I20260709 12:26:03.544919  8721 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:03.566216  8988 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:03.567106  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 612) took 2281 ms. Trace:
I20260709 12:26:03.568248  8875 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:03.569123  5944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 444) took 2946 ms (client timeout 2029 ms). Trace:
W20260709 12:26:03.569386  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.577732  5944 rpcz_store.cc:269] 0709 12:26:00.622226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.818124 (+195898us) service_pool.cc:224] Handling call
0709 12:26:03.568600 (+2750476us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b5400674654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:03.568860 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948322048}
I20260709 12:26:03.579663  6762 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:26:03.571097  5866 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:26:03.527276  8644 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:03.570353  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 858) took 1909 ms. Trace:
I20260709 12:26:03.493463  8842 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:03.582783  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:03.560655  6775 rpcz_store.cc:276] 0709 12:26:02.529953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.530339 (+   386us) service_pool.cc:224] Handling call
0709 12:26:03.556549 (+1026210us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b540052a154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:03.556769 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1889271552}
I20260709 12:26:03.567854  8988 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:03.564649  5915 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:03.552497  8667 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:03.567735  6016 rpcz_store.cc:276] 0709 12:26:01.285724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.304266 (+ 18542us) service_pool.cc:224] Handling call
0709 12:26:02.871234 (+1566968us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.911453 (+ 40219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.566601 (+655148us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b540041e950. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:03.566898 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3524008192}
I20260709 12:26:03.565052  6020 rpcz_store.cc:276] 0709 12:26:02.503417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.999154 (+495737us) service_pool.cc:224] Handling call
0709 12:26:02.999661 (+   507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.103329 (+103668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.563689 (+460360us) spinlock_profiling.cc:243] Waited 97.4 ms on lock 0x7b54001612d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:03.564112 (+   423us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223932800}
I20260709 12:26:03.570025  8936 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:03.608155  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:26:03.604763  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 117903us
I20260709 12:26:03.609550  5944 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.610821  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:03.609740  6766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 724) took 2241 ms (client timeout 1954 ms). Trace:
I20260709 12:26:03.601509  8515 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:03.628762  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:03.610461  8936 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:03.622135  6766 rpcz_store.cc:269] 0709 12:26:01.339397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.352249 (+ 12852us) service_pool.cc:224] Handling call
0709 12:26:03.580525 (+2228276us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b540004d8d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:03.580716 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":997610240}
I20260709 12:26:03.610332  8147 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:03.616243  8915 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:03.642699  5946 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:26:03.645656  6024 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
W20260709 12:26:03.646322  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 81831.3us
W20260709 12:26:03.648053  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 54315.7us
I20260709 12:26:03.601128  8813 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:03.584211  6022 rpcz_store.cc:276] 0709 12:26:01.624593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.823618 (+1199025us) service_pool.cc:224] Handling call
0709 12:26:02.837472 (+ 13854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.837926 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:03.320930 (+483004us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.472020 (+151090us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:03.478307 (+  6287us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:03.526098 (+ 47791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.531594 (+  5496us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.534221 (+  2627us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4266,"prepare.queue_time_us":156299,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":38740,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":4390,"thread_start_us":572417,"threads_started":2,"wal-append.queue_time_us":140974}
W20260709 12:26:03.640192  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 82267.4us
I20260709 12:26:03.625254  8904 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:03.683887  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:03.640825  8915 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.639176  8936 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:03.695436  8515 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:03.697089  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:03.697355  9020 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:03.698189  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:03.699232  8721 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:03.699523  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 208768us
W20260709 12:26:03.693037  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 56958.1us
I20260709 12:26:03.709759  8915 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:03.712090  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:03.712391  8931 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:03.693261  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 45826us
W20260709 12:26:03.695495  8945 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.601s	user 0.011s	sys 0.012s
I20260709 12:26:03.719704  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:03.723074  8888 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:03.724043  8743 consensus_queue.cc:1059] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:03.724768  5913 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:03.707334  6611 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:03.681373  8875 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.737502  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:03.737843  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:03.739051  8848 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:03.746279  6762 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:03.747654  6021 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 12:26:03.751448  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.774662  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1092) took 3039 ms. Trace:
I20260709 12:26:03.823346  5873 rpcz_store.cc:276] 0709 12:26:00.734907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.916777 (+2181870us) service_pool.cc:224] Handling call
0709 12:26:02.936596 (+ 19819us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.002587 (+ 65991us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.002645 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:03.002702 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:03.002743 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.774182 (+771439us) spinlock_profiling.cc:243] Waited 65.6 ms on lock 0x7b5400027954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:03.774248 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.774474 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150945664}
I20260709 12:26:03.756053  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:03.825799  5942 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:03.826824  5949 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:26:03.779033  8931 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.830029  8931 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:03.746325  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:03.835553  9020 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:03.836115  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 89989.6us
I20260709 12:26:03.776850  8708 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:03.838444  8708 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:03.839747  8708 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:03.843600  8996 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:03.780462  8945 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.775s	user 0.012s	sys 0.012s
I20260709 12:26:03.841423  8708 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:03.846048  5838 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:03.837363  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:03.749526  8974 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 1.745s	user 0.005s	sys 0.005s
I20260709 12:26:03.770279  8842 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:03.768567  8721 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.771543  5990 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:03.755419  8515 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:03.764484  8937 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:03.768932  8668 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:03.845031  8389 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:03.868173  9035 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 12:26:03.873464  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 166580us
I20260709 12:26:03.819106  8813 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:03.875780  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 58359.8us
I20260709 12:26:03.877333  8842 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:03.881587  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 110528us
I20260709 12:26:03.868438  8950 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:03.882782  5990 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:03.882899  8950 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:03.884249  8878 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:03.884636  8980 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:03.885506  8615 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.886397  8615 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:03.887482  8886 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:03.888139  8886 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:03.889834  8957 raft_consensus.cc:3055] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:03.890476  8957 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:03.891223  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:03.892174  5941 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:03.891803  8957 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:03.881975  8888 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:03.902783  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:03.903295  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:03.904567  6019 raft_consensus.cc:2410] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:03.904685  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:03.905498  6021 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:03.905546  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:03.906950  5874 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:03.860513  8875 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:03.911876  8813 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:03.875983  9034 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.912469  8875 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:03.916086  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.860072  8721 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:03.920747  9034 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:03.922083  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:03.924149  9034 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:03.925377  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:03.864948  8996 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:26:03.927119  6020 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:03.921231  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:03.928869  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 88426.1us
I20260709 12:26:03.921986  8842 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:03.929262  6017 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:03.928332  8996 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:03.934666  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:03.936514  5865 pending_rounds.cc:85] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 4
I20260709 12:26:03.937058  5873 pending_rounds.cc:85] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 3
I20260709 12:26:03.937482  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:03.924273  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:03.902839  8944 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:03.945492  5872 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:03.946178  5867 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:03.947347  5942 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:03.948236  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:03.949010  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:03.950764  5948 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:26:03.940320  5865 pending_rounds.cc:107] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:26:03.953415  5865 raft_consensus.cc:2889] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:03.955458  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 725) took 2615 ms. Trace:
I20260709 12:26:03.941025  5873 pending_rounds.cc:107] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:26:03.897914  8888 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:03.959496  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 76885.4us
I20260709 12:26:03.956266  5873 raft_consensus.cc:2889] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:03.927201  8950 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:03.966256  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.975790  5873 pending_rounds.cc:107] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:26:03.976425  5873 raft_consensus.cc:2889] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:03.969718  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:03.944770  8871 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:03.990767  5944 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
W20260709 12:26:03.994731  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 64327.1us
I20260709 12:26:03.995200  5946 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:03.955919  8944 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.940083  8996 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:03.937565  5697 catalog_manager.cc:5697] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:03.959215  8888 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:04.006883  5871 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:03.940770  8937 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:26:03.889459  8886 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:04.012784  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.001328  8944 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:04.017427  5838 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:04.034427  6020 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:04.028133  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:04.051491  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 47371.7us
W20260709 12:26:04.057210  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 45346.2us
W20260709 12:26:04.058009  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 333546us
I20260709 12:26:04.017648  8615 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:03.956059  6773 rpcz_store.cc:276] 0709 12:26:01.339684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.368488 (+ 28804us) service_pool.cc:224] Handling call
0709 12:26:01.398402 (+ 29914us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.443454 (+ 45052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.443507 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:01.457187 (+ 13680us) log.cc:844] Serialized 26 byte log entry
0709 12:26:01.921906 (+464719us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:02.410745 (+488839us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.433567 (+ 22822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.433731 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.748147 (+1314416us) spinlock_profiling.cc:243] Waited 9.62 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:03.954776 (+206629us) raft_consensus.cc:1685] finished
0709 12:26:03.954915 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.955193 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57595,"prepare.queue_time_us":616482,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":47717,"raft.run_cpu_time_us":8867,"raft.run_wall_time_us":53318,"spinlock_wait_cycles":22127360,"thread_start_us":861292,"threads_started":2,"wal-append.queue_time_us":518439}
W20260709 12:26:04.056981  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 56085.8us
I20260709 12:26:04.057364  9020 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:04.074606  8979 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:04.075470  9020 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.076079  9020 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:04.077121  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:04.079576  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:04.080418  5913 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:04.081739  8515 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:04.082008  8912 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:04.082242  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 90818.8us
I20260709 12:26:04.082777  8515 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:04.083253  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 88519.5us
I20260709 12:26:04.084897  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:04.085953  8615 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:04.088972  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.088801  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.089792  6766 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:26:04.089001  8915 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:04.090726  9040 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:26:04.092700  8915 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:04.101821  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.089814  8940 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.918s	user 0.012s	sys 0.021s
W20260709 12:26:04.110342  8940 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.098s	user 0.012s	sys 0.021s
I20260709 12:26:04.117694  8948 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:04.122184  8706 consensus_meta.cc:287] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.316s	user 0.011s	sys 0.018s
W20260709 12:26:04.126582  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.047905  8994 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:04.128335  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 75387.2us
I20260709 12:26:04.129834  8948 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:04.130354  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 50099.9us
I20260709 12:26:04.130205  8539 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:26:04.131101  8957 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:04.149395  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 1101) took 1175 ms. Trace:
I20260709 12:26:04.154227  5869 rpcz_store.cc:276] 0709 12:26:02.973543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.973690 (+   147us) service_pool.cc:224] Handling call
0709 12:26:04.148956 (+1175266us) spinlock_profiling.cc:243] Waited 68 ms on lock 0x7b54002f99d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:04.149172 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156391296}
I20260709 12:26:04.157599  8936 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:04.159132  5943 raft_consensus.cc:2368] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
I20260709 12:26:04.159930  8936 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:04.159785  8948 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:04.163728  5913 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:04.159663  8976 raft_consensus.cc:3055] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:04.168203  8976 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:04.170787  6793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 718) took 3424 ms (client timeout 2067 ms). Trace:
W20260709 12:26:04.171382  6793 rpcz_store.cc:269] 0709 12:26:00.746757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.872873 (+126116us) service_pool.cc:224] Handling call
0709 12:26:04.170359 (+3297486us) spinlock_profiling.cc:243] Waited 2.93 s on lock 0x7b5400005854. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:04.170588 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6738259328}
I20260709 12:26:04.174074  8996 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:04.176120  8996 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:04.181411  5874 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:26:04.197835  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 110222us
I20260709 12:26:04.166823  8668 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:04.164472  8953 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:26:04.231298  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.232856  6762 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:04.233263  6757 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:04.235070  8706 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:04.235497  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.233268  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:04.237862  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.238580  9034 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:04.239346  9034 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:04.240294  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.210177  8668 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.242512  8668 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:04.241840  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 44522.1us
I20260709 12:26:04.243409  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:04.246012  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:04.246888  8915 raft_consensus.cc:2764] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:26:04.249405  8817 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:04.250000  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.252872  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:04.254508  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.255414  6762 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:04.259473  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.250171  8817 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:04.259200  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:04.267469  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.269033  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:04.270465  8994 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:04.271607  8915 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:04.215857  8948 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 12:26:04.272090  8915 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:04.272238  8948 consensus_queue.cc:1243] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:04.273345  8915 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:04.275347  8915 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:04.276547  8615 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:04.278878  8904 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:04.279891  8848 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:04.280529  8994 raft_consensus.cc:3055] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:04.281113  8994 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:04.267526  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:04.302028  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.165553  8743 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:26:04.305423  8945 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:04.307250  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:04.308425  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:04.308624  9034 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:04.309180  5940 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:04.309836  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:04.308720  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:04.310530  5869 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:04.311234  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.282389  8994 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:04.170205  8976 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:04.268697  6768 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:04.316721  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:04.329130  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 186082us
W20260709 12:26:04.339810  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.376369  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.263208  8817 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:04.378556  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 54022.7us
I20260709 12:26:04.381382  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.381507  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:04.382328  9044 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.376s	user 0.005s	sys 0.001s
I20260709 12:26:04.383306  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:04.383574  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:04.389369  5944 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:04.383308  5940 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:04.382582  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:04.385591  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.382485  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:04.383551  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 298811us
I20260709 12:26:04.384431  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:04.394951  5868 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:04.386430  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.402241  5944 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:04.403962  5942 raft_consensus.cc:2410] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:04.406220  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:04.407691  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:04.407264  5697 catalog_manager.cc:5697] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:04.412547  6766 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:04.409142  5697 catalog_manager.cc:5697] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 4 to 9, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 9 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:04.413009  6766 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:04.393128  8708 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.537s	user 0.014s	sys 0.000s
I20260709 12:26:04.420550  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:04.429349  6778 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:04.421950  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:04.422907  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.423142  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.424310  9013 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:04.427075  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.427680  5865 pending_rounds.cc:85] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 3
I20260709 12:26:04.434146  5865 pending_rounds.cc:107] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:26:04.429510  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.429647  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.391022  8994 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:04.413571  8931 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.583s	user 0.014s	sys 0.010s
I20260709 12:26:04.434444  5865 raft_consensus.cc:2889] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:04.424975  6766 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:04.398036  8721 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:04.444751  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:04.446101  5947 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:26:04.446832  8950 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:04.446844  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:04.442988  6762 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:04.448370  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:04.454041  5872 raft_consensus.cc:2410] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:26:04.433833  9013 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:04.465751  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 81338.6us
I20260709 12:26:04.454308  6773 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 4.
I20260709 12:26:04.442584  8743 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.469851  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:04.497957  8743 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:04.498728  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 64782.1us
I20260709 12:26:04.499853  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:04.485330  5696 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40812 (request call id 141) took 27587 ms (client timeout 15000 ms). Trace:
I20260709 12:26:04.488090  5945 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:04.500695  8743 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:04.501037  8965 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.502147  6766 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:04.501834  8965 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:04.458464  8976 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:04.505991  5697 catalog_manager.cc:5697] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 3 to 6, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 6 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:04.508260  8965 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:04.489660  8950 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:04.509668  8950 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:04.505271  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.511035  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:04.512518  8950 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:04.513373  8708 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:04.514307  8708 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:04.516973  5872 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:04.517462  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:04.518450  8915 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:04.520956  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:04.452270  8931 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:04.500759  5696 rpcz_store.cc:269] 0709 12:25:36.897928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.994216 (+ 96288us) service_pool.cc:224] Handling call
0709 12:26:04.484975 (+27490759us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:55.719316 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:55.720631 (+  1315us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:55.720703 (+    72us) write_op.cc:435] Acquired schema lock
0709 12:25:55.720735 (+    32us) tablet.cc:662] Decoding operations
0709 12:25:55.727217 (+  6482us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:55.727344 (+   127us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:55.727398 (+    54us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:25:55.863436 (+136038us) tablet.cc:701] Row locks acquired
0709 12:25:55.863498 (+    62us) write_op.cc:260] PREPARE: finished
0709 12:25:55.863731 (+   233us) write_op.cc:270] Start()
0709 12:25:55.863932 (+   201us) write_op.cc:276] Timestamp: P: 1783599955863654 usec, L: 0
0709 12:25:55.863958 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:25:55.864873 (+   915us) log.cc:844] Serialized 5501 byte log entry
0709 12:25:58.249003 (+2384130us) op_driver.cc:464] REPLICATION: finished
0709 12:25:59.253594 (+1004591us) write_op.cc:301] APPLY: starting
0709 12:25:59.290535 (+ 36941us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:59.396069 (+105534us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:59.396264 (+   195us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:59.398246 (+  1982us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:59.398610 (+   364us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 12:25:59.398672 (+    62us) write_op.cc:312] APPLY: finished
0709 12:25:59.399152 (+   480us) log.cc:844] Serialized 100 byte log entry
0709 12:25:59.687863 (+288711us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:59.714578 (+ 26715us) write_op.cc:454] Released schema lock
0709 12:25:59.732598 (+ 18020us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:59.745990 (+ 13392us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:59.746043 (+    53us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":66797,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":902971,"apply.run_cpu_time_us":25474,"apply.run_wall_time_us":544127,"num_ops":14,"prepare.queue_time_us":725344,"prepare.run_cpu_time_us":18152,"prepare.run_wall_time_us":294055,"raft.queue_time_us":895714,"raft.run_cpu_time_us":34696,"raft.run_wall_time_us":787875,"replication_time_us":2302009,"thread_start_us":1838681,"threads_started":5,"wal-append.queue_time_us":672043,"wal-append.run_cpu_time_us":33970,"wal-append.run_wall_time_us":4048477}]]}
I20260709 12:26:04.456426  8721 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:04.522873  5870 pending_rounds.cc:85] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 3
W20260709 12:26:04.523851  5945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 441) took 3903 ms (client timeout 1684 ms). Trace:
I20260709 12:26:04.526145  5870 pending_rounds.cc:107] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:26:04.526532  5870 raft_consensus.cc:2889] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:04.510004  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:04.526240  5945 rpcz_store.cc:269] 0709 12:26:00.620002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.620490 (+   488us) service_pool.cc:224] Handling call
0709 12:26:04.523622 (+3903132us) spinlock_profiling.cc:243] Waited 3.45 s on lock 0x7b5400530f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:04.523772 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7941691776}
I20260709 12:26:04.509083  5697 catalog_manager.cc:5697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:26:04.534782  8721 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.545472  8721 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:04.548175  8915 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:04.470963  8615 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 12:26:04.551829  5949 raft_consensus.cc:2368] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11.
I20260709 12:26:04.555107  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:04.556754  8948 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:04.476063  8953 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:04.556794  8965 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:04.597110  5697 catalog_manager.cc:5697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 3 to 4, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 4 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:04.606017  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.606585  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.607717  5868 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:04.599685  5838 leader_election.cc:400] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11.
I20260709 12:26:04.610958  5838 leader_election.cc:403] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:04.612449  8988 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:26:04.615489  8953 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:04.616070  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:04.618968  8708 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:04.619742  8708 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:04.620263  8708 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:04.541101  8389 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.049s
W20260709 12:26:04.625221  5948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.020s	sys 0.014s
W20260709 12:26:04.628933  5948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.021s	sys 0.014s
I20260709 12:26:04.583165  8931 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:04.616796  8953 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:04.635515  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.637440  6762 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:04.657567  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:04.703459  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:04.704912  5868 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
W20260709 12:26:04.709420  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 52009.9us
I20260709 12:26:04.711800  5697 catalog_manager.cc:5697] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 5 to 7, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 7 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:04.713804  8931 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:26:04.716933  8944 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.657s	user 0.006s	sys 0.006s
I20260709 12:26:04.718020  8950 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.718648  8950 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:04.722764  5697 catalog_manager.cc:5697] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:04.670715  9013 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:04.659756  8615 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:04.725925  8615 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:04.727746  8931 raft_consensus.cc:697] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:04.728720  8615 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:04.729017  8988 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11.
I20260709 12:26:04.720817  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:04.726203  5697 catalog_manager.cc:5697] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:04.734122  5913 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:04.739183  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:04.729205  8931 consensus_queue.cc:237] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:04.745927  8948 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:04.726202  9013 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:04.746547  8948 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.748191  8948 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:04.746570  8950 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:04.753103  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.766460  8996 consensus_meta.cc:287] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.590s	user 0.002s	sys 0.030s
W20260709 12:26:04.772734  8668 consensus_meta.cc:287] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.530s	user 0.016s	sys 0.005s
I20260709 12:26:04.781802  6775 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:04.785337  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:04.717573  8944 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:04.789620  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.773847  8389 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:04.790980  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:04.771032  8667 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.795382  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:04.789717  8936 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
W20260709 12:26:04.782122  8888 consensus_meta.cc:287] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.746s	user 0.015s	sys 0.001s
I20260709 12:26:04.791402  5697 catalog_manager.cc:5697] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:04.794061  9020 consensus_meta.cc:287] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.715s	user 0.022s	sys 0.001s
I20260709 12:26:04.820807  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:04.816839  8667 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:04.823516  6793 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:04.825160  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:04.799649  8706 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:04.803462  8976 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:04.809523  8936 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:04.834408  5874 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:04.787851  8668 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:04.848592  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:04.837836  5697 catalog_manager.cc:5697] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 5 to 7, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 7 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:04.825042  8667 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:04.830497  8706 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.808727  8888 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:04.853183  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:04.850373  6017 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:04.859567  6766 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:26:04.838274  8976 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.868432  5697 catalog_manager.cc:5697] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:26:04.874185  5943 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:26:04.875061  6609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.005s	sys 0.022s
W20260709 12:26:04.875684  6609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.005s	sys 0.022s
I20260709 12:26:04.819281  9020 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:04.844329  8936 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:04.880365  6020 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:26:04.884135  5697 catalog_manager.cc:5697] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 7, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 7 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
W20260709 12:26:04.864629  6611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.009s	sys 0.017s
W20260709 12:26:04.887429  6611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.010s	sys 0.017s
I20260709 12:26:04.897704  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:04.888638  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 607911us
I20260709 12:26:04.872398  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:04.865706  8706 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:04.906108  5946 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:04.839823  8721 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:04.878687  8976 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:04.914568  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:04.918462  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:04.918669  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46421.3us
I20260709 12:26:04.918965  6775 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:04.915537  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:04.883548  8871 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:26:04.918511  5697 catalog_manager.cc:5697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 7, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 7 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:04.920032  6762 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:04.948973  6766 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:04.894905  8936 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:04.945343  6757 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:04.945405  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.925438  5866 raft_consensus.cc:2393] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 9: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:04.950547  5697 catalog_manager.cc:5697] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:04.916468  9034 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:04.967795  5990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.005s	sys 0.032s
W20260709 12:26:04.969988  5990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.006s	sys 0.033s
I20260709 12:26:04.919272  8940 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:04.972945  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 525830us
I20260709 12:26:04.986169  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:04.989418  9034 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:04.975302  8940 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.998255  8516 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:04.999096  8516 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:05.003054  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:05.003364  5913 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:05.005568  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.989077  5987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.017s	sys 0.024s
W20260709 12:26:05.013278  5987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.018s	sys 0.024s
I20260709 12:26:05.016040  6757 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
W20260709 12:26:05.019790  5869 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.007s	sys 0.031s
W20260709 12:26:05.022333  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 601802us
I20260709 12:26:04.919286  8948 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:04.998281  8940 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:05.020931  5697 catalog_manager.cc:5697] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:04.966415  8708 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:05.039258  5869 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.008s	sys 0.031s
W20260709 12:26:05.040000  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.981333  5868 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:05.062714  5874 raft_consensus.cc:3055] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:05.063396  5874 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:05.064585  5874 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:05.058485  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:05.102890  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:05.104620  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 51096.6us
I20260709 12:26:05.015164  8945 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:26:05.057233  8944 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:05.060910  8667 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:26:05.027100  8886 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.012s	sys 0.014s
I20260709 12:26:05.106294  8957 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
W20260709 12:26:05.121794  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.124804  5697 catalog_manager.cc:5697] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
W20260709 12:26:05.136282  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 167915us
I20260709 12:26:05.110387  8915 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.142107  5990 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:05.166383  5697 catalog_manager.cc:5697] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 1 to 6, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 6 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:05.166430  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 113137us
W20260709 12:26:05.129253  5941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.029s	sys 0.025s
W20260709 12:26:05.169948  5941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.029s	sys 0.025s
I20260709 12:26:05.176440  5874 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:05.183986  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 107368us
W20260709 12:26:05.192209  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.130688  8668 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:05.218673  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:05.132093  8886 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.012s	sys 0.015s
I20260709 12:26:05.217684  5697 catalog_manager.cc:5697] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 1 to 3, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 3 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:05.138046  9034 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:05.166939  8950 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:05.186993  8389 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:05.186394  8888 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:05.253414  5697 catalog_manager.cc:5697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:05.222755  8842 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:05.186857  9020 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:05.224776  8721 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:05.255389  5872 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.022s	sys 0.021s
W20260709 12:26:05.258174  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 116118us
W20260709 12:26:05.278049  5872 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.022s	sys 0.022s
I20260709 12:26:05.278992  5990 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:05.280540  5990 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:05.243868  5840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.047s	sys 0.053s
W20260709 12:26:05.249748  5870 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.008s	sys 0.032s
W20260709 12:26:05.281980  5870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.008s	sys 0.032s
W20260709 12:26:05.282712  5840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.048s	sys 0.053s
I20260709 12:26:05.277467  5987 leader_election.cc:400] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
W20260709 12:26:05.228137  8953 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.010s	sys 0.021s
I20260709 12:26:05.239432  8615 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:05.245138  8950 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:05.188673  8915 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:05.293501  8953 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.011s	sys 0.021s
W20260709 12:26:05.360116  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 744014us
I20260709 12:26:05.246659  8667 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 12:26:05.268584  8539 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 12:26:05.314728  8950 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:05.381919  6015 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:05.403651  9020 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.404551  9020 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:05.407265  9020 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:05.410519  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:05.401439  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.411441  8663 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
W20260709 12:26:05.414714  9013 consensus_meta.cc:287] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.667s	user 0.014s	sys 0.000s
I20260709 12:26:05.411468  6018 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:05.357689  8842 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:05.357474  8721 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:05.424510  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:05.426692  6016 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 10: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:26:05.441546  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.453019  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 204718us
W20260709 12:26:05.424486  8743 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.009s	sys 0.031s
I20260709 12:26:05.372339  8615 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
W20260709 12:26:05.448446  5867 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.027s	sys 0.017s
W20260709 12:26:05.468501  5867 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.027s	sys 0.017s
I20260709 12:26:05.419960  8944 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:05.432081  8842 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.428059  8708 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:05.480751  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:05.487602  5945 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:05.473855  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:05.465837  8743 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.010s	sys 0.031s
I20260709 12:26:05.483614  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:05.432408  8721 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:05.438202  8948 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:05.495103  5868 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:26:05.501619  5874 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:05.494328  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:05.502377  5869 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:05.502926  8721 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:05.497049  5868 pending_rounds.cc:85] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 4
I20260709 12:26:05.556456  5868 pending_rounds.cc:107] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:26:05.556768  5868 raft_consensus.cc:2889] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:05.530812  9013 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.547575  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:05.526049  5947 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:05.516546  9064 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:05.526961  6021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.010s	sys 0.046s
W20260709 12:26:05.562613  6021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.010s	sys 0.046s
W20260709 12:26:05.529842  5871 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.003s	sys 0.019s
W20260709 12:26:05.532102  6022 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.019s	sys 0.003s
W20260709 12:26:05.564002  6022 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.020s	sys 0.003s
W20260709 12:26:05.535008  5913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.010s	sys 0.043s
W20260709 12:26:05.540623  5865 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.019s	sys 0.042s
W20260709 12:26:05.565162  5865 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.019s	sys 0.042s
W20260709 12:26:05.566340  5913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.010s	sys 0.044s
W20260709 12:26:05.566840  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 559640us
I20260709 12:26:05.487567  8842 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:05.508121  8944 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:05.552367  8708 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.538398  8945 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:05.559851  6018 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
W20260709 12:26:05.563606  5871 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.004s	sys 0.019s
W20260709 12:26:05.538147  5873 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.000s	sys 0.022s
W20260709 12:26:05.587437  5873 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.000s	sys 0.023s
I20260709 12:26:05.577044  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:05.581269  5838 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:05.578161  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:05.589274  6016 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:05.593170  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:05.592227  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:05.592235  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:05.604884  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:05.606491  5869 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
W20260709 12:26:05.607770  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 468195us
I20260709 12:26:05.545964  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:26:05.613446  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.571718  9064 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:05.620323  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.640385  6775 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.680s	user 0.009s	sys 0.008s
W20260709 12:26:05.644773  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.661785  6775 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
W20260709 12:26:05.640722  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.667557  5874 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:26:05.591255  8888 raft_consensus.cc:2764] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 12:26:05.697213  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 415765us
I20260709 12:26:05.698854  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:05.710579  5874 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 5: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:05.708360  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:05.709653  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:05.699504  5866 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.011s	sys 0.004s
I20260709 12:26:05.609876  9020 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:05.578702  9013 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:05.713881  5869 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:05.688289  8979 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.026s	sys 0.031s
W20260709 12:26:05.714357  5866 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.011s	sys 0.004s
W20260709 12:26:05.714344  8988 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.022s	sys 0.070s
I20260709 12:26:05.717707  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:05.615487  8668 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:05.584820  9063 consensus_queue.cc:1059] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.131s
I20260709 12:26:05.588963  8708 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:05.618108  8915 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:05.722807  6024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.017s	sys 0.020s
I20260709 12:26:05.700519  8994 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:05.698091  8886 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:05.591876  8945 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:05.737460  6756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.014s	sys 0.011s
I20260709 12:26:05.746483  5874 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:05.755805  6756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.015s	sys 0.011s
I20260709 12:26:05.756699  8945 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:05.757485  8886 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:05.759925  8886 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:05.760425  6024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.017s	sys 0.020s
I20260709 12:26:05.761808  8668 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:05.762099  9077 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:05.763406  9066 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.226s	user 0.000s	sys 0.000s
W20260709 12:26:05.764917  9067 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:26:05.766315  8708 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:05.767261  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1122) took 1349 ms. Trace:
I20260709 12:26:05.724277  9020 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.774088  9020 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:05.774471  9071 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 12:26:05.775117  5874 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
W20260709 12:26:05.732409  8979 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.029s	sys 0.032s
W20260709 12:26:05.777242  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 303455us
I20260709 12:26:05.778239  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:05.748407  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:05.751113  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:05.747419  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:05.754624  8994 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.762980  8668 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:05.779628  6768 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:05.749761  6775 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 12:26:05.738076  9068 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.213s	user 0.002s	sys 0.000s
I20260709 12:26:05.767791  5865 rpcz_store.cc:276] 0709 12:26:04.418281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.427040 (+  8759us) service_pool.cc:224] Handling call
0709 12:26:04.427438 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.476959 (+ 49521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.477026 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.525885 (+ 48859us) log.cc:844] Serialized 26 byte log entry
0709 12:26:04.962647 (+436762us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:05.565605 (+602958us) log.cc:844] Serialized 14 byte log entry
0709 12:26:05.565962 (+   357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.566070 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.766975 (+200905us) raft_consensus.cc:1685] finished
0709 12:26:05.766999 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.767154 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":774634,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":8574,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":9729,"thread_start_us":992433,"threads_started":2,"wal-append.queue_time_us":916132}
I20260709 12:26:05.779395  6793 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:05.786563  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:05.860327  6016 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:05.784822  8994 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:05.743184  8888 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.120s
I20260709 12:26:05.881913  5865 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:05.873245  9085 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:05.906044  9057 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.346s	user 0.000s	sys 0.001s
I20260709 12:26:05.883882  9088 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:26:05.880461  8888 consensus_queue.cc:1243] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:05.918393  8915 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:05.919124  8915 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:05.922436  9078 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:05.922173  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1124) took 1503 ms. Trace:
I20260709 12:26:05.924118  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1127) took 1404 ms. Trace:
I20260709 12:26:05.924592  5873 rpcz_store.cc:276] 0709 12:26:04.519923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.837355 (+317432us) service_pool.cc:224] Handling call
0709 12:26:04.856743 (+ 19388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.883925 (+ 27182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.883965 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.885165 (+  1200us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.588340 (+703175us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:05.588404 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.588510 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.923878 (+335368us) raft_consensus.cc:1685] finished
0709 12:26:05.923905 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.924054 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7845,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":24297,"spinlock_wait_cycles":1271424,"thread_start_us":632597,"threads_started":1,"wal-append.queue_time_us":826478}
I20260709 12:26:05.926360  9013 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:05.926755  5873 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:05.911747  8706 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.986s	user 0.014s	sys 0.002s
W20260709 12:26:05.913189  8976 consensus_meta.cc:287] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.951s	user 0.017s	sys 0.000s
I20260709 12:26:05.944916  9064 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:05.945305  8516 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:05.946027  9064 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:05.947369  8944 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:05.950027  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 882) took 1097 ms. Trace:
I20260709 12:26:05.950639  6021 rpcz_store.cc:276] 0709 12:26:04.852650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.855452 (+  2802us) service_pool.cc:224] Handling call
0709 12:26:04.856025 (+   573us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:04.864670 (+  8645us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.864710 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:04.900951 (+ 36241us) log.cc:844] Serialized 52 byte log entry
0709 12:26:05.563114 (+662163us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:05.563182 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.563282 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.949712 (+386430us) raft_consensus.cc:1685] finished
0709 12:26:05.949736 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.949926 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":159,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":12346,"thread_start_us":587429,"threads_started":1,"wal-append.queue_time_us":851107}
I20260709 12:26:05.921482  8915 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:05.923373  5872 rpcz_store.cc:276] 0709 12:26:04.418905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.518065 (+ 99160us) service_pool.cc:224] Handling call
0709 12:26:04.518470 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.518942 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:04.518970 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.543248 (+ 24278us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.278498 (+735250us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:05.278540 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.278617 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.921767 (+643150us) raft_consensus.cc:1685] finished
0709 12:26:05.921825 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.922074 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":398,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":87,"thread_start_us":635313,"threads_started":1,"wal-append.queue_time_us":920486}
W20260709 12:26:05.956020  9062 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.347s	user 0.000s	sys 0.002s
I20260709 12:26:06.010928  5873 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:26:06.087982  8871 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.058s	sys 0.062s
W20260709 12:26:06.088550  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 484454us
W20260709 12:26:06.096076  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 560089us
W20260709 12:26:06.101683  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.107908  5869 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
W20260709 12:26:06.110366  6022 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.010s	sys 0.030s
W20260709 12:26:06.110985  6022 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.010s	sys 0.030s
I20260709 12:26:06.118317  5913 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:06.142438  5872 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:26:06.143111  5872 pending_rounds.cc:85] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 4
I20260709 12:26:06.143338  5872 pending_rounds.cc:107] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:26:06.143569  5872 raft_consensus.cc:2889] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:06.143962  5873 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:06.140376  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 884) took 1266 ms. Trace:
I20260709 12:26:06.146107  6022 rpcz_store.cc:276] 0709 12:26:04.873609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.884457 (+ 10848us) service_pool.cc:224] Handling call
0709 12:26:04.909802 (+ 25345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.969736 (+ 59934us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:05.564729 (+594993us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.125764 (+561035us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:06.126240 (+   476us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:06.139130 (+ 12890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.140050 (+   920us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.140271 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":751410,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":66,"thread_start_us":1048212,"threads_started":2}
W20260709 12:26:06.150080  6609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.012s	sys 0.005s
W20260709 12:26:06.150463  6609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.012s	sys 0.005s
W20260709 12:26:06.155719  6611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.009s	sys 0.013s
W20260709 12:26:06.156183  6611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.009s	sys 0.013s
I20260709 12:26:06.101115  9070 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:06.160986  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.914403  9085 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:05.952752  8976 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:05.942868  8706 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:06.149350  9096 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:06.168009  8940 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.080s	user 0.012s	sys 0.003s
I20260709 12:26:06.206178  5940 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:06.209898  5867 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.005s	sys 0.033s
W20260709 12:26:06.212347  5867 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.006s	sys 0.033s
W20260709 12:26:06.219463  5942 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.018s	sys 0.012s
W20260709 12:26:06.220981  5942 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.018s	sys 0.012s
I20260709 12:26:06.217139  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 476) took 1773 ms. Trace:
W20260709 12:26:06.222841  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 444648us
I20260709 12:26:06.208297  9077 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:06.224009  5941 rpcz_store.cc:276] 0709 12:26:04.437244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.438251 (+  1007us) service_pool.cc:224] Handling call
0709 12:26:04.445071 (+  6820us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.445527 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.445552 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.446107 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.170548 (+724441us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:05.170619 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.170740 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.210448 (+1039708us) raft_consensus.cc:1685] finished
0709 12:26:06.210481 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.210674 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1948,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":119,"thread_start_us":723951,"threads_started":1,"wal-append.queue_time_us":999025}
W20260709 12:26:06.221951  8931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.022s	sys 0.021s
I20260709 12:26:06.252681  6022 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:06.203752  8940 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:06.233464  8389 consensus_meta.cc:287] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.975s	user 0.012s	sys 0.004s
I20260709 12:26:06.171808  9070 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:06.232880  8936 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.258s	user 0.017s	sys 0.000s
W20260709 12:26:06.255146  8931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.023s	sys 0.021s
W20260709 12:26:06.270510  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 152263us
W20260709 12:26:06.263623  9084 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.176s	user 0.001s	sys 0.001s
I20260709 12:26:06.259992  9077 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:06.202132  9096 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:06.268826  8389 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:06.306895  9085 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:06.278548  8936 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:06.331331  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 636) took 1013 ms. Trace:
I20260709 12:26:06.335909  6023 rpcz_store.cc:276] 0709 12:26:05.318170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.578533 (+260363us) service_pool.cc:224] Handling call
0709 12:26:05.603071 (+ 24538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.686517 (+ 83446us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:06.120477 (+433960us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.252963 (+132486us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.330816 (+ 77853us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:06.330895 (+    79us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:06.330960 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.331058 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.331242 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":69893,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":81,"thread_start_us":557220,"threads_started":3,"wal-append.queue_time_us":171415}
W20260709 12:26:06.335047  9093 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.225s	user 0.000s	sys 0.002s
W20260709 12:26:06.357889  6768 consensus_meta.cc:287] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.500s	user 0.011s	sys 0.003s
I20260709 12:26:06.364270  6768 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:06.369436  5945 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:06.332973  9088 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:06.342767  8948 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:06.343379  9083 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 12:26:06.394896  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 883) took 1521 ms. Trace:
I20260709 12:26:06.362888  8976 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:06.399725  8706 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:06.406721  9082 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:26:06.408267  8940 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:06.410961  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 885) took 1536 ms. Trace:
W20260709 12:26:06.412488  8721 consensus_meta.cc:287] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.824s	user 0.015s	sys 0.000s
W20260709 12:26:06.414345  9105 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:26:06.416424  9070 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:06.400293  6019 rpcz_store.cc:276] 0709 12:26:04.873263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.873912 (+   649us) service_pool.cc:224] Handling call
0709 12:26:04.907037 (+ 33125us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:04.972763 (+ 65726us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.972824 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:05.025749 (+ 52925us) log.cc:844] Serialized 52 byte log entry
0709 12:26:05.602766 (+577017us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:05.602809 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.602882 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.394530 (+791648us) raft_consensus.cc:1685] finished
0709 12:26:06.394561 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.394790 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17507,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":22094,"thread_start_us":496424,"threads_started":1,"wal-append.queue_time_us":792835}
I20260709 12:26:06.411517  6024 rpcz_store.cc:276] 0709 12:26:04.874717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.874934 (+   217us) service_pool.cc:224] Handling call
0709 12:26:04.902879 (+ 27945us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.946686 (+ 43807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.946723 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.977363 (+ 30640us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.760953 (+783590us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:05.760999 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.761076 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.410681 (+649605us) raft_consensus.cc:1685] finished
0709 12:26:06.410706 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.410882 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6833,"raft.queue_time_us":33834,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":12711,"thread_start_us":705120,"threads_started":1,"wal-append.queue_time_us":941386}
I20260709 12:26:06.424588  9113 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:06.416481  8842 consensus_meta.cc:287] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.824s	user 0.013s	sys 0.002s
W20260709 12:26:06.426765  9097 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 12:26:06.426436  8842 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:06.427376  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1126) took 1979 ms. Trace:
I20260709 12:26:06.427850  5867 rpcz_store.cc:276] 0709 12:26:04.448306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.736690 (+288384us) service_pool.cc:224] Handling call
0709 12:26:04.737190 (+   500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.769106 (+ 31916us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:05.468938 (+699832us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.216018 (+747080us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.253898 (+ 37880us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.254108 (+   210us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:06.254159 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.254264 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.427076 (+172812us) raft_consensus.cc:1685] finished
0709 12:26:06.427100 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.427293 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":938031,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":32983,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":12734,"thread_start_us":1397006,"threads_started":2,"wal-append.queue_time_us":878807}
I20260709 12:26:06.425474  9113 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:06.439386  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 212401us
W20260709 12:26:06.430436  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.000s	sys 0.022s
W20260709 12:26:06.450287  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.001s	sys 0.022s
I20260709 12:26:06.455416  5942 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:26:06.465130  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:06.476135  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:06.476514  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:06.429423  8842 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:06.413260  8721 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:06.496044  9079 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.742s	user 0.003s	sys 0.001s
I20260709 12:26:06.497797  9124 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:06.498523  9124 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:06.500001  8721 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:06.386121  8948 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:06.501423  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 629) took 1417 ms. Trace:
I20260709 12:26:06.502677  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1131) took 1868 ms. Trace:
I20260709 12:26:06.503106  9070 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.504201  9096 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:06.503978  9070 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:06.505761  9069 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.907s	user 0.004s	sys 0.014s
I20260709 12:26:06.507640  9070 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:06.507928  8936 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:06.502050  6017 rpcz_store.cc:276] 0709 12:26:05.083884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.084190 (+   306us) service_pool.cc:224] Handling call
0709 12:26:05.114505 (+ 30315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.114845 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:05.587801 (+472956us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.488550 (+900749us) spinlock_profiling.cc:243] Waited 7.17 ms on lock 0x7b180003bda0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39d282b4bc 00007f39cfb7948b 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287
0709 12:26:06.489049 (+   499us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.489251 (+   202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:06.489312 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:06.489380 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.489485 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.501325 (+ 11840us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8285,"prepare.queue_time_us":619191,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":86,"raft.queue_time_us":148571,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":23414,"spinlock_wait_cycles":16490368,"thread_start_us":931033,"threads_started":3,"wal-append.queue_time_us":654641}
I20260709 12:26:06.515208  9113 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:06.516496  9107 log.cc:927] Time spent T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.284s	user 0.000s	sys 0.002s
I20260709 12:26:06.483219  9112 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.120s
W20260709 12:26:06.484249  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 798008us
W20260709 12:26:06.484968  5987 connection.cc:256] RPC call timeout handler was delayed by -2.040781270000025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1564s for a response
W20260709 12:26:06.517917  5987 connection.cc:256] RPC call timeout handler was delayed by -1.9930142149996755s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1877s for a response
W20260709 12:26:06.518266  5987 connection.cc:256] RPC call timeout handler was delayed by -1.8238142149998566s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20650000000000002s for a response
W20260709 12:26:06.486442  9103 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:26:06.518778  9112 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:06.456521  9077 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:06.519732  9112 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:06.487787  9114 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:06.516637  8994 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.626s	user 0.006s	sys 0.005s
I20260709 12:26:06.518438  8948 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:06.518571  5987 connection.cc:256] RPC call timeout handler was delayed by -1.6415299669997694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1648s for a response
W20260709 12:26:06.538780  5987 connection.cc:256] RPC call timeout handler was delayed by -1.6257481109996661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31010000000000004s for a response
W20260709 12:26:06.539162  5987 connection.cc:256] RPC call timeout handler was delayed by -1.4462260449995483s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2215s for a response
W20260709 12:26:06.539594  5987 connection.cc:256] RPC call timeout handler was delayed by -1.2989258129996415s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 12:26:06.539875  5987 connection.cc:256] RPC call timeout handler was delayed by -1.1834299669999382s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2157s for a response
I20260709 12:26:06.540421  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:06.522539  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:06.541713  5947 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:06.536176  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1125) took 2092 ms. Trace:
W20260709 12:26:06.479396  6016 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.002s	sys 0.006s
I20260709 12:26:06.530586  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:06.543452  5946 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:06.544203  6016 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.002s	sys 0.007s
I20260709 12:26:06.503260  5866 rpcz_store.cc:276] 0709 12:26:04.634177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.005276 (+371099us) service_pool.cc:224] Handling call
0709 12:26:05.010231 (+  4955us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:05.012710 (+  2479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:05.012737 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:05.053346 (+ 40609us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.749269 (+695923us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:05.749421 (+   152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.749689 (+   268us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.502398 (+752709us) raft_consensus.cc:1685] finished
0709 12:26:06.502426 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.502575 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":619348,"threads_started":1,"wal-append.queue_time_us":870069}
I20260709 12:26:06.522378  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 639) took 1186 ms. Trace:
W20260709 12:26:06.518904  9095 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
W20260709 12:26:06.549983  5840 connection.cc:256] RPC call timeout handler was delayed by -3.9651377420000244s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1779s for a response
I20260709 12:26:06.553822  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:06.562194  5915 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:06.536917  9020 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.762s	user 0.013s	sys 0.003s
I20260709 12:26:06.501024  9124 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:06.565487  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:06.566234  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:06.504647  8842 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:06.555322  5840 connection.cc:256] RPC call timeout handler was delayed by -3.1547106259995417s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19540000000000002s for a response
I20260709 12:26:06.550722  8994 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:06.573931  5948 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 12:26:06.574384  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:06.550930  8950 consensus_meta.cc:287] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.118s	user 0.012s	sys 0.006s
I20260709 12:26:06.544106  5870 rpcz_store.cc:276] 0709 12:26:04.443377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.522218 (+ 78841us) service_pool.cc:224] Handling call
0709 12:26:04.522608 (+   390us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:04.628010 (+105402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.628074 (+    64us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:04.640773 (+ 12699us) log.cc:844] Serialized 52 byte log entry
0709 12:26:05.282443 (+641670us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:05.282490 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.282572 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.505021 (+1222449us) spinlock_profiling.cc:243] Waited 3.97 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:06.522897 (+ 17876us) raft_consensus.cc:1685] finished
0709 12:26:06.522943 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.536051 (+ 13108us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12263,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":13260,"spinlock_wait_cycles":9133952,"thread_start_us":634056,"threads_started":1,"wal-append.queue_time_us":901809}
I20260709 12:26:06.545552  6018 rpcz_store.cc:276] 0709 12:26:05.335795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.731684 (+395889us) service_pool.cc:224] Handling call
0709 12:26:05.747958 (+ 16274us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:06.197674 (+449716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.197717 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:06.213083 (+ 15366us) log.cc:844] Serialized 52 byte log entry
0709 12:26:06.342686 (+129603us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:06.342732 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.342833 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.521978 (+179145us) raft_consensus.cc:1685] finished
0709 12:26:06.522011 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.522286 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":270458,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":40213,"thread_start_us":493563,"threads_started":2,"wal-append.queue_time_us":219949}
I20260709 12:26:06.579828  5944 raft_consensus.cc:2368] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
I20260709 12:26:06.580950  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:06.586256  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.586704  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:06.596982  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:06.584025  8668 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:06.598924  5915 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:06.599601  6019 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:06.603996  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:06.612962  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:06.618124  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:06.618430  5874 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 12: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:06.607002  5987 leader_election.cc:400] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
I20260709 12:26:06.620223  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:06.591683  9124 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:06.542689  9114 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:06.582643  9020 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:06.595923  9096 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:26:06.632654  9128 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:06.636818  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:06.626866  5870 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:06.599716  8976 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:06.624539  5987 leader_election.cc:403] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:26:06.602530  8668 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:06.637884  6022 raft_consensus.cc:2410] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:06.638424  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:06.638496  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:06.589224  8842 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:06.626413  9122 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.620777  8888 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:06.592705  8950 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:06.644881  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:06.647141  5987 leader_election.cc:336] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.546s (SENT)
I20260709 12:26:06.652525  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:06.659304  5866 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:06.660317  6024 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:06.660590  6017 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:06.652120  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:06.651602  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 750) took 2224 ms. Trace:
I20260709 12:26:06.658778  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:06.664266  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:06.664809  9112 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:06.665262  6022 raft_consensus.cc:2410] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:26:06.666050  9123 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:26:06.666451  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:06.667892  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:06.667964  9114 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:06.669148  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:06.670356  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:06.671772  8516 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:06.672417  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.672374  8516 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:06.652981  8721 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:06.673728  9117 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 12:26:06.674149  8721 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:06.674592  8516 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:06.676460  8994 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:06.678413  8950 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:06.679049  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:06.679702  6022 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:26:06.680073  5946 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:06.680466  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 483) took 1127 ms. Trace:
I20260709 12:26:06.680663  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:06.662036  6756 rpcz_store.cc:276] 0709 12:26:04.427301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.427532 (+   231us) service_pool.cc:224] Handling call
0709 12:26:04.428029 (+   497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.428367 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:04.904741 (+476374us) log.cc:844] Serialized 14 byte log entry
0709 12:26:05.786977 (+882236us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.494032 (+707055us) spinlock_profiling.cc:243] Waited 65.5 ms on lock 0x7b18000f3400. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb642a6 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db
0709 12:26:06.495195 (+  1163us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:06.651106 (+155911us) spinlock_profiling.cc:243] Waited 71.4 ms on lock 0x7b18000f3400. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb10c0a 00007f39d9250219 00007f39cfb98a40 00007f39cfb934e4 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287 00007f39d2c9e208
0709 12:26:06.651274 (+   168us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:06.651322 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:06.651369 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.651432 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.651537 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":815636,"prepare.queue_time_us":1269508,"prepare.run_cpu_time_us":2348,"prepare.run_wall_time_us":18789,"raft.queue_time_us":896457,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":171,"spinlock_wait_cycles":372412672,"thread_start_us":1315082,"threads_started":2,"wal-append.queue_time_us":848326}
I20260709 12:26:06.681893  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:06.661857  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:06.684006  6019 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:06.685601  5913 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:06.685989  5947 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:06.645572  8668 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:06.682276  6609 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:06.681002  5940 rpcz_store.cc:276] 0709 12:26:05.553188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.559267 (+  6079us) service_pool.cc:224] Handling call
0709 12:26:05.582397 (+ 23130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:06.209536 (+627139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.680087 (+470551us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b54008dfed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:06.680383 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1409558400}
I20260709 12:26:06.694810  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:06.696111  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:06.668198  5990 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:06.696830  5867 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:06.648671  8888 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:06.631978  9124 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:06.646687  9130 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:06.701970  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:06.683189  8389 raft_consensus.cc:2764] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:26:06.714743  8842 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:06.715046  9111 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:06.689085  9070 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:06.707100  9128 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:06.715423  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:06.716022  8743 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:06.720568  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:06.643920  9133 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 12:26:06.693373  9112 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:06.719151  6019 raft_consensus.cc:2393] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 9: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:06.717437  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:06.725723  9116 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:26:06.726532  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:06.726970  9108 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:26:06.709249  8888 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:06.750176  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.707129  8915 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:06.709890  8950 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:06.727034  8762 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.652233  9129 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.111s
I20260709 12:26:06.727361  9059 raft_consensus.cc:2764] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:26:06.827934  8842 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:06.829444  9127 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:26:06.850895  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:06.851004  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 642) took 1428 ms. Trace:
I20260709 12:26:06.831940  9130 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
I20260709 12:26:06.832944  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 499) took 1341 ms. Trace:
W20260709 12:26:06.839004  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 191888us
I20260709 12:26:06.812009  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:06.819154  5867 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
W20260709 12:26:06.818526  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.844061  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:06.811667  8944 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.852233  6021 rpcz_store.cc:276] 0709 12:26:05.423072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.120431 (+697359us) service_pool.cc:224] Handling call
0709 12:26:06.150416 (+ 29985us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.183614 (+ 33198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.183674 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.184478 (+   804us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.569091 (+384613us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:06.569132 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.569211 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.850784 (+281573us) raft_consensus.cc:1685] finished
0709 12:26:06.850803 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.850940 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8498,"raft.queue_time_us":146176,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":9515,"thread_start_us":340989,"threads_started":1,"wal-append.queue_time_us":475861}
I20260709 12:26:06.856124  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:06.867120  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:06.868198  5870 raft_consensus.cc:2410] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:06.869107  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:06.872005  9060 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.174s	user 0.002s	sys 0.012s
I20260709 12:26:06.872473  6793 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:06.826581  9070 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:06.875746  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:06.876796  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 491) took 2480 ms. Trace:
I20260709 12:26:06.877210  5871 rpcz_store.cc:276] 0709 12:26:04.396082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.396240 (+   158us) service_pool.cc:224] Handling call
0709 12:26:04.409617 (+ 13377us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:04.410240 (+   623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.410280 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:04.411152 (+   872us) log.cc:844] Serialized 52 byte log entry
0709 12:26:04.892391 (+481239us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:05.600425 (+708034us) log.cc:844] Serialized 14 byte log entry
0709 12:26:05.604024 (+  3599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.604109 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.674068 (+1069959us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b180023f020. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:26:06.676986 (+  2918us) spinlock_profiling.cc:243] Waited 7.18 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:06.871621 (+194635us) spinlock_profiling.cc:243] Waited 8.3 ms on lock 0x7b240043b218. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfa1c9c4 00007f39cfa1aacb 00007f39cfbd5dc9 00007f39cfbd5a55 00007f39cfbd509d 00007f39d280710e 00007f39d281e88c 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221
0709 12:26:06.876518 (+  4897us) raft_consensus.cc:1685] finished
0709 12:26:06.876543 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.876706 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3129,"prepare.queue_time_us":834016,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":48,"raft.queue_time_us":4015,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":8330,"spinlock_wait_cycles":47974272,"thread_start_us":1159675,"threads_started":2,"wal-append.queue_time_us":1153662}
I20260709 12:26:06.848920  8936 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.885320  8936 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:06.838006  9128 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:06.887197  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:06.888025  5872 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:06.823916  8965 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:26:06.888927  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:06.842581  8915 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:06.891709  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:06.892154  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:06.892439  6766 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:06.893580  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:06.840289  9096 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.895761  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:06.903934  6766 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:06.901263  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:06.856757  5865 rpcz_store.cc:276] 0709 12:26:05.491676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.882990 (+391314us) service_pool.cc:224] Handling call
0709 12:26:05.903236 (+ 20246us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:05.903732 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:05.903765 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:05.904494 (+   729us) log.cc:844] Serialized 52 byte log entry
0709 12:26:06.232495 (+328001us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:06.232551 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.232635 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.832721 (+600086us) raft_consensus.cc:1685] finished
0709 12:26:06.832742 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.832885 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3341,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":5209,"thread_start_us":327522,"threads_started":1,"wal-append.queue_time_us":629240}
I20260709 12:26:06.895045  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 641) took 1515 ms. Trace:
W20260709 12:26:06.897554  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 85666.7us
I20260709 12:26:06.896679  5870 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:06.900501  5866 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:06.905742  6016 rpcz_store.cc:276] 0709 12:26:05.379446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.869355 (+489909us) service_pool.cc:224] Handling call
0709 12:26:05.886710 (+ 17355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.552345 (+665635us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.552380 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.574248 (+ 21868us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.651926 (+ 77678us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:06.651969 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.652063 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.894733 (+242670us) raft_consensus.cc:1685] finished
0709 12:26:06.894767 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.894962 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":346221,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":11941,"thread_start_us":686884,"threads_started":2,"wal-append.queue_time_us":254773}
I20260709 12:26:06.837809  8743 raft_consensus.cc:697] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:06.818176  9114 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:06.901484  5872 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:26:06.652808  9122 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:06.889081  8936 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:06.886124  9020 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:06.920627  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:06.923959  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:06.924067  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:06.925606  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 208087us
I20260709 12:26:06.926164  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:06.932890  6766 raft_consensus.cc:2410] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:26:06.926501  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:06.918473  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:06.919992  5987 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.564s (SENT) [suppressed 9 similar messages]
I20260709 12:26:06.928922  6793 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:06.933972  5866 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:06.935549  6016 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 9: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:06.892454  8915 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:06.941726  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:06.942719  5867 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:26:06.944366  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:06.937235  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:06.913777  8976 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:06.936273  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:06.915994  8950 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:06.947826  8945 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:26:06.947335  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.945302  6015 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:06.952358  6018 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:26:06.954814  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 858) took 1224 ms. Trace:
I20260709 12:26:06.955538  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:06.954730  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:06.853698  8842 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:06.982366  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:06.960083  5870 raft_consensus.cc:2410] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:06.967300  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:06.971074  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:06.973485  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:06.974040  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:06.854045  8762 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:06.986958  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:06.987612  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:06.895988  8944 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:06.991158  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:06.992758  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:06.949856  9109 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.994127  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:07.001053  6793 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:06.994252  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:06.994570  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:06.994788  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:06.995285  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:06.995559  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:07.003122  5868 raft_consensus.cc:2376] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9. Re-sending same reply.
I20260709 12:26:07.003638  6016 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:06.995796  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:06.996320  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:06.950882  8950 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:06.998543  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:07.008627  6766 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:06.999542  5946 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:07.000074  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:07.005005  6021 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:06.998127  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:07.010946  5865 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:07.011937  5865 pending_rounds.cc:85] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 3
I20260709 12:26:07.012121  5865 pending_rounds.cc:107] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:26:07.012272  5865 raft_consensus.cc:2889] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:07.006335  6015 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:06.955384  6778 rpcz_store.cc:276] 0709 12:26:05.724817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.725137 (+   320us) service_pool.cc:224] Handling call
0709 12:26:05.725735 (+   598us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:05.762243 (+ 36508us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:05.762555 (+   312us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.688529 (+925974us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:06.700581 (+ 12052us) log.cc:844] Serialized 52 byte log entry
0709 12:26:06.700812 (+   231us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:06.700845 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.700907 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.949171 (+248264us) raft_consensus.cc:1685] finished
0709 12:26:06.949192 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.949375 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":525661,"prepare.queue_time_us":121305,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":561788,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":257,"thread_start_us":399773,"threads_started":1,"wal-append.queue_time_us":595466}
I20260709 12:26:06.887238  9128 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:07.022583  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:07.023156  6785 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:06.982157  8965 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:06.983906  8871 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.926007  9096 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:07.023770  5868 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:07.028023  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:07.028574  6611 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:07.029168  5868 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
W20260709 12:26:06.990681  9063 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.224s	user 0.015s	sys 0.011s
W20260709 12:26:07.031395  9063 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.267s	user 0.015s	sys 0.011s
I20260709 12:26:07.031899  9063 consensus_queue.cc:1243] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:26:07.030267  8516 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:07.032971  8516 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:07.033540  8516 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:07.036051  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 76666.5us
I20260709 12:26:06.986143  9034 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.006693  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:07.037457  5913 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:07.043691  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:07.038524  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:07.046131  5866 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:06.950897  9114 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:07.050340  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 130361us
I20260709 12:26:07.051371  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:07.066895  5868 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:07.078497  6016 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:07.080343  6015 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:07.037508  8965 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.984068  9111 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.064s
W20260709 12:26:07.089344  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 60862us
I20260709 12:26:07.040410  8931 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:07.002409  8979 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
W20260709 12:26:07.095831  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 86015.7us
I20260709 12:26:07.096514  6017 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:26:07.052438  8945 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:07.039710  9088 consensus_queue.cc:1059] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:26:07.103148  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:07.116720  5868 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:07.006927  9109 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:07.121212  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.097318  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 109414us
I20260709 12:26:07.127962  6766 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:07.129398  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:07.130355  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:07.085994  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:07.146840  6793 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:07.052199  9034 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:07.151615  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.151254  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:07.153663  5868 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:26:07.085994  8965 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:07.043263  8871 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:07.155526  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:07.161545  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:07.182343  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:07.190646  5987 leader_election.cc:336] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 2.157s (SENT)
I20260709 12:26:07.191768  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 480) took 1697 ms. Trace:
I20260709 12:26:07.192274  5945 rpcz_store.cc:276] 0709 12:26:05.494095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.494382 (+   287us) service_pool.cc:224] Handling call
0709 12:26:05.513724 (+ 19342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:06.370026 (+856302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.191502 (+821476us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b54006a3bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:07.191699 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1967311616}
I20260709 12:26:07.196127  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.163546  9118 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:07.201190  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:07.100461  8950 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:07.100687  8931 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:07.090493  8743 consensus_queue.cc:237] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:07.234596  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:07.237748  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:07.188568  8667 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:07.174401  8663 consensus_queue.cc:1059] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:26:07.204001  9118 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.187757  8957 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:26:07.184087  9013 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:26:07.250866  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:07.250430  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 95486.3us
I20260709 12:26:07.257062  6611 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:07.258622  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 211401us
I20260709 12:26:07.260591  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.261611  5947 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:07.262069  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:07.266623  6611 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:07.248747  8842 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:07.274492  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.274878  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:07.255373  8944 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:07.243670  9144 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:07.249183  9118 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:07.284092  5945 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:07.257733  8663 consensus_queue.cc:1243] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:26:07.268488  9137 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.274s	user 0.000s	sys 0.002s
W20260709 12:26:07.288419  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.288501  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:07.287079  9139 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.216s	user 0.001s	sys 0.001s
W20260709 12:26:07.292164  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 107096us
I20260709 12:26:07.293159  5990 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:07.271787  8762 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:07.295342  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.302488  9114 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:07.303340  9147 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:26:07.251116  8667 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:07.305989  8667 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:07.306191  9138 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:26:07.306365  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:07.307319  5869 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:26:07.284657  8950 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:07.279155  9096 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:07.274955  9148 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:07.325922  9148 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:07.326710  8976 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:07.327176  8931 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:07.329048  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:07.299288  5990 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:07.268893  9149 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:07.306025  9109 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:07.298892  5987 leader_election.cc:336] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 2.204s (SENT)
I20260709 12:26:07.334496  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:07.287541  8539 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:07.319406  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:07.319497  8871 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:07.323405  9058 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:07.324265  9064 raft_consensus.cc:3055] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:07.348899  5945 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:07.300221  9063 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:07.334846  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:07.328538  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:07.339736  9142 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
W20260709 12:26:07.340911  9150 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:26:07.342105  8842 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:07.350517  6785 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
W20260709 12:26:07.302597  9143 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:26:07.331226  5990 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:07.327243  8976 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:07.344338  9144 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:07.344616  9064 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:07.328538  9148 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:07.349826  8668 consensus_meta.cc:287] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.643s	user 0.011s	sys 0.002s
I20260709 12:26:07.355584  9063 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.377955  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:07.380501  6024 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:07.334515  9149 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.345916  9058 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:07.388725  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 57566.6us
I20260709 12:26:07.383821  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:07.392159  9064 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:07.404354  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:07.377512  8539 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:07.406199  8539 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:07.407034  8888 consensus_meta.cc:287] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.002s
I20260709 12:26:07.407560  8888 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:07.408747  5940 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:07.400352  6017 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:07.408475  8668 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:07.393261  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:07.414162  6762 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:07.405001  9064 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:07.419802  8842 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:07.416065  9149 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:07.434193  6020 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:07.437916  5870 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:07.399686  9144 raft_consensus.cc:697] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:07.438488  8888 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:07.420248  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:07.439414  9144 consensus_queue.cc:237] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:07.422425  8516 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:07.412364  9063 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:07.424366  8721 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:07.428323  9122 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:07.411736  9124 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:07.440104  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:07.443252  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:07.399993  9114 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:26:07.445104  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 61420us
I20260709 12:26:07.459352  6762 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:07.460443  5949 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:07.462774  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:07.464357  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:07.465754  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:07.467029  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:07.467572  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:07.468160  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:07.468823  6757 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:07.474534  9070 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:07.476068  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 496) took 1989 ms. Trace:
I20260709 12:26:07.476531  5874 rpcz_store.cc:276] 0709 12:26:05.486641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.776123 (+289482us) service_pool.cc:224] Handling call
0709 12:26:07.475826 (+1699703us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b54001446d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:07.475992 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1880316672}
W20260709 12:26:07.477110  9158 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.087s	user 0.003s	sys 0.000s
I20260709 12:26:07.482383  8904 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:07.403944  9058 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:07.483160  8668 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:07.452206  8888 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:07.483798  9058 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:07.484359  9034 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:07.474610  6016 raft_consensus.cc:2410] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:26:07.484220  8888 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:07.489189  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:07.489959  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 230626us
I20260709 12:26:07.451884  8842 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:07.492235  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:07.492024  5838 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:07.511832  5838 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:07.494428  9114 raft_consensus.cc:697] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:07.494179  9124 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:07.502676  8615 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:07.502503  8842 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:07.523064  6757 raft_consensus.cc:2410] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:26:07.506114  8994 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:07.546674  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:07.559103  9149 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:07.563282  9149 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:07.582836  8516 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:07.584138  9114 consensus_queue.cc:237] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:07.590963  8888 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:07.594646  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:07.596283  6757 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:07.518874  8950 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 12:26:07.602867  9063 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:07.605962  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:07.606312  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:07.606848  5942 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:07.606921  8945 raft_consensus.cc:3055] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:07.607648  8945 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:07.607767  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.608227  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.608628  8945 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:07.609745  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.518106  8615 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:07.620775  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.621779  5944 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 12: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:07.620828  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.623426  5948 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 10: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:07.549556  9020 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:07.627753  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:07.549253  8988 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:07.632552  5868 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:07.612399  8615 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:07.628484  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:07.628980  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.629572  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:07.635170  5865 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:07.552922  8915 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.552170  8965 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:07.535148  8976 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:07.619170  9122 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:07.656201  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.695549  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.714517  5943 raft_consensus.cc:2368] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 11. Current term is 12.
I20260709 12:26:07.636945  8945 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:07.714555  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.634254  9063 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:07.716046  5941 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:26:07.635826  8988 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.581383  9118 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:07.731005  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.648442  9020 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:07.733448  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.739531  5944 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:07.741676  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.652777  8915 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:07.750217  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.722383  8988 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:07.716701  9122 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:07.742858  9059 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:07.761822  8957 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.776809  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:07.779493  5944 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:07.734098  9020 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:07.789081  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.796113  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:07.798738  6793 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:26:07.798981  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 421015us
I20260709 12:26:07.799013  5990 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:07.783023  8957 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:07.811285  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:07.811977  9168 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:26:07.812158  6793 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:26:07.822733  9118 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:07.823678  5840 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:07.825129  9176 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:07.825646  9176 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:07.826015  9176 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:07.832284  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:07.834122  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.855648  6762 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:07.807992  9112 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:07.857280  5948 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:26:07.863633  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:07.897194  5913 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:07.864589  6778 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:07.864751  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:07.904219  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 730) took 1014 ms. Trace:
I20260709 12:26:07.904664  6775 rpcz_store.cc:276] 0709 12:26:06.890252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.893717 (+  3465us) service_pool.cc:224] Handling call
0709 12:26:07.864568 (+970851us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b580001f898. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:26:07.904143 (+ 39575us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4338048}
I20260709 12:26:07.869915  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:07.874168  9169 log.cc:927] Time spent T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:26:07.816495  9144 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:26:07.878265  8965 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:07.834810  8976 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:07.880815  8957 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:07.881289  9180 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:26:07.882592  9122 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:07.889484  5944 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:07.893759  5865 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:07.932899  5948 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:26:07.894785  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:07.899555  6793 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:07.900856  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:07.902637  5943 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:07.913560  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:07.970276  5867 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:26:07.989835  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 74723.7us
W20260709 12:26:07.991222  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 374394us
I20260709 12:26:07.914633  9128 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:07.910089  9112 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:07.996786  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 102054us
I20260709 12:26:07.996989  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:08.003029  5866 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:08.019819  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:08.065380  8915 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:08.017685  9112 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:08.029026  8957 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:26:08.076217  8615 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:08.017040  8976 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 12:26:08.080137  9013 consensus_queue.cc:1059] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:08.016829  9128 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:08.083343  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 92406us
I20260709 12:26:08.182009  9128 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:08.182193  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 326807us
I20260709 12:26:08.183260  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:08.183517  5840 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:08.185837  8976 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:08.187063  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:08.187719  5867 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:08.188555  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:08.189604  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:08.190305  5945 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:08.191526  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:08.025985  9122 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:26:08.147330  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.196826  9122 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.197412  9122 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:08.153932  8539 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:26:08.199479  8539 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:08.192148  9063 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:08.199949  8539 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:08.195604  8994 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:08.200896  8539 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:08.209414  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:08.246268  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:08.029649  8663 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.179s
I20260709 12:26:08.150784  5866 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
W20260709 12:26:08.303145  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56996.9us
I20260709 12:26:08.083897  9144 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:08.090998  9058 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.207s
I20260709 12:26:08.293398  8915 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:08.358497  5873 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:26:08.357856  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 763) took 1368 ms. Trace:
I20260709 12:26:08.359975  6785 rpcz_store.cc:276] 0709 12:26:06.968275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.968454 (+   179us) service_pool.cc:224] Handling call
0709 12:26:08.336633 (+1368179us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b54002a17d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:08.336856 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46398592}
I20260709 12:26:08.281183  9114 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 12:26:08.315615  9185 raft_consensus.cc:2764] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
W20260709 12:26:08.361085  9186 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.195s	user 0.001s	sys 0.001s
I20260709 12:26:08.325310  9064 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:08.386409  8663 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:08.375110  9020 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:08.324944  8539 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:08.410815  9064 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:08.424814  8663 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:08.428390  9020 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:08.493877  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 411216us
I20260709 12:26:08.461261  9064 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:08.507968  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 770) took 1322 ms. Trace:
I20260709 12:26:08.480041  9020 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:08.480336  9114 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:08.508606  6757 rpcz_store.cc:276] 0709 12:26:07.185156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.597860 (+412704us) service_pool.cc:224] Handling call
0709 12:26:07.598175 (+   315us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:07.614849 (+ 16674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.614880 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:07.621895 (+  7015us) log.cc:844] Serialized 52 byte log entry
0709 12:26:07.802936 (+181041us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:07.802979 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.803058 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.507594 (+704536us) raft_consensus.cc:1685] finished
0709 12:26:08.507628 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.507880 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20043,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":3160,"thread_start_us":155095,"threads_started":1,"wal-append.queue_time_us":389164}
W20260709 12:26:08.520804  9187 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
W20260709 12:26:08.521451  8988 consensus_meta.cc:287] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.756s	user 0.012s	sys 0.001s
I20260709 12:26:08.521907  8988 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:08.511904  8945 consensus_meta.cc:287] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.784s	user 0.010s	sys 0.005s
W20260709 12:26:08.525974  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 334498us
W20260709 12:26:08.530417  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.536247  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:08.539559  5867 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:08.523089  8615 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:08.549196  8615 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.538031  8957 consensus_queue.cc:1059] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:26:08.556910  5867 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:08.523368  9114 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:08.539330  9176 consensus_meta.cc:287] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.711s	user 0.010s	sys 0.005s
I20260709 12:26:08.562687  5947 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:26:08.547003  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 238487us
I20260709 12:26:08.549988  8945 raft_consensus.cc:2764] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:26:08.535552  9144 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:26:08.553755  8615 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:08.577802  5945 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:08.557595  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:08.565739  9175 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.185s	user 0.000s	sys 0.002s
W20260709 12:26:08.572180  9178 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 12:26:08.642200  8945 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:08.636722  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:08.642814  8945 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:08.567158  9128 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:08.644342  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 86854.9us
I20260709 12:26:08.643141  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 727) took 1804 ms. Trace:
I20260709 12:26:08.544106  8663 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:08.646186  6768 rpcz_store.cc:276] 0709 12:26:06.838427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.838587 (+   160us) service_pool.cc:224] Handling call
0709 12:26:06.838983 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.839339 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.839369 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.498859 (+659490us) spinlock_profiling.cc:243] Waited 7.28 ms on lock 0x7b2c0001ffa8. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:26:07.504463 (+  5604us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.741402 (+236939us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:07.741465 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.741588 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.642767 (+901179us) raft_consensus.cc:1685] finished
0709 12:26:08.642794 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.643062 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":461,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"spinlock_wait_cycles":16753920,"thread_start_us":223220,"threads_started":1,"wal-append.queue_time_us":729997}
I20260709 12:26:08.646191  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:08.565245  9176 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:08.657737  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:08.659511  9133 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:26:08.660144  9193 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:26:08.647755  8945 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:08.662045  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 868) took 1216 ms. Trace:
I20260709 12:26:08.662437  6773 rpcz_store.cc:276] 0709 12:26:07.445265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.818076 (+372811us) service_pool.cc:224] Handling call
0709 12:26:07.818494 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.818940 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.818973 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.835380 (+ 16407us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.418506 (+583126us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:08.418557 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.418640 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.661758 (+243118us) raft_consensus.cc:1685] finished
0709 12:26:08.661782 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.661975 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":34288,"thread_start_us":477331,"threads_started":1,"wal-append.queue_time_us":612309}
I20260709 12:26:08.665263  8615 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:08.665853  9196 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:08.669274  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:08.670543  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:08.670806  5871 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:08.660101  9133 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:08.673454  9013 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:08.673949  9013 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:08.674315  9013 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:08.678517  9133 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:08.680076  9133 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:08.690114  8988 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:08.693522  5871 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:08.695691  9176 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:08.703876  9196 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.703213  9092 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:08.709333  9196 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:08.710115  9191 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:26:08.710232  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:08.712396  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:08.704324  9192 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:26:08.709357  9092 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:08.716512  9092 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:08.718465  9092 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:08.703775  9183 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:26:08.712998  5870 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:26:08.724669  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:08.659569  9194 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:26:08.725895  5870 raft_consensus.cc:2368] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8.
I20260709 12:26:08.727397  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 777) took 1236 ms. Trace:
I20260709 12:26:08.727958  6756 rpcz_store.cc:276] 0709 12:26:07.490816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.833516 (+342700us) service_pool.cc:224] Handling call
0709 12:26:07.833892 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.834129 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.834154 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.902625 (+ 68471us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.433385 (+530760us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:08.433438 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.433559 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.727124 (+293565us) raft_consensus.cc:1685] finished
0709 12:26:08.727151 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.727329 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":430623,"threads_started":1,"wal-append.queue_time_us":603275}
I20260709 12:26:08.728919  9128 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:08.727313  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:08.730206  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:08.730803  5870 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:08.713042  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:08.732491  5871 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:26:08.653699  8888 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:08.701783  9122 consensus_meta.cc:287] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.504s	user 0.011s	sys 0.004s
I20260709 12:26:08.712845  9196 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:08.744807  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:08.746833  5867 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:26:08.764026  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:08.765300  5867 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:08.748840  9085 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:08.751859  9202 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:26:08.778307  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 246889us
I20260709 12:26:08.761615  9063 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:08.783560  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:08.790126  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:08.790376  6021 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:08.784582  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.791028  6022 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
W20260709 12:26:08.787601  5840 connection.cc:256] RPC call timeout handler was delayed by -1.2519689519999702s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
I20260709 12:26:08.742014  8888 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:08.746095  9122 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:08.788661  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:08.796341  5867 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 13: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:26:08.792284  5840 connection.cc:256] RPC call timeout handler was delayed by -1.2222689519999221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
W20260709 12:26:08.797722  5840 connection.cc:256] RPC call timeout handler was delayed by -1.1826689519998581s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1617s for a response
I20260709 12:26:08.799095  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:08.798164  5840 connection.cc:256] RPC call timeout handler was delayed by -1.1502689519998057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1653s for a response
W20260709 12:26:08.800591  5840 connection.cc:256] RPC call timeout handler was delayed by -1.0926689519997126s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17170000000000002s for a response
W20260709 12:26:08.800989  5840 connection.cc:256] RPC call timeout handler was delayed by -1.0107689520000349s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
W20260709 12:26:08.801460  5840 connection.cc:256] RPC call timeout handler was delayed by -1.0080689520000305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1811s for a response
W20260709 12:26:08.802908  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 131098us
I20260709 12:26:08.803740  9203 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:08.804168  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:08.804334  9203 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:08.805140  5868 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:08.805195  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:08.806967  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:08.807188  8539 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:08.807916  9211 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:08.808441  9211 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.808830  9211 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:08.810467  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1245) took 1626 ms. Trace:
I20260709 12:26:08.810691  9203 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:08.810968  5940 rpcz_store.cc:276] 0709 12:26:07.184343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.184473 (+   130us) service_pool.cc:224] Handling call
0709 12:26:07.184814 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.409306 (+224492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.810193 (+1400887us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b5400308ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:08.810401 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":514496896}
I20260709 12:26:08.814280  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:08.815096  5943 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:08.816356  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:08.828400  5947 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:26:08.832700  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.834097  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:08.834976  6024 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 13: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:08.827425  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:08.835867  5945 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:08.836206  6022 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:08.840626  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:08.841646  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:08.834667  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:08.844183  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:08.844918  5871 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:08.844184  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:08.846217  6017 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:08.848664  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:08.849575  6024 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:08.792273  9088 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:08.851918  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:08.815418  8888 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:08.857975  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:08.856043  5840 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:08.858759  6019 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:08.854614  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:08.830345  9198 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:08.857499  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 295665us
I20260709 12:26:08.831241  9144 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.859582  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:08.836738  9203 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:08.863427  5867 raft_consensus.cc:2410] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:26:08.778558  9085 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:08.861974  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:08.864497  6017 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:08.856225  9112 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.678s	user 0.016s	sys 0.000s
I20260709 12:26:08.844995  9114 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:08.880512  5987 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:08.883482  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:08.884506  6019 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:08.872278  9133 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:08.892037  9063 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:08.894814  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1244) took 1710 ms. Trace:
I20260709 12:26:08.895465  5949 rpcz_store.cc:276] 0709 12:26:07.184046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.184385 (+   339us) service_pool.cc:224] Handling call
0709 12:26:07.184739 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.470447 (+285708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.894371 (+1423924us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b540058fa54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:08.894654 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":630678272}
I20260709 12:26:08.899619  9064 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:08.900101  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 518) took 1294 ms. Trace:
I20260709 12:26:08.900969  6020 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:08.901549  9203 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:08.902029  9203 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:08.902381  9203 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:08.903118  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:08.903857  6020 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:08.903685  8945 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:08.905956  8945 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:08.906009  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:08.906802  5867 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:08.793340  9202 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:08.909538  9195 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.910188  9195 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:08.910928  9202 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:08.912636  9195 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:08.912940  9122 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:08.872548  9198 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:08.858001  9088 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:08.914945  6016 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:08.916067  9198 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:08.916378  9088 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:08.878707  9144 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:08.919039  6023 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 10: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:08.920257  9144 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:08.922783  9203 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:08.923127  9094 raft_consensus.cc:3055] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:08.944361  9094 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:08.944726  9064 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:08.945544  9094 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:08.947053  9094 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:08.925864  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:08.907851  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:08.952034  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 933) took 1205 ms. Trace:
I20260709 12:26:08.900786  5873 rpcz_store.cc:276] 0709 12:26:07.605830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.609720 (+  3890us) service_pool.cc:224] Handling call
0709 12:26:07.610041 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.359169 (+749128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.899644 (+540475us) spinlock_profiling.cc:243] Waited 748 ms on lock 0x7b5400027e54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:08.899986 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1720193408}
I20260709 12:26:08.876631  5868 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:08.882597  9020 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:08.954115  9119 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:08.954595  9119 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:08.954747  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:08.955489  9119 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:08.927492  9085 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:08.933560  9133 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:08.957204  9119 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:08.957418  9133 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:08.958804  9133 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:08.933969  8615 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.961544  8516 raft_consensus.cc:3055] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:08.962059  8615 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:08.964949  8615 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:08.966187  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:08.966886  5873 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:26:08.967762  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:08.968824  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:08.932734  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 52299.7us
I20260709 12:26:08.970167  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:08.971891  5987 leader_election.cc:400] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8.
I20260709 12:26:08.972167  5987 leader_election.cc:403] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:08.972718  9114 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:08.973215  6022 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
W20260709 12:26:08.941846  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 61497.3us
I20260709 12:26:08.942523  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:08.974828  8516 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:08.976382  6020 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 10: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:26:08.939134  9188 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.552s	user 0.003s	sys 0.000s
I20260709 12:26:08.977537  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:08.982925  6020 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:08.983942  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:08.973448  9114 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8.
I20260709 12:26:08.935719  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 771) took 1749 ms. Trace:
I20260709 12:26:08.976163  8516 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:09.013669  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:08.977912  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1152) took 1709 ms. Trace:
I20260709 12:26:09.010337  6762 rpcz_store.cc:276] 0709 12:26:07.186232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.764365 (+578133us) service_pool.cc:224] Handling call
0709 12:26:07.764765 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.901674 (+136909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.935414 (+1033740us) spinlock_profiling.cc:243] Waited 70.4 ms on lock 0x7b5400004454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:08.935645 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":162032512}
I20260709 12:26:08.951048  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:08.952486  5941 rpcz_store.cc:276] 0709 12:26:07.746566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.925404 (+1178838us) service_pool.cc:224] Handling call
0709 12:26:08.951944 (+ 26540us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:08.882550  9112 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:08.980448  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:08.985117  8936 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.956245  9020 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:08.985141  5867 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:26:08.962282  9211 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:08.928118  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:09.028919  5867 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:08.974473  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:08.994585  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:08.996254  6021 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:26:08.997174  5865 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:26:09.014880  5872 rpcz_store.cc:276] 0709 12:26:07.268195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.435974 (+167779us) service_pool.cc:224] Handling call
0709 12:26:07.436319 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.774766 (+338447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.774799 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.777425 (+  2626us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.813951 (+ 36526us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:07.813997 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.814074 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.938323 (+1124249us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:08.977569 (+ 39246us) raft_consensus.cc:1685] finished
0709 12:26:08.977591 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.977852 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59795,"raft.queue_time_us":73555,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":3893,"spinlock_wait_cycles":11048832,"thread_start_us":297044,"threads_started":2,"wal-append.queue_time_us":513345}
I20260709 12:26:08.998389  5870 raft_consensus.cc:2410] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:08.956147  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:09.004702  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:09.033586  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:09.007467  5871 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:09.002055  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:09.034245  5868 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:09.022608  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:08.933166  9013 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:09.030321  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:09.046751  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:09.020318  9064 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.037174  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:09.000694  6017 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
W20260709 12:26:09.038259  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 63839.1us
I20260709 12:26:09.027716  8936 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:09.042554  5947 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:26:09.043182  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:09.036044  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:09.056717  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.057487  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:09.062702  5872 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:09.027863  9202 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:09.064746  5866 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:09.068197  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:09.009737  8915 consensus_queue.cc:1059] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:26:09.014242  9195 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:26:09.048422  5873 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:09.034339  9176 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.072160  6021 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:26:09.084630  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:09.089783  6019 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:09.051734  9114 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:09.086093  5987 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:09.021283  8988 consensus_queue.cc:1059] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:26:09.051551  9159 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:09.028115  9085 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:26:09.066512  8976 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:09.105747  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:09.066726  9171 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:09.070109  9167 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.113055  6015 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:09.079080  9092 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:09.118279  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:09.085332  9202 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:09.099522  8615 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:09.065289  9064 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:09.127657  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:09.115384  9198 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:26:09.143054  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 149086us
I20260709 12:26:09.117781  8988 raft_consensus.cc:3055] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:09.148147  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:09.113334  8888 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:09.098917  9176 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:09.154969  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 299038us
I20260709 12:26:09.160319  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:09.120280  9144 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:09.154872  5944 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:09.118901  8842 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:09.170682  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:09.171819  5941 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:09.170351  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:09.129691  9171 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:09.170285  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:09.122727  8976 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:09.176555  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:09.182583  6019 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:09.129379  8615 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:09.186399  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:09.139088  9063 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:09.191394  5945 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 8: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:09.192046  9216 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:09.193231  9112 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:09.154603  8988 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:09.193609  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:09.198325  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:09.197181  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:26:09.202826  9166 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.730s	user 0.002s	sys 0.011s
I20260709 12:26:09.177459  8888 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:09.181150  9144 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:09.156754  9092 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:09.192498  9013 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:09.150555  9198 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:09.159438  9213 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:09.133682  9088 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:26:09.136323  9114 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.049s
W20260709 12:26:09.217847  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 65606.7us
I20260709 12:26:09.221094  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:09.237401  6017 raft_consensus.cc:2376] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8. Re-sending same reply.
I20260709 12:26:09.238125  6611 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.227139  5944 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:09.121964  9167 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:09.229822  5840 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.209774  8615 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:09.210882  9063 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:09.240509  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:09.228183  9092 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.246043  9092 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:09.248669  6015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "048b40ab425f451c983713b7df96c732"
dest_uuid: "6add331326ca47038308a447bf798fa7"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:48972
I20260709 12:26:09.227926  9144 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:09.254293  8888 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:09.255995  8976 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:09.257776  9165 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 1.776s	user 0.010s	sys 0.006s
I20260709 12:26:09.258004  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:09.258724  5840 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.259434  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1242) took 2076 ms. Trace:
I20260709 12:26:09.259959  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:09.232937  9198 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:09.259903  5946 rpcz_store.cc:276] 0709 12:26:07.183364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.183553 (+   189us) service_pool.cc:224] Handling call
0709 12:26:07.190752 (+  7199us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.191009 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.191033 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.200556 (+  9523us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.389851 (+189295us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:07.389888 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.389985 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.256964 (+1866979us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:09.259076 (+  2112us) raft_consensus.cc:1685] finished
0709 12:26:09.259099 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.259358 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29956352,"thread_start_us":186626,"threads_started":1,"wal-append.queue_time_us":262071}
I20260709 12:26:09.261059  9214 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:09.262012  9214 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:09.262701  9214 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:09.244114  9171 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:09.231843  9202 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:09.245328  9167 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:09.268589  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:09.231652  9112 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:09.231389  8915 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:09.270704  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:09.233397  9203 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:09.233425  9213 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:09.273746  5840 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.275624  6611 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:09.276304  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.277218  9094 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:26:09.266880  9063 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:09.285342  9213 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:09.287222  9213 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:09.269865  9216 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:09.289374  9216 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:09.273119  8988 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:09.282383  6017 raft_consensus.cc:2410] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:09.290740  8988 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:09.291817  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:09.293376  6017 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:09.294445  9064 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:09.283733  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:09.279364  8936 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:09.295608  9203 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:09.270637  9211 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:09.289934  9216 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:09.296657  9225 raft_consensus.cc:2764] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:26:09.271652  9112 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:09.297221  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.297783  9112 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:09.298153  9203 raft_consensus.cc:2764] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:26:09.276335  9219 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:09.299528  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:09.274003  9180 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:09.302520  8915 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:09.303006  9203 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:09.298842  8615 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:09.304605  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:09.261446  9176 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:09.312067  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.313145  6762 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:09.314661  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:09.316898  9180 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:09.318959  9221 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:26:09.320186  9064 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:09.320840  9064 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:09.323344  9064 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:09.324535  6020 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:09.324874  9180 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.325492  9180 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:09.330335  9180 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:09.332206  9112 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:09.334679  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.335021  6015 raft_consensus.cc:606] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:09.335655  6015 raft_consensus.cc:614] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:09.337914  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:09.352138  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:09.352766  6017 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:09.358634  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:09.359470  8988 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:09.359964  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:09.360783  8615 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:09.361945  8871 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:09.362213  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.362393  8871 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:09.364648  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.351249  9112 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:09.350626  9180 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:09.370543  9180 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:09.371215  8615 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:09.372043  8615 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:09.372311  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.371097  8871 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:09.375437  8871 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:09.371186  9180 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:09.381217  8915 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:09.381994  8915 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:09.383688  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:09.384264  8915 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:09.384604  5866 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 8: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:09.384675  8615 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:09.386142  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:09.386379  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:09.387290  5866 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 13: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:09.388525  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:09.389171  5866 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:26:09.389395  9058 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:09.389500  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:09.390453  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:09.390386  9058 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:09.391712  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.391896  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:09.392901  9058 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:09.392997  6017 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:09.394575  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:09.396155  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:09.397526  8915 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:09.398026  8915 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:09.398423  8915 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:09.399029  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:09.399690  6024 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:09.400550  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:09.401104  6024 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:09.402011  8615 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:09.409291  8615 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:09.409984  8615 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:09.394057  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.403528  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:09.405231  5945 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:09.405278  5840 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.405999  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:09.406984  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:09.413388  5940 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:09.407615  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:09.402222  8988 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:09.415112  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:09.415187  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.418732  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:09.419013  5915 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:09.419714  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:26:09.420382  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.422047  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:09.422219  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:09.422713  5868 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:09.424129  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:09.427448  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:09.427495  5945 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:09.426116  9202 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:09.470974  9064 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:09.410897  9063 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:09.434074  5867 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:26:09.446959  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 53026.8us
I20260709 12:26:09.476893  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.478169  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.480675  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:09.482213  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.354633  9176 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
W20260709 12:26:09.485342  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56070.3us
W20260709 12:26:09.486529  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 58109.4us
I20260709 12:26:09.487617  5915 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:09.489889  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.490792  6775 raft_consensus.cc:2410] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:26:09.488152  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.488780  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.491402  6785 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:09.491922  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.487325  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.492861  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:09.474730  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
W20260709 12:26:09.498078  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.499575  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:09.503639  9063 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:09.504451  9063 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:09.505203  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:09.506067  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.506302  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:09.422631  8948 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:09.508977  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:09.510124  6785 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:09.514822  6775 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:09.508977  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.512301  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:09.510478  6762 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 13: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:09.428133  9203 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:09.431613  9215 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:09.515865  6768 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:09.408046  9198 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:09.473425  9064 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:09.475450  8976 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:09.418061  9144 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:09.432763  9199 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:09.475697  8936 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:09.421746  8904 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:09.525110  9225 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:09.521276  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.496398  9211 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:09.511803  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.597486  6793 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:09.474342  9214 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:09.537312  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.527415  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:09.544276  5868 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:09.545255  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:09.548079  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.551540  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:09.553789  6785 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:09.520998  9063 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:09.568152  9119 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:09.573577  6775 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:09.577004  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:09.519927  9202 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:09.526638  9167 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:09.520274  8948 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:09.526979  9215 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.601188  6017 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:09.497658  9112 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:09.500705  9205 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 12:26:09.571327  9199 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:09.604643  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 56625.2us
I20260709 12:26:09.538084  9140 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:26:09.630193  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.632412  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:09.565187  8976 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:09.607237  6757 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:09.600322  8936 raft_consensus.cc:697] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:09.648551  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 293279us
I20260709 12:26:09.615312  9059 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.621284  9215 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:09.612672  8842 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:09.642576  5874 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:09.660051  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.630121  9205 consensus_queue.cc:1243] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:26:09.631178  9211 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:09.637847  9207 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:09.651788  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.607522  8904 raft_consensus.cc:697] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:09.649765  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:09.615350  9063 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:09.654183  6785 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:09.674448  9063 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:09.677637  9203 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:09.678300  9203 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.678774  9203 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:09.636477  9122 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:09.636579  9199 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:09.684484  9122 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:09.685138  9122 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:09.643556  8976 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:09.657320  6757 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:09.657879  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.696684  6778 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:09.609535  9214 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:09.700855  9214 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:09.616652  8948 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:09.661098  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.657274  6019 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:09.646832  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.666218  5990 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.704201  6775 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:09.674803  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.608551  8516 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:09.719940  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 46692.9us
W20260709 12:26:09.721992  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 200808us
W20260709 12:26:09.721031  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 194450us
I20260709 12:26:09.725944  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:09.667917  9211 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:09.747445  6611 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:09.752946  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51044.9us
I20260709 12:26:09.747573  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.750456  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:09.757547  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:09.718533  9214 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:09.784865  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.663554  9059 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:09.792268  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.793402  6775 raft_consensus.cc:2393] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 11: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:09.788354  6778 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:09.752554  9211 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:09.704273  9216 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:09.740877  8945 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:26:09.763099  9225 consensus_queue.cc:1243] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:26:09.810318  6766 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:09.776084  9119 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:09.760672  9242 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:09.816409  6016 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:09.818435  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:09.787688  9202 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:26:09.826155  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 160000us
I20260709 12:26:09.822232  5838 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.824671  6778 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:09.827291  6778 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:09.828296  5990 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.831444  5990 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.761351  9171 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:26:09.817041  9243 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:09.835996  9243 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:09.836177  9235 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:26:09.836496  9243 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:09.756783  9224 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
W20260709 12:26:09.838816  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 79418.7us
I20260709 12:26:09.782652  8931 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:26:09.841609  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:09.787438  8888 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:26:09.756945  9241 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:26:09.842837  6024 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:09.818114  9119 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.847957  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:09.765961  9207 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:26:09.848763  6775 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:09.837679  6778 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:09.856323  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:09.857046  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.844502  9202 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:09.785284  9214 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:09.821034  9198 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:09.819311  9176 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:09.791102  9219 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:26:09.856353  6778 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:09.868678  6016 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
W20260709 12:26:09.876844  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 45490.3us
I20260709 12:26:09.821938  9205 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:26:09.786082  8936 consensus_queue.cc:237] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:09.874919  9233 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:26:09.851680  9119 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:09.843933  9144 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:09.939513  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 163543us
W20260709 12:26:09.952172  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 111941us
W20260709 12:26:09.942444  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 86194.8us
I20260709 12:26:09.855141  9250 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:09.939330  9254 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:26:09.939242  9261 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:26:09.947145  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 124976us
W20260709 12:26:09.949378  9238 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
W20260709 12:26:09.950320  9236 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 12:26:09.815881  8904 consensus_queue.cc:237] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:09.866967  9242 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:09.950390  9199 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:09.951519  5990 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.940078  9257 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:09.967068  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 108434us
I20260709 12:26:09.937789  8945 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:09.933037  9088 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:09.971716  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:09.975518  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:09.965507  9237 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
I20260709 12:26:09.989349  5990 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:09.873945  9180 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:09.987633  5915 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:09.999977  5990 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:09.969008  9144 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.001555  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:10.002537  5941 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:09.948066  9176 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:09.961905  9198 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:26:09.991930  9250 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:10.013101  5990 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:10.014935  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:10.015543  5942 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 5: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:09.993387  9241 consensus_queue.cc:1243] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:10.005095  9207 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:09.995664  8945 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:10.036168  5869 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 9: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:10.039016  9214 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:10.020402  9261 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.019773  9201 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:09.994160  8888 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:26:09.993976  9203 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:10.038779  9247 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:10.019671  9088 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
W20260709 12:26:10.062564  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 49518us
I20260709 12:26:10.067061  6016 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:10.012156  9251 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:10.070081  6757 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:10.040095  8931 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:10.064393  9240 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
W20260709 12:26:10.090652  9249 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:26:10.018496  9144 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:10.088891  9180 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:10.071167  9250 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:10.071934  9201 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:10.070051  9262 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:26:10.107187  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 115150us
I20260709 12:26:10.110095  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 653) took 1269 ms. Trace:
I20260709 12:26:10.112222  6023 rpcz_store.cc:276] 0709 12:26:08.831903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.834426 (+  2523us) service_pool.cc:224] Handling call
0709 12:26:10.101017 (+1266591us) spinlock_profiling.cc:243] Waited 74.9 ms on lock 0x7b5400172ad4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:10.101192 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172372736}
I20260709 12:26:10.111428  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:10.075955  9216 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:10.080672  9059 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:10.019263  9202 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:10.083921  9251 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.085465  8931 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:10.120416  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 166156us
W20260709 12:26:10.123560  9092 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.877s	user 0.010s	sys 0.002s
I20260709 12:26:10.124943  5866 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:10.125844  8871 consensus_meta.cc:287] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.750s	user 0.005s	sys 0.014s
I20260709 12:26:10.126523  8976 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:10.127955  9239 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.267s	user 0.000s	sys 0.003s
W20260709 12:26:10.128418  9244 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.249s	user 0.001s	sys 0.001s
I20260709 12:26:10.129838  6762 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
W20260709 12:26:10.145239  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.157668  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:10.114859  9201 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:10.149119  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:10.090471  8945 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:10.160732  6778 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
W20260709 12:26:10.161743  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 157565us
I20260709 12:26:10.165777  6778 pending_rounds.cc:85] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 4
I20260709 12:26:10.166604  6778 pending_rounds.cc:107] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:26:10.167021  6778 raft_consensus.cc:2889] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:10.119180  9250 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:10.167595  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:10.167654  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:10.168411  6023 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:10.138331  8931 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:10.129194  9243 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:10.178084  5949 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:10.176508  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:10.173290  5866 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:26:10.168834  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:10.131665  9251 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:10.190076  6778 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:10.076967  9261 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:10.204430  9064 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:10.205140  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:26:10.205533  8915 consensus_meta.cc:287] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.807s	user 0.013s	sys 0.000s
I20260709 12:26:10.206157  8915 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:10.207352  8842 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:10.208051  8842 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:10.208698  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:10.209558  5868 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 9: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:10.210510  9243 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:10.211253  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:10.211627  9256 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:26:10.212867  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:10.213076  9211 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:10.214186  6768 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:10.214098  8842 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:10.189221  9122 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.502s	user 0.012s	sys 0.001s
I20260709 12:26:10.215469  5874 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:10.142828  9092 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:10.225782  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:10.225924  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:10.226262  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:10.251695  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:10.189918  6757 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:10.252481  6757 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:10.235314  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:10.076591  9242 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:10.269042  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 659) took 1433 ms. Trace:
I20260709 12:26:10.270395  6020 rpcz_store.cc:276] 0709 12:26:08.835613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.983658 (+148045us) service_pool.cc:224] Handling call
0709 12:26:10.268819 (+1285161us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54001716d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:10.268968 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556617088}
W20260709 12:26:10.231345  9244 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
W20260709 12:26:10.228299  9239 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:26:10.188280  9070 consensus_queue.cc:1059] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.085s
I20260709 12:26:10.278925  8871 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:10.281292  5838 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:10.290300  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:10.291378  5947 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 5.
I20260709 12:26:10.271615  6757 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:10.244048  9199 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:10.243685  9171 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.281872  9266 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:10.338191  9266 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:10.341012  9266 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:10.247980  9185 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:10.240253  9276 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:10.324179  5838 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:10.284723  9215 consensus_meta.cc:287] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.618s	user 0.010s	sys 0.003s
I20260709 12:26:10.327872  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:10.328438  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:10.296662  9176 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:10.331202  9092 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:10.331511  9211 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:10.360733  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:10.333139  8948 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.631s	user 0.011s	sys 0.000s
W20260709 12:26:10.334079  8615 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.924s	user 0.015s	sys 0.000s
I20260709 12:26:10.371080  8615 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:10.343358  9185 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:10.372884  6024 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:10.334478  6757 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:10.334580  8994 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:10.259161  9122 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:10.286492  8871 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:10.351725  6773 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:10.283145  9203 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:10.285024  9144 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:10.369953  6023 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:10.245595  9119 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:10.374745  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 140948us
W20260709 12:26:10.375171  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 262730us
I20260709 12:26:10.345203  8915 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:10.341600  8976 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:10.374709  9070 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:10.449295  6017 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:10.448158  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:10.450353  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:10.376613  9094 raft_consensus.cc:3055] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:10.434377  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:10.378712  9196 consensus_queue.cc:1059] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:26:10.445658  9280 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:26:10.463989  8615 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:10.473260  9058 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:26:10.474064  9058 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:10.476684  6023 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:10.416946  9176 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.449169  9070 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.282312  9250 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:10.462008  9140 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:10.462239  9122 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:10.462742  9257 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:10.466925  6611 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:10.464418  9247 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:26:10.469991  9245 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:10.422464  9092 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.484943  6757 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:10.491752  9070 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:10.354449  9171 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:10.543973  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:10.545665  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:10.546916  6757 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:10.469530  8957 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:26:10.477234  9211 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:10.490352  9058 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:10.376955  8948 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:10.353291  9215 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:10.459896  9094 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:10.497232  9140 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:10.498020  8915 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:10.530933  9276 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.491469  8888 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:26:10.536059  9277 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:10.585286  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:10.494875  9176 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:10.540271  9185 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:10.543438  9224 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.549209  9122 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:10.583197  9058 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:10.609179  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 108720us
I20260709 12:26:10.545400  9092 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:10.614431  5913 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:10.608800  9276 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:10.641994  9094 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:10.642957  9276 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:10.616897  9122 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:10.643323  9094 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:10.643491  9201 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:10.645226  9122 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:10.613070  9224 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:10.647367  9251 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:10.647943  9224 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:10.616190  9277 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:10.649937  6015 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 12:26:10.635704  8994 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.636232  9159 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:10.606460  8915 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:10.608533  8615 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:10.635814  9174 raft_consensus.cc:3055] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:10.635708  9059 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.637797  9058 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:10.637696  9140 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:10.639753  8948 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:10.640687  9176 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:10.652204  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:10.653328  6016 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:26:10.678128  6766 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:10.678805  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:10.679471  6016 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:10.680712  6611 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:10.639786  9171 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:10.641997  9092 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:10.654582  9174 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:10.652879  9225 raft_consensus.cc:3055] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:10.695698  6757 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:10.650074  8706 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:10.649379  9277 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:10.708887  8706 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:10.720249  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:10.721195  6766 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:10.702005  9058 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:10.688643  9251 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:10.687850  9140 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:10.658282  8615 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:10.661877  9059 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:10.720479  8706 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:10.737468  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:10.659325  8994 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:10.698688  8957 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:10.759500  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:10.760845  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:10.764801  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 108714us
W20260709 12:26:10.688233  8888 consensus_queue.cc:1175] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7a0204b931394fc48ffc503716e0d111" responder_term: 9 status { last_received { term: 9 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 9 index: 5 } }
W20260709 12:26:10.747789  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 55012.3us
I20260709 12:26:10.774130  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:10.776113  5990 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:10.706406  9225 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:10.754421  6016 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:10.708899  9119 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:10.768708  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:10.778178  5990 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:10.746780  6766 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:10.758491  8957 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.790385  6016 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:10.779906  5913 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:10.791687  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:10.794699  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:10.796310  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:10.797004  6785 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:10.797573  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:10.798377  5941 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:10.798568  5944 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:10.801903  9287 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:26:10.802457  9167 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:10.803073  9167 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:10.804082  9289 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
W20260709 12:26:10.805941  9283 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 12:26:10.807616  9291 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:26:10.808439  9296 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:10.808276  6775 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:10.809079  9296 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.809613  9296 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:10.804373  9167 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:10.811424  9167 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:10.813452  9059 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:10.801270  9119 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:10.814435  9119 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:10.815683  9119 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:10.817976  9254 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:10.819090  8615 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:10.822741  9070 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:10.793910  8957 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:10.828323  8871 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:10.780972  8706 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:10.829198  9225 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:10.783589  9251 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:10.837980  9215 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:10.798569  9201 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:10.840039  8994 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:10.768401  9174 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:10.841500  9288 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.205s	user 0.000s	sys 0.004s
I20260709 12:26:10.802970  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:10.843822  5942 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:10.847959  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 56341.4us
I20260709 12:26:10.848860  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:10.850483  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:10.783231  9122 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:10.851581  5941 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 5.
I20260709 12:26:10.825385  9070 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:10.855413  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:10.852939  5990 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:10.885460  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.887288  5940 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:10.856606  5944 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:10.795280  9092 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:10.912343  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:10.892853  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:26:10.828413  9290 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:10.784842  9292 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 12:26:10.930589  9292 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:10.931073  9292 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:10.932010  9292 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:10.898293  9285 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.262s	user 0.000s	sys 0.004s
I20260709 12:26:10.905198  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1263) took 1281 ms. Trace:
I20260709 12:26:10.935111  5949 rpcz_store.cc:276] 0709 12:26:09.623892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.624043 (+   151us) service_pool.cc:224] Handling call
0709 12:26:09.624364 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.178678 (+554314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.904923 (+726245us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b54003089d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:10.905126 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1266264320}
I20260709 12:26:10.936789  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:10.792991  9159 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:10.905825  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.905825  9251 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:10.938449  9251 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:10.906599  9112 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:10.907742  9246 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:26:10.785334  8976 consensus_queue.cc:1059] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.101s
I20260709 12:26:10.908538  8615 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:10.909230  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:10.910175  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:10.910496  5948 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:10.953974  9070 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:10.955509  9070 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:10.956254  5871 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:10.933482  9292 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:10.912376  8957 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:10.913350  5872 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:10.914397  5867 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:10.794255  9144 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.121s
I20260709 12:26:10.915911  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:10.924978  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:10.893649  9254 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:10.908349  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:10.958787  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:10.951970  6785 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
W20260709 12:26:10.912194  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.969089  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 44227.1us
I20260709 12:26:10.974922  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:10.944428  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:10.918812  9092 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:10.939097  9122 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:10.978614  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:10.983747  6766 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:10.984721  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:10.985931  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:10.986657  5941 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:10.976482  5870 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:10.939592  8994 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.799996  9224 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.146s
I20260709 12:26:10.978103  5867 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:10.958300  9276 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:10.958029  9203 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.938323  9159 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:11.004038  9203 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:11.005956  6773 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 874) took 1529 ms (client timeout 1572 ms). Trace:
I20260709 12:26:10.980819  9252 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:26:10.985674  5990 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:10.953773  9174 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:10.988652  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:10.988889  5940 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:10.970209  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:11.016203  5944 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:26:10.994946  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:10.943030  9257 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:10.999469  9122 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:11.002727  8976 raft_consensus.cc:3055] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:10.993885  9058 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:11.027632  9224 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.028329  9224 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:11.003607  9247 raft_consensus.cc:3055] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:11.029677  9247 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:11.030686  9224 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:11.006469  6773 rpcz_store.cc:269] 0709 12:26:09.476808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.480312 (+  3504us) service_pool.cc:224] Handling call
0709 12:26:11.005735 (+1525423us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b54002a0dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:11.005886 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1539638144}
I20260709 12:26:11.008045  6766 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:10.967886  9176 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:11.003664  9276 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:11.016928  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:10.993898  9092 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:11.019125  5940 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:11.006649  9203 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:11.086200  9224 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:26:11.021147  8931 consensus_meta.cc:287] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.832s	user 0.016s	sys 0.000s
I20260709 12:26:11.022209  9198 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:10.958393  9112 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:11.024371  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:11.022847  9257 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:11.027110  9122 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:11.027281  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.034874  5838 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:10.976675  8904 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:26:10.969941  8615 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:11.033084  8976 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:11.014349  8915 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:11.030956  9247 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:11.068445  8957 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:11.037249  5942 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:11.069247  9176 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:11.081748  9195 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:26:11.101310  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 66561.2us
I20260709 12:26:10.983944  9254 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:11.111423  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.114514  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:11.116715  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:11.035959  9070 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:11.007864  8994 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:11.146096  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.097770  9296 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:11.305174  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:11.306000  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.210543  5940 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:11.182041  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:11.098889  9198 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.218209  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:11.157205  9203 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:11.262105  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:11.173346  9094 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:11.246495  9245 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.001s	sys 0.012s
I20260709 12:26:11.023927  9251 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:11.272883  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:11.274647  9092 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:11.326709  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:11.327105  5947 raft_consensus.cc:2368] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13.
I20260709 12:26:11.281903  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:11.283118  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:11.122928  8931 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:11.287456  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 793) took 1632 ms. Trace:
I20260709 12:26:11.334446  6762 rpcz_store.cc:276] 0709 12:26:09.659251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.659756 (+   505us) service_pool.cc:224] Handling call
0709 12:26:11.286777 (+1627021us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b5400007b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:11.287305 (+   528us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1078581632}
I20260709 12:26:11.287648  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 671) took 1429 ms. Trace:
I20260709 12:26:11.337113  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:11.219424  9122 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:11.339563  9070 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:11.340024  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:11.340809  9070 raft_consensus.cc:3055] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:11.179378  8915 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.341701  8915 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:11.326366  9094 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:11.340803  6019 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:11.318861  9245 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.002s	sys 0.012s
I20260709 12:26:11.348330  8615 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:11.341316  9070 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:11.143112  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:11.315382  5941 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:26:11.317672  9203 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.317795  9198 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:11.327224  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:11.328883  5865 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 9: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:11.285815  5872 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:26:11.331032  9140 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.588s	user 0.004s	sys 0.010s
W20260709 12:26:11.359848  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 216904us
I20260709 12:26:11.361361  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:11.309383  5942 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 7: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:11.264770  9247 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:11.349336  8615 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:11.332909  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:11.365353  5870 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 12: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:11.365424  5946 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:11.335930  6017 rpcz_store.cc:276] 0709 12:26:09.857887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.127962 (+270075us) service_pool.cc:224] Handling call
0709 12:26:10.128328 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.450119 (+321791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.286733 (+836614us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b5400160654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:11.287395 (+   662us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":737317248}
I20260709 12:26:11.341347  9296 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:11.368472  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.368943  5945 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:11.374054  8615 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:11.375002  9063 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.375401  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:11.376210  5872 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:11.375681  8615 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:11.377264  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:11.359759  9119 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.015s	sys 0.008s
I20260709 12:26:11.378893  5872 raft_consensus.cc:2376] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13. Re-sending same reply.
I20260709 12:26:11.386881  5838 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:11.365834  9203 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:11.366703  9242 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:26:11.419178  8936 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.008s	sys 0.019s
I20260709 12:26:11.413393  6019 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:11.378489  9058 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:11.381973  9176 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:11.401579  9250 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.009s	sys 0.040s
W20260709 12:26:11.407323  9219 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.002s	sys 0.017s
W20260709 12:26:11.384024  9119 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.015s	sys 0.009s
I20260709 12:26:11.381271  9094 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:11.438840  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:11.468019  6016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.005s	sys 0.013s
W20260709 12:26:11.475198  5868 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.000s	sys 0.024s
I20260709 12:26:11.338469  8976 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:11.375742  9063 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:11.435822  8936 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.009s	sys 0.019s
W20260709 12:26:11.466281  9219 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.003s	sys 0.017s
I20260709 12:26:11.434473  9112 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:11.463131  9250 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.010s	sys 0.040s
W20260709 12:26:11.482131  5868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.000s	sys 0.025s
W20260709 12:26:11.484292  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 498740us
I20260709 12:26:11.467484  9176 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:11.490324  9185 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.009s	sys 0.017s
I20260709 12:26:11.522406  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:11.474022  9276 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:11.524479  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:11.532912  6018 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:11.530853  9185 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.009s	sys 0.018s
I20260709 12:26:11.497162  8976 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:11.525743  9119 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:11.548388  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 161657us
I20260709 12:26:11.489866  9122 raft_consensus.cc:697] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:11.461834  9070 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:11.550865  6773 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
W20260709 12:26:11.564087  6020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.014s	sys 0.020s
W20260709 12:26:11.564617  6020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.014s	sys 0.020s
I20260709 12:26:11.542831  9276 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:11.561093  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 174180us
W20260709 12:26:11.529482  5987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.013s	sys 0.015s
W20260709 12:26:11.556926  9159 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.029s	sys 0.043s
I20260709 12:26:11.557421  8904 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:26:11.591223  6021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.017s	sys 0.012s
W20260709 12:26:11.594581  5987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.013s	sys 0.016s
I20260709 12:26:11.596910  5990 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:11.601231  6021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.018s	sys 0.012s
I20260709 12:26:11.602427  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:11.603122  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 628275us
I20260709 12:26:11.531590  9176 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:11.596096  6609 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:11.603180  5874 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:11.562806  9119 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:11.593573  8994 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:11.598153  9159 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.029s	sys 0.043s
I20260709 12:26:11.587859  9112 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
W20260709 12:26:11.658036  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 62018.9us
I20260709 12:26:11.640832  5874 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:11.641829  6773 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:11.624547  8904 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:11.624202  8931 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:11.601961  9276 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:11.659523  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:26:11.645825  9292 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.672s	user 0.011s	sys 0.004s
I20260709 12:26:11.643278  9264 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:11.673802  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:11.653882  9251 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:11.680449  9286 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.045s	user 0.005s	sys 0.007s
W20260709 12:26:11.682435  9308 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 12:26:11.673410  9306 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.190s	user 0.001s	sys 0.001s
I20260709 12:26:11.698048  9063 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:11.700867  9176 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:11.645640  9122 consensus_queue.cc:237] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:11.673285  9323 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:11.664744  9241 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:11.661475  9322 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:11.709386  9094 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:11.712513  9313 log.cc:927] Time spent T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:26:11.706359  9251 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:11.730226  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:11.739501  5945 raft_consensus.cc:2393] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:11.740378  9251 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:11.707717  9250 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:26:11.741729  9322 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:11.729844  8904 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:11.742846  9322 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:11.744249  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:11.728323  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.746814  6019 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:11.747049  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:11.731288  5941 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:11.737125  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 897) took 1460 ms. Trace:
I20260709 12:26:11.730613  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:11.677242  9213 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:11.747079  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:11.728590  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.749541  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:11.747816  6016 raft_consensus.cc:2376] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9. Re-sending same reply.
W20260709 12:26:11.712042  9307 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 12:26:11.754103  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:11.755568  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.749755  6766 rpcz_store.cc:276] 0709 12:26:10.277049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.984631 (+707582us) service_pool.cc:224] Handling call
0709 12:26:11.007656 (+ 23025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.034834 (+ 27178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.736758 (+701924us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b54002a0dd0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:11.737028 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52227328}
I20260709 12:26:11.714619  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:11.790889  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:11.790127  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:11.791258  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:11.791868  6020 raft_consensus.cc:2376] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9. Re-sending same reply.
I20260709 12:26:11.715262  5874 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:11.758931  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:11.774339  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.792359  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:11.793066  6766 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:11.796576  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:11.754910  9167 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.938s	user 0.016s	sys 0.000s
I20260709 12:26:11.758931  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:11.732693  9174 consensus_meta.cc:287] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.713s	user 0.008s	sys 0.005s
I20260709 12:26:11.809087  6766 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
W20260709 12:26:11.798535  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.799043  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 912) took 2738 ms. Trace:
I20260709 12:26:11.792956  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:11.727644  9241 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.800573  9094 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.802611  6016 raft_consensus.cc:2410] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:26:11.811025  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:11.817549  6019 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:11.818979  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:11.811306  6021 rpcz_store.cc:276] 0709 12:26:09.060867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.111834 (+ 50967us) service_pool.cc:224] Handling call
0709 12:26:09.112193 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.112446 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:09.232271 (+119825us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.957527 (+1725256us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b180026e7e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:26:11.797896 (+840369us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.798117 (+   221us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.798172 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:11.798230 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.798837 (+   607us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.798986 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":163993,"prepare.queue_time_us":118990,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":12058,"raft.queue_time_us":54750,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"spinlock_wait_cycles":11164672,"thread_start_us":782567,"threads_started":3,"wal-append.queue_time_us":2413091}
I20260709 12:26:11.796705  8706 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.858062  5869 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:26:11.836563  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:11.823469  6020 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:11.863531  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:11.862725  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.866382  6766 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:11.727933  8871 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 12:26:11.801694  9114 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:11.804261  9159 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:11.880420  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 89264.1us
I20260709 12:26:11.881572  8871 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.815392  9063 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.822228  9094 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:11.884337  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:11.815999  9119 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:11.821988  9241 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:11.814432  9247 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:11.795094  9112 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 12:26:11.859095  9251 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.805495  9213 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:11.881503  9332 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
W20260709 12:26:11.869380  9302 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:26:11.888418  6766 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:11.822386  9176 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:11.912079  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.751068  9323 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:11.840190  8904 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:11.920142  6766 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:11.869781  9325 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:11.862152  8706 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:11.932775  6766 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:11.936532  6017 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:11.920147  8888 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:11.941277  8706 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:11.921345  9085 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:26:11.964025  6017 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:11.887907  8871 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:11.931739  9251 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:11.890211  9247 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:11.933192  9176 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:11.943353  9261 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:11.936323  8936 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:26:11.905047  9063 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:11.958204  9334 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:11.977490  9198 consensus_meta.cc:287] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.599s	user 0.014s	sys 0.000s
I20260709 12:26:11.933055  8904 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:11.936631  9325 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:11.989296  9250 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.006114  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 894) took 1850 ms. Trace:
I20260709 12:26:12.014816  6778 rpcz_store.cc:276] 0709 12:26:10.156073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.199046 (+ 42973us) service_pool.cc:224] Handling call
0709 12:26:10.199397 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.227497 (+ 28100us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.227527 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.527061 (+1299534us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b2c0001ffa8. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:26:11.527476 (+   415us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.631550 (+104074us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:11.631592 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.631675 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.999384 (+367709us) raft_consensus.cc:1685] finished
0709 12:26:11.999420 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.006012 (+  6592us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":128974,"raft.queue_time_us":24714,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":18624,"spinlock_wait_cycles":3083136,"thread_start_us":73917,"threads_started":1,"wal-append.queue_time_us":158424}
W20260709 12:26:12.000757  9203 consensus_meta.cc:287] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.566s	user 0.014s	sys 0.000s
I20260709 12:26:11.999037  9218 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:12.014101  8871 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:12.032405  9257 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:12.034386  9276 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:12.041831  6609 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:12.041989  6778 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:12.043594  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:12.045790  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:12.047148  6609 leader_election.cc:400] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13.
I20260709 12:26:12.049831  9196 raft_consensus.cc:3055] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:12.049507  9272 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:26:12.050714  9299 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.651s	user 0.000s	sys 0.002s
I20260709 12:26:12.051221  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 951) took 2783 ms. Trace:
I20260709 12:26:12.051403  9272 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:12.051661  5946 rpcz_store.cc:276] 0709 12:26:09.268026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.205207 (+937181us) service_pool.cc:224] Handling call
0709 12:26:10.205613 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.366779 (+1161166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.050925 (+684146us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400530f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:12.051151 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2666497408}
I20260709 12:26:12.052337  9272 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.053005  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 673) took 1543 ms. Trace:
I20260709 12:26:12.050429  9196 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:12.054636  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 788) took 2408 ms. Trace:
I20260709 12:26:12.054889  9196 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:12.049355  9241 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:12.059641  5866 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 909) took 2669 ms (client timeout 1561 ms). Trace:
W20260709 12:26:12.060156  5866 rpcz_store.cc:269] 0709 12:26:09.390110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.390224 (+   114us) service_pool.cc:224] Handling call
0709 12:26:12.059343 (+2669119us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b54006db5d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:12.059511 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1687174528}
I20260709 12:26:12.060736  9241 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:12.061599  6773 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:26:12.056243  9196 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:12.063508  9063 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:12.064702  5873 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:12.067269  8904 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:12.052927  9272 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:12.071149  9325 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:12.053507  6022 rpcz_store.cc:276] 0709 12:26:10.509689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.648987 (+139298us) service_pool.cc:224] Handling call
0709 12:26:10.649334 (+   347us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:10.655249 (+  5915us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.655281 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:10.658775 (+  3494us) log.cc:844] Serialized 52 byte log entry
0709 12:26:11.166284 (+507509us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:11.259736 (+ 93452us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.739104 (+479368us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b18002ce600. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb10c0a 00007f39d9250219 00007f39cfb98a40 00007f39cfb934e4 00007f39cfb93455 00007f39d667b3c7 00007f39d2c8d9a7 00007f39d2c9d0bf 00007f39d2c9d038
0709 12:26:11.739335 (+   231us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.739564 (+   229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.739655 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.050070 (+310415us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:12.052771 (+  2701us) raft_consensus.cc:1685] finished
0709 12:26:12.052792 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.052947 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":473985,"prepare.queue_time_us":591490,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":84,"raft.queue_time_us":229473,"raft.run_cpu_time_us":514,"raft.run_wall_time_us":21318,"spinlock_wait_cycles":98198528,"thread_start_us":460162,"threads_started":1,"wal-append.queue_time_us":713758}
W20260709 12:26:12.058254  9333 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:26:12.075148  6768 rpcz_store.cc:276] 0709 12:26:09.645910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.646081 (+   171us) service_pool.cc:224] Handling call
0709 12:26:09.656743 (+ 10662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.214725 (+557982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.054410 (+1839685us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b54005294d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:12.054578 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1281520000}
I20260709 12:26:12.088135  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.097285  5871 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:26:12.040004  9058 consensus_meta.cc:287] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.602s	user 0.014s	sys 0.001s
I20260709 12:26:12.071694  9272 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:12.027861  9218 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:12.012336  9198 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:12.008251  9334 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:12.040596  9247 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:11.999699  9213 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:12.111593  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:12.072377  8539 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:12.109248  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.023860  9203 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:12.106277  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.135672  5866 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:12.135910  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.136547  5871 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:12.138104  5872 raft_consensus.cc:2393] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:26:12.138762  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.021936  9250 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:12.147401  6609 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:12.105206  9058 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:12.099574  9251 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:12.113445  8904 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.163712  5870 raft_consensus.cc:2368] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:26:12.100168  9059 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:26:12.190853  5866 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:12.192363  6793 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 5.
I20260709 12:26:12.192896  6609 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:12.161954  8539 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:12.166726  8663 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:12.201274  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.158432  9323 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:12.152666  9063 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:12.098559  9325 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 12:26:12.207522  9341 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.208281  9059 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.208231  9341 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:12.209048  9059 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:12.210336  9337 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:26:12.211804  9334 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:12.208245  9339 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:26:12.213160  9250 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:12.214123  9176 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:12.214711  9341 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:12.216193  9059 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:12.216794  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 1186) took 1276 ms. Trace:
I20260709 12:26:12.201025  8539 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.218405  8539 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:12.222864  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 669) took 2570 ms. Trace:
I20260709 12:26:12.223515  6024 rpcz_store.cc:276] 0709 12:26:09.652376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.847380 (+195004us) service_pool.cc:224] Handling call
0709 12:26:09.847702 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.373324 (+525622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.222386 (+1849062us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b54006b8fd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:12.222758 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1207181696}
I20260709 12:26:12.232219  9214 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:12.232746  9214 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:12.233596  9214 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:12.234686  9214 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:12.235054  9198 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:12.241981  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.217343  5869 rpcz_store.cc:276] 0709 12:26:10.940259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.953496 (+ 13237us) service_pool.cc:224] Handling call
0709 12:26:12.216595 (+1263099us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b54002f94d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:12.216722 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2035235584}
I20260709 12:26:12.258234  5872 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:26:12.258340  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.260561  5866 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:12.260885  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:12.262032  5869 raft_consensus.cc:2376] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7. Re-sending same reply.
I20260709 12:26:12.197052  9277 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:12.267961  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:12.278810  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 923) took 2241 ms. Trace:
I20260709 12:26:12.280418  6015 rpcz_store.cc:276] 0709 12:26:10.036987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.287062 (+250075us) service_pool.cc:224] Handling call
0709 12:26:10.287448 (+   386us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:10.651219 (+363771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.278039 (+1626820us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b5400171bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:12.278691 (+   652us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832939264}
I20260709 12:26:12.283811  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:12.285245  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.304829  9159 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:12.258574  9063 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:12.199231  9218 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:12.310850  9218 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:12.278668  9092 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:12.324748  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.339887  9264 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:12.340694  9264 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:12.356977  9092 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.357957  9092 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:12.361150  9214 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.361716  9092 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:12.362138  9214 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:12.364743  9214 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:12.364435  9264 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:12.371711  9203 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:12.371922  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.374612  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 528) took 2981 ms. Trace:
I20260709 12:26:12.373027  9264 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:12.375658  9316 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.676s	user 0.001s	sys 0.002s
I20260709 12:26:12.375203  5873 rpcz_store.cc:276] 0709 12:26:09.393293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.393513 (+   220us) service_pool.cc:224] Handling call
0709 12:26:12.064460 (+2670947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.065223 (+   763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.374335 (+309112us) spinlock_profiling.cc:243] Waited 2.67 s on lock 0x7b54006db5d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:12.374529 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6141412224}
I20260709 12:26:12.380321  5865 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:12.380749  9144 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:12.381414  9144 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:12.382302  6024 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:12.384688  6024 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 6. Current term is 7. Ops: []
W20260709 12:26:12.385923  9298 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.994s	user 0.003s	sys 0.002s
I20260709 12:26:12.387521  9292 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.388551  6024 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:12.388231  9292 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:12.389961  9303 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.992s	user 0.005s	sys 0.000s
I20260709 12:26:12.389981  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:12.391037  9305 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.994s	user 0.004s	sys 0.001s
I20260709 12:26:12.391273  6015 raft_consensus.cc:2393] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:12.391356  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:12.392309  6022 raft_consensus.cc:2393] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:12.392678  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 895) took 2217 ms. Trace:
W20260709 12:26:12.385365  9327 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.683s	user 0.001s	sys 0.004s
I20260709 12:26:12.393056  6775 rpcz_store.cc:276] 0709 12:26:10.175063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.809988 (+634925us) service_pool.cc:224] Handling call
0709 12:26:10.810360 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.818105 (+  7745us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.818142 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.818849 (+   707us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.205744 (+386895us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:11.205791 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.205875 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.331281 (+1125406us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:12.392321 (+ 61040us) raft_consensus.cc:1685] finished
0709 12:26:12.392347 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.392590 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42127,"raft.queue_time_us":8578,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":98,"spinlock_wait_cycles":76972800,"thread_start_us":347463,"threads_started":1,"wal-append.queue_time_us":577590}
I20260709 12:26:12.375018  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.395354  8945 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:12.399305  9292 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:12.400043  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 924) took 2362 ms. Trace:
I20260709 12:26:12.400236  6017 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:12.400483  6023 rpcz_store.cc:276] 0709 12:26:10.037286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.425461 (+388175us) service_pool.cc:224] Handling call
0709 12:26:10.425892 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.477524 (+ 51632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.399800 (+1922276us) spinlock_profiling.cc:243] Waited 50.4 ms on lock 0x7b5400172ad4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:12.399978 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116028672}
W20260709 12:26:12.404753  9094 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.002s
I20260709 12:26:12.308614  9063 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:12.413722  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:12.416508  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:12.475862  5873 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:12.419150  5866 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:12.419553  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.421109  5871 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:12.383724  9144 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:12.428443  5838 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.195:43329 timed out after 1.512s (SENT) [suppressed 29 similar messages]
W20260709 12:26:12.429569  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.433503  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:12.482788  6773 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:12.483721  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:12.432780  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:12.484951  6775 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:12.483280  6611 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:12.435714  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 925) took 2398 ms. Trace:
I20260709 12:26:12.485910  6018 rpcz_store.cc:276] 0709 12:26:10.037583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.426294 (+388711us) service_pool.cc:224] Handling call
0709 12:26:10.457470 (+ 31176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.533409 (+1075939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.435450 (+902041us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400562554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:12.435644 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2472340352}
I20260709 12:26:12.425408  9277 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:12.436976  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:12.487202  6793 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 6.
I20260709 12:26:12.488075  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:12.488886  6793 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 10: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:12.432729  8957 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.438508  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:12.440539  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:12.440706  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:12.489799  6778 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:12.445508  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:12.453926  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:12.455297  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:12.490787  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:12.490959  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:12.459720  9092 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:12.460516  9251 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:12.460538  9250 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:12.500185  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:12.500412  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:12.195233  8904 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:12.461997  9094 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:12.463557  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:26:12.512488  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 49000.3us
I20260709 12:26:12.511245  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:12.514343  6762 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:12.464335  8915 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:12.464182  9322 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:26:12.465317  6021 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.003s	sys 0.007s
W20260709 12:26:12.467437  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.520841  6021 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.003s	sys 0.007s
I20260709 12:26:12.493606  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:12.493727  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:12.495167  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:12.495678  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:12.496871  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:12.449733  9252 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:12.507474  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.013s	sys 0.003s
I20260709 12:26:12.510366  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:12.534859  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.013s	sys 0.004s
I20260709 12:26:12.517000  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:26:12.535410  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 606902us
I20260709 12:26:12.535995  5946 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 6: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:12.518453  5948 raft_consensus.cc:3055] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:26:12.521546  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.536610  5944 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 10: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:12.537042  5840 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:12.536902  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:12.537866  5945 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:26:12.521062  9144 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:12.543167  5872 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:12.543375  5915 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:12.498093  9292 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.544397  5915 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:12.536478  5948 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:12.545677  5948 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:12.546708  5948 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:12.540678  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:12.512010  9224 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:12.511176  9261 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:12.561141  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.559731  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.563628  5915 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:12.566881  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.551024  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:12.539261  9214 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.503428  9258 consensus_queue.cc:1059] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:26:12.571147  6775 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:12.541173  9259 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.578008  5947 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:12.575073  6611 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:12.549185  9128 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:12.499799  8957 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:12.507113  9092 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:12.517079  9250 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:12.551406  9276 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:12.545374  9176 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:12.560949  8931 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:12.534258  9122 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:26:12.586387  5942 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:12.596395  5942 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:12.542562  9252 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:12.590233  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.545408  9159 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:26:12.512248  9251 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:12.546298  9334 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:26:12.552066  9144 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:12.622066  9144 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:12.623274  9094 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:12.624990  5948 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:26:12.625608  5948 pending_rounds.cc:85] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 5
I20260709 12:26:12.578060  9214 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:12.628124  9214 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:12.630259  9122 raft_consensus.cc:3055] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:12.630975  9122 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:12.586239  8663 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:26:12.632149  9122 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:12.570864  9261 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:12.636360  9261 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:12.591795  9348 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:26:12.637769  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 513) took 1523 ms. Trace:
I20260709 12:26:12.638221  5940 rpcz_store.cc:276] 0709 12:26:11.114365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.114581 (+   216us) service_pool.cc:224] Handling call
0709 12:26:11.210278 (+ 95697us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.307457 (+ 97179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.637475 (+1330018us) spinlock_profiling.cc:243] Waited 95.2 ms on lock 0x7b54004678d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:12.637683 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218883328}
I20260709 12:26:12.639106  9250 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:12.576802  9277 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:12.640406  5949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1280) took 1712 ms (client timeout 1512 ms). Trace:
W20260709 12:26:12.640887  5949 rpcz_store.cc:269] 0709 12:26:10.927800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.936454 (+  8654us) service_pool.cc:224] Handling call
0709 12:26:12.640122 (+1703668us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b5400467dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:12.640280 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1681583872}
I20260709 12:26:12.643113  9225 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:12.565610  9345 consensus_queue.cc:1059] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:26:12.561169  8965 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:26:12.600485  6766 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:12.606663  6611 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:12.616066  5987 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.561s (SENT)
I20260709 12:26:12.654614  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:12.594029  8915 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:12.657943  5873 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:12.658517  9250 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:12.594352  9176 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:12.659708  9176 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:12.619975  8904 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:12.620635  9251 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:12.625954  5948 pending_rounds.cc:107] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:26:12.555748  9292 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:12.666307  5948 raft_consensus.cc:2889] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:12.667229  5943 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:12.668633  9292 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:12.597698  5942 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:12.590727  9205 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:12.669762  9205 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.670058  5942 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:12.670253  9205 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:12.595048  9128 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:12.674787  9343 raft_consensus.cc:3055] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:12.675256  9343 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:12.675073  9128 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:12.675729  9345 raft_consensus.cc:3055] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:12.676498  9345 raft_consensus.cc:740] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:12.585326  9259 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:12.682062  9092 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:12.687302  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 80757.7us
I20260709 12:26:12.689085  9246 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:12.673544  8539 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:12.650496  9213 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:12.674077  9122 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:12.696008  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 145066us
I20260709 12:26:12.696952  5840 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:12.697839  8904 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:12.698014  9345 consensus_queue.cc:260] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:12.653844  9224 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:12.699604  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:12.699852  9357 consensus_queue.cc:1059] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:26:12.700942  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 111868us
I20260709 12:26:12.636546  9252 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:12.706837  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 278380us
I20260709 12:26:12.710306  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:12.731585  5942 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:26:12.732300  5942 pending_rounds.cc:85] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 5
I20260709 12:26:12.732609  5942 pending_rounds.cc:107] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:26:12.732937  5942 raft_consensus.cc:2889] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:12.733790  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 131950us
I20260709 12:26:12.737382  6757 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:12.742949  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.745175  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.746317  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:12.746345  5866 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:12.747605  5871 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:12.751681  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.752094  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.753149  5866 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:26:12.754040  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:12.754462  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:12.755312  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:12.756238  6793 raft_consensus.cc:2410] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:26:12.756278  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:12.752424  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:12.758378  6773 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:26:12.757269  6775 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:26:12.760567  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:12.761513  6775 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
W20260709 12:26:12.763595  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 55171.7us
I20260709 12:26:12.765091  6762 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 11: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:12.700173  9224 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:12.772815  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 68644.2us
I20260709 12:26:12.773903  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.777173  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.781082  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.740082  8904 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:12.789203  5865 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:12.790571  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.791589  5866 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:12.794354  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.795173  5868 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:12.795595  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:12.796265  6021 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:12.798369  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:12.801172  6773 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:12.807071  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.785132  8516 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:12.812896  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:12.815274  5871 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:12.736217  9246 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:26:12.822093  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:12.823230  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:12.801785  9345 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:12.802152  8965 consensus_queue.cc:1243] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:26:12.676288  9343 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:12.806872  8904 raft_consensus.cc:697] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:12.829293  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 123147us
I20260709 12:26:12.837414  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:12.806681  9092 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:12.806918  8931 raft_consensus.cc:697] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:12.821837  9348 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:12.906040  9348 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:12.906715  8931 consensus_queue.cc:237] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:12.907115  9348 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:12.908541  9348 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:12.821934  9063 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:12.911381  8965 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:12.911711  9362 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:26:12.914664  9331 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 1.212s	user 0.002s	sys 0.004s
I20260709 12:26:12.916682  9362 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:12.900403  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:12.807222  9252 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:26:12.918251  9252 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:12.918663  9252 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:12.919637  9252 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:12.921033  9252 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:12.923255  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:12.924078  9349 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 12:26:12.925760  9092 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:12.926193  9092 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.926541  9092 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:12.928056  5872 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:26:12.929878  9250 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:26:12.930332  9321 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.529s	user 0.002s	sys 0.004s
I20260709 12:26:12.932462  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:12.934015  5873 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:12.933887  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:12.934929  5944 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:26:12.940388  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.935675  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:12.947449  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:12.949685  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 539) took 1695 ms. Trace:
I20260709 12:26:12.950834  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.950119  5867 rpcz_store.cc:276] 0709 12:26:11.254807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.255177 (+   370us) service_pool.cc:224] Handling call
0709 12:26:11.255608 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.267433 (+ 11825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.267480 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.274696 (+  7216us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.402801 (+128105us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:11.402892 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.403034 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.929646 (+1526612us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:12.936730 (+  7084us) raft_consensus.cc:1685] finished
0709 12:26:12.936767 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.949581 (+ 12814us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":62320,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":6891,"spinlock_wait_cycles":70080384,"thread_start_us":127496,"threads_started":1,"wal-append.queue_time_us":126021}
W20260709 12:26:12.952096  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.948542  5948 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:12.822811  9251 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 12:26:12.955780  8539 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:12.826628  8516 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:12.965013  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.966475  9159 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:12.967783  9159 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:12.968197  9159 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:12.971601  9159 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:12.974978  5873 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:12.976207  9159 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:12.982246  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:12.983253  5873 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:12.983696  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:12.985061  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.989689  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:12.998560  9355 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:26:12.946622  9063 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:26:13.002223  9063 consensus_queue.cc:1243] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:26:13.005285  8904 consensus_queue.cc:237] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:13.008030  9363 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:26:12.977469  9357 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:26:12.949005  8957 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:12.992296  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:12.958732  8945 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.563s	user 0.015s	sys 0.001s
I20260709 12:26:13.024003  5867 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 9: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:12.937610  9202 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 12:26:13.026275  6785 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:12.990577  9144 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:13.028383  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:13.028487  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:13.031836  5870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 546) took 1746 ms (client timeout 1990 ms). Trace:
W20260709 12:26:13.032266  5870 rpcz_store.cc:269] 0709 12:26:11.285123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.418342 (+133219us) service_pool.cc:224] Handling call
0709 12:26:13.031635 (+1613293us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b5400026f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:13.031771 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1600883968}
I20260709 12:26:13.032549  5941 pending_rounds.cc:85] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 4
I20260709 12:26:13.033171  5941 pending_rounds.cc:107] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:26:13.033490  5941 raft_consensus.cc:2889] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:12.955773  9322 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:26:13.036986  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:12.957001  9358 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 12:26:13.047371  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:13.050172  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:12.986538  9198 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:26:13.052228  5840 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:13.031385  9340 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.038349  9144 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.045188  9214 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:13.052711  9354 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:13.076921  9276 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:13.056798  9334 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:13.122504  9114 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.141997  9214 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:13.157281  9092 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:13.157281  9205 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:13.109122  9144 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:13.178279  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.111159  9058 raft_consensus.cc:3055] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:13.171303  9114 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:13.167631  9214 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.177057  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:13.108079  9340 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:13.160866  9225 consensus_meta.cc:287] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.000s
I20260709 12:26:13.182093  5870 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:13.135444  9213 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:13.136169  9354 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:13.162528  9334 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.163622  9276 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:13.179354  9092 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:13.204612  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.207497  5870 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:26:13.182616  9205 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:13.183919  9214 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:13.209905  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:13.188970  9354 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:13.183004  9058 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:13.187948  8516 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:13.195694  9334 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:13.195683  9341 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.199234  9276 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:13.221187  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.224409  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:13.227237  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 175091us
I20260709 12:26:13.228332  6017 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:13.230283  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:13.232124  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.233073  6017 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:13.233494  6018 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:13.233480  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.231709  5840 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:13.216859  9376 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:13.243216  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:13.246217  6015 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:13.248867  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.230219  9259 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:13.249576  6022 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
W20260709 12:26:13.255523  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.236946  9254 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:13.260334  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.254123  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:13.261039  6022 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:13.261670  6015 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.245536  9215 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:13.225665  9341 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:13.253095  9376 raft_consensus.cc:697] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:13.288924  6015 raft_consensus.cc:2410] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:26:13.293920  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.294896  6018 raft_consensus.cc:2410] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 14 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:26:13.294688  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.295940  6015 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:13.272832  9114 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:13.231684  9058 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:13.298112  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.311115  6022 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:26:13.286799  9368 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:26:13.314380  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.315821  6015 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 8: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:26:13.295877  9371 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:26:13.274912  9215 raft_consensus.cc:697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:13.310703  9369 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
W20260709 12:26:13.354542  9381 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 12:26:13.361863  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.365376  9314 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 1.683s	user 0.003s	sys 0.009s
W20260709 12:26:13.368569  9176 consensus_meta.cc:287] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.708s	user 0.014s	sys 0.000s
I20260709 12:26:13.373553  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 906) took 2434 ms. Trace:
I20260709 12:26:13.374354  6785 rpcz_store.cc:276] 0709 12:26:10.939518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.988053 (+ 48535us) service_pool.cc:224] Handling call
0709 12:26:10.988485 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.029614 (+2041129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.373072 (+343458us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b5400008054. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:13.373395 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4685152000}
I20260709 12:26:13.378990  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.381122  6023 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 10: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:13.382223  9315 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:13.383090  9315 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:13.384723  9215 consensus_queue.cc:237] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:13.388062  9315 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:13.369274  9176 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:13.408033  9292 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:13.408627  9292 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:13.413255  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 590043us
I20260709 12:26:13.413496  9277 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:13.414454  9277 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:13.415671  9277 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:13.418040  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:13.418870  5872 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:13.421859  9246 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:13.422313  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.422797  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.422521  9246 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:13.423930  9350 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.824s	user 0.000s	sys 0.002s
I20260709 12:26:13.424894  9246 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:13.425990  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.425921  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:13.427479  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:13.428413  9340 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:13.429339  6762 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:13.433161  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.429787  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:13.438611  6023 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:13.429895  6775 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:13.439002  9252 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:13.439850  9196 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.439713  9252 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:13.440501  9196 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:13.441435  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:13.434255  6793 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 12:26:13.434648  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.434317  6018 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:26:13.447068  9202 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.447345  9377 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.447784  9202 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:13.448141  9377 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:13.452623  9176 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:13.454229  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.012s	sys 0.013s
I20260709 12:26:13.430189  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:13.435271  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.454658  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.012s	sys 0.013s
W20260709 12:26:13.455039  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 664996us
I20260709 12:26:13.430136  5866 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:26:13.428987  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:13.430527  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:13.430951  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:13.431424  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:13.429730  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:13.436933  6611 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:13.458735  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.459261  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.459985  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:13.428617  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:13.421263  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:13.434751  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.462652  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.463552  6018 raft_consensus.cc:2410] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:26:13.463887  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:13.460819  6611 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:13.456976  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:13.467757  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:13.468608  6762 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:13.468735  5942 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:13.471165  5871 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:13.474506  5866 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:13.481341  5942 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:26:13.487422  5874 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:13.487739  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 66566.9us
I20260709 12:26:13.489516  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:13.494293  5868 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:13.448542  9070 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:13.528551  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 67839.7us
I20260709 12:26:13.547710  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.549014  6018 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 11: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:13.546710  9246 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:13.551580  9246 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.552055  9246 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:13.554212  8539 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:13.555269  8957 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.556025  8539 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.555920  8957 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:13.556602  8539 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:13.557880  9375 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.558485  9252 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:13.558880  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:13.559592  6762 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:13.409546  9292 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:13.560864  9292 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:13.558490  9375 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:13.488021  9264 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:13.493223  9122 consensus_meta.cc:287] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.797s	user 0.014s	sys 0.002s
W20260709 12:26:13.572070  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 116423us
I20260709 12:26:13.576458  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.581071  9375 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:13.493503  9315 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:13.583796  9315 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:13.584237  9315 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:13.583133  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:13.585832  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 92458.3us
I20260709 12:26:13.586771  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.584118  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.586630  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.587666  6015 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:26:13.587785  6024 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:13.573879  8957 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:13.589967  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.590886  6762 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:13.570188  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:13.589278  6015 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:26:13.608608  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.093070  9348 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
W20260709 12:26:13.969321  5872 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.002s	sys 0.011s
W20260709 12:26:14.095377  5872 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.006s	sys 0.011s
W20260709 12:26:13.979620  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.096567  9300 consensus_queue.cc:1059] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 12:26:14.097338  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.597477  6019 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:26:13.646114  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:14.098794  6019 pending_rounds.cc:85] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 2
I20260709 12:26:14.099414  6019 pending_rounds.cc:107] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:26:14.099565  6020 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:14.099820  6019 raft_consensus.cc:2889] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:13.598094  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:13.985773  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:13.988364  6021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.003s	sys 0.010s
W20260709 12:26:14.103323  6021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.003s	sys 0.011s
I20260709 12:26:14.103478  9094 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.103960  9219 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:13.567451  9203 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.104698  9094 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:13.579422  9251 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:13.572057  9264 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.600510  9340 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.499070  9376 consensus_queue.cc:237] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:14.019598  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:14.020021  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 435991us
I20260709 12:26:14.190174  6023 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:14.022423  8904 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:14.201606  8904 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:14.025882  9367 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.205700  9367 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:14.045450  9092 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:14.210312  9340 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:14.046176  9214 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.825s	user 0.020s	sys 0.042s
I20260709 12:26:14.212208  9214 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:14.050390  9202 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:14.051621  9377 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:14.057245  9159 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.080s	user 0.010s	sys 0.013s
I20260709 12:26:14.063500  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:14.077874  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 464313us
I20260709 12:26:14.218144  9094 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:14.219780  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.219982  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:14.049333  9140 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
W20260709 12:26:14.082821  9361 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.488s	user 0.011s	sys 0.000s
I20260709 12:26:14.225394  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:14.088115  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.226691  6020 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:14.089443  8516 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:14.227841  8516 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:14.087869  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 799) took 3930 ms. Trace:
I20260709 12:26:14.102366  9343 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:14.091220  8957 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.230607  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 781) took 1477 ms. Trace:
I20260709 12:26:14.230281  8957 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:14.105127  9219 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:14.106345  9203 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:13.584895  9246 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:14.235162  9214 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:14.235579  9203 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:14.107844  9114 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:14.236244  9246 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:14.109118  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.108062  9058 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:14.237488  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.103485  5948 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:14.110345  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:14.110369  9247 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.095388  9348 raft_consensus.cc:3055] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:14.108275  6016 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:26:14.111912  9372 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.113189  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.115370  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.237555  9356 raft_consensus.cc:3055] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:14.244666  9356 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:14.110157  5840 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:14.237907  9360 raft_consensus.cc:3055] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:14.246250  9360 raft_consensus.cc:740] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:12.475296  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:14.247131  9300 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:14.247645  9360 consensus_queue.cc:260] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:14.237958  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.116905  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:14.249684  9140 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:26:14.118340  6757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 901) took 3475 ms (client timeout 2037 ms). Trace:
I20260709 12:26:14.250478  9140 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:14.119195  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 548969us
I20260709 12:26:14.229079  6756 rpcz_store.cc:276] 0709 12:26:10.156705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.269504 (+112799us) service_pool.cc:224] Handling call
0709 12:26:10.269902 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.270117 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:10.301348 (+ 31231us) log.cc:844] Serialized 14 byte log entry
0709 12:26:12.231258 (+1929910us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b1800073720. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:26:12.923456 (+692198us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:13.404115 (+480659us) spinlock_profiling.cc:243] Waited 6.32 ms on lock 0x7b1800073720. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d667b3c7 00007f39d2c8d9a7 00007f39d2c9d0bf 00007f39d2c9d038
0709 12:26:13.593881 (+189766us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.086687 (+492806us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:14.086848 (+   161us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:14.087028 (+   180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.087221 (+   193us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.087515 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41052,"prepare.queue_time_us":209074,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":8817,"raft.queue_time_us":495683,"raft.run_cpu_time_us":572,"raft.run_wall_time_us":5102,"spinlock_wait_cycles":71930880,"thread_start_us":389875,"threads_started":2,"wal-append.queue_time_us":2298060}
I20260709 12:26:14.239872  6015 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:26:14.134419  9264 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:14.252063  9140 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:14.253259  9159 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:14.254140  9159 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:14.249198  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 87852.2us
I20260709 12:26:14.251183  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:14.197834  9357 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:14.258683  9357 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:14.258921  9377 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:14.260340  9357 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:14.226874  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.228350  9370 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:14.108498  9375 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.239166  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:14.252321  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:14.134253  9251 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:14.213243  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 966) took 1720 ms. Trace:
I20260709 12:26:14.199345  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:14.241076  6016 pending_rounds.cc:85] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 5
I20260709 12:26:14.263616  9063 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:14.240885  9348 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:14.243853  5946 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:14.295436  5948 raft_consensus.cc:2410] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:26:14.243254  5949 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:14.239969  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.257366  9159 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:14.295413  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:14.250893  6757 rpcz_store.cc:269] 0709 12:26:10.642968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.760523 (+117555us) service_pool.cc:224] Handling call
0709 12:26:14.117536 (+3357013us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b5400003a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:14.117943 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4543131392}
I20260709 12:26:14.300143  6016 pending_rounds.cc:107] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:26:14.300828  6016 raft_consensus.cc:2889] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:14.306878  6015 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:14.306973  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 290620us
I20260709 12:26:14.310359  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.310678  5948 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:14.312299  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.312484  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:14.238168  8957 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:14.313200  5945 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:14.314080  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.313411  5948 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:14.317482  9340 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:14.318042  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 208429us
I20260709 12:26:14.293843  5943 rpcz_store.cc:276] 0709 12:26:12.492762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.492963 (+   201us) service_pool.cc:224] Handling call
0709 12:26:12.520882 (+ 27919us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.667571 (+146689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.211440 (+1543869us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b5400602a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:14.212906 (+  1466us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":335571328}
I20260709 12:26:14.318994  9412 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:14.319151  9348 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:14.319850  9375 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:14.327015  9357 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:14.310791  9219 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:14.314514  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 53301.7us
W20260709 12:26:14.339655  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.292133  9251 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:14.292788  9367 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:14.355161  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.296685  9252 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:26:14.241170  9247 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:14.339354  9063 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:14.368251  9196 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:14.241515  9372 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:14.380417  9372 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:14.381523  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 129440us
I20260709 12:26:14.356523  9412 raft_consensus.cc:697] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:14.381946  9092 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:14.386979  5946 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:14.388841  9384 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:14.389773  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:14.390026  9356 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:14.393018  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:14.405179  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:14.406924  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 232073us
W20260709 12:26:14.409049  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 107544us
W20260709 12:26:14.402590  9315 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.818s	user 0.031s	sys 0.004s
I20260709 12:26:14.419800  9341 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:14.420950  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:14.421656  9386 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.422912  9386 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:14.421952  8904 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:14.430437  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:14.431468  5866 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
W20260709 12:26:14.435356  9070 consensus_meta.cc:287] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.887s	user 0.024s	sys 0.018s
I20260709 12:26:14.436908  9412 consensus_queue.cc:237] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:14.438747  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:14.439278  9391 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.874s	user 0.004s	sys 0.011s
I20260709 12:26:14.439579  9247 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:14.439724  5865 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
W20260709 12:26:14.442953  9276 consensus_meta.cc:287] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.213s	user 0.002s	sys 0.033s
I20260709 12:26:14.443760  9348 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:14.444957  9070 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:14.421685  9219 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:14.447230  9219 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.447795  9219 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:14.448578  9334 consensus_meta.cc:287] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.220s	user 0.016s	sys 0.015s
I20260709 12:26:14.445808  9070 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:14.449999  9261 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.450565  9261 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:14.453197  6611 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:14.456095  9272 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:14.456571  9272 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.456969  9272 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:14.457269  9315 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:14.458678  6757 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:26:14.459285  5838 leader_election.cc:336] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Timed out: RequestConsensusVote RPC to 127.2.4.195:43329 timed out after 1.512s (SENT)
I20260709 12:26:14.458480  9094 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:14.460101  9070 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:14.460387  9315 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:14.411175  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.462443  5946 raft_consensus.cc:2410] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:26:14.462649  6609 leader_election.cc:400] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:26:14.463213  6609 leader_election.cc:403] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:14.463809  9354 consensus_meta.cc:287] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.239s	user 0.025s	sys 0.005s
I20260709 12:26:14.464573  8663 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8.
I20260709 12:26:14.464533  9354 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:14.467388  9354 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:14.443637  9276 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:14.470047  5949 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:26:14.413431  9372 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:14.449158  9334 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:14.489790  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.490320  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:14.490697  5946 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:14.491016  6757 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:26:14.493098  9362 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:14.494251  9386 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:14.494686  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:14.495543  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 111876us
I20260709 12:26:14.496479  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:14.496646  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:14.497480  8539 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:14.498148  9362 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:14.489876  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:14.503876  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:14.505316  9360 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:14.507102  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:14.509871  9356 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:14.507103  9372 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.513913  9384 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:14.514529  9384 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.514360  9372 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:14.509356  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:14.482890  5990 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:14.514945  9144 consensus_meta.cc:287] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.336s	user 0.011s	sys 0.021s
I20260709 12:26:14.514972  9384 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:14.534395  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 69202us
I20260709 12:26:14.544615  5869 raft_consensus.cc:2393] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:14.544797  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.544932  9386 raft_consensus.cc:2764] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:26:14.546101  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:14.547040  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:14.548189  9362 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:14.548514  9276 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:14.552268  5948 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:14.553335  6015 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:14.555377  5867 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:14.562115  9058 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.562989  9058 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:14.555307  9114 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:14.584821  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.555799  9272 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:14.591626  6611 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:14.597566  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:14.585379  9292 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.963s	user 0.017s	sys 0.024s
I20260709 12:26:14.597411  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 986) took 1016 ms. Trace:
I20260709 12:26:14.603313  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 987) took 1020 ms. Trace:
I20260709 12:26:14.556320  9144 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:14.610798  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.613967  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:14.623086  9272 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:14.624152  6756 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 8: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:26:14.625838  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 217327us
I20260709 12:26:14.631553  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:14.632459  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.633090  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.644071  5943 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:14.634505  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:14.645349  5949 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:14.636245  5915 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:14.636525  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:14.637418  9167 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:14.638556  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.634353  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:14.639441  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.639338  9144 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:14.640008  9375 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:14.640236  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:14.642729  9334 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:14.647712  6757 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
W20260709 12:26:14.649662  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 45097.7us
I20260709 12:26:14.605052  5941 rpcz_store.cc:276] 0709 12:26:13.466394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.466544 (+   150us) service_pool.cc:224] Handling call
0709 12:26:13.530344 (+ 63800us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:14.084959 (+554615us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.096323 (+ 11364us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:14.179360 (+ 83037us) log.cc:844] Serialized 78 byte log entry
0709 12:26:14.336142 (+156782us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:14.336184 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.346293 (+ 10109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.471873 (+125580us) raft_consensus.cc:1685] finished
0709 12:26:14.471901 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.481904 (+ 10003us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":457747,"raft.queue_time_us":534868,"raft.run_cpu_time_us":599,"raft.run_wall_time_us":203366,"thread_start_us":205194,"threads_started":2,"wal-append.queue_time_us":238885}
I20260709 12:26:14.634371  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:14.658138  5990 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:14.660822  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:14.661700  6023 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:14.664731  5838 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:14.665985  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 75687.2us
I20260709 12:26:14.605449  5940 rpcz_store.cc:276] 0709 12:26:13.466914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.483489 (+ 16575us) service_pool.cc:224] Handling call
0709 12:26:13.483860 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.023080 (+539220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.023202 (+   122us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.024806 (+  1604us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.341272 (+316466us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:14.341338 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.357676 (+ 16338us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.472295 (+114619us) raft_consensus.cc:1685] finished
0709 12:26:14.472317 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.487358 (+ 15041us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":500215,"raft.queue_time_us":580775,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":381,"thread_start_us":319984,"threads_started":2,"wal-append.queue_time_us":432068}
W20260709 12:26:14.669179  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.604453  8663 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:26:14.679215  6609 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:14.680272  9122 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:14.681706  9252 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:14.682128  9252 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:14.683252  9252 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:14.687240  9251 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:14.677246  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:14.689929  9219 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:14.665560  9070 consensus_queue.cc:1059] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
W20260709 12:26:14.692375  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.679556  9386 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:14.693823  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 989) took 1184 ms. Trace:
I20260709 12:26:14.694526  5947 rpcz_store.cc:276] 0709 12:26:13.482171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.482380 (+   209us) service_pool.cc:224] Handling call
0709 12:26:13.482912 (+   532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.567904 (+ 84992us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.567935 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:13.568593 (+   658us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.067849 (+499256us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:14.067978 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.078988 (+ 11010us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.666471 (+587483us) raft_consensus.cc:1685] finished
0709 12:26:14.666496 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.666640 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":134344,"raft.queue_time_us":90938,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":7113,"thread_start_us":485337,"threads_started":1,"wal-append.queue_time_us":933088}
I20260709 12:26:14.697062  9375 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:14.700271  9404 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:14.704623  9404 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.705580  9404 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:14.706427  9408 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:14.707237  9408 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.707688  9408 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:14.708103  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.708864  9404 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:14.709229  9251 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:14.710183  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 74081.3us
I20260709 12:26:14.710518  9219 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:14.677600  6015 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
W20260709 12:26:14.718590  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.670249  9264 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:14.723656  5874 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:14.725306  9070 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:14.726301  9070 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:14.727454  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:14.729236  9276 raft_consensus.cc:3055] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:14.730055  9276 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:14.730259  9360 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:14.666913  9340 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:26:14.732275  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 67733.8us
I20260709 12:26:14.732950  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:14.673202  9114 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:14.734539  9404 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:14.735260  9404 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.735680  8904 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:14.231103  6768 rpcz_store.cc:276] 0709 12:26:12.752898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.756706 (+  3808us) service_pool.cc:224] Handling call
0709 12:26:12.785699 (+ 28993us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:12.786112 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:12.811040 (+ 24928us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.412136 (+601096us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b18001ab840. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb10c0a 00007f39d9250219 00007f39d6b5758f 00007f39cfb96571 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd
0709 12:26:13.428591 (+ 16455us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.179654 (+751063us) spinlock_profiling.cc:243] Waited 38.6 ms on lock 0x7b2c0001ffa8. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:26:14.180495 (+   841us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.180800 (+   305us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:14.181298 (+   498us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.181548 (+   250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.181712 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.230137 (+ 48425us) raft_consensus.cc:1685] finished
0709 12:26:14.230164 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.230401 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":492134,"prepare.queue_time_us":41787,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":495981,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":6474,"spinlock_wait_cycles":97005824,"thread_start_us":132556,"threads_started":1,"wal-append.queue_time_us":235245}
I20260709 12:26:14.736608  9334 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:26:14.737396  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:14.737771  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.737942  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 917) took 1239 ms. Trace:
I20260709 12:26:14.738119  9340 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:14.761418  9340 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:14.740201  5838 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:14.741139  9070 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:14.731307  9276 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:14.734082  9251 raft_consensus.cc:2764] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:26:14.741562  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:14.743119  9343 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.514s	user 0.006s	sys 0.008s
I20260709 12:26:14.744459  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:14.745132  5944 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:14.745385  9264 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:14.741835  9315 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.731315  9360 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:14.757781  6778 rpcz_store.cc:276] 0709 12:26:13.427907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.429867 (+  1960us) service_pool.cc:224] Handling call
0709 12:26:13.430168 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.430555 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.430585 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.382393 (+951808us) spinlock_profiling.cc:243] Waited 41.6 ms on lock 0x7b2c0001ffa8. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:26:14.384651 (+  2258us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.482473 (+ 97822us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:14.488354 (+  5881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.523654 (+ 35300us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.644267 (+120613us) raft_consensus.cc:1685] finished
0709 12:26:14.644287 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.667350 (+ 23063us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":885,"raft.queue_time_us":448,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"spinlock_wait_cycles":95705856,"thread_start_us":93724,"threads_started":1,"wal-append.queue_time_us":205193}
I20260709 12:26:14.756484  9404 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:14.780947  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.785812  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.790122  9365 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.790915  9365 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:14.773512  8904 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:14.795113  5949 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:14.796483  5871 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
W20260709 12:26:14.811151  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 71043.2us
I20260709 12:26:14.796854  9315 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:14.816367  9360 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:14.826668  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:14.788358  9122 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:14.799088  8904 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:14.840830  8915 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:14.808930  9334 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:14.841027  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 99621.9us
I20260709 12:26:14.787308  9372 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:14.856068  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 123335us
I20260709 12:26:14.847939  8915 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:14.858397  8915 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:14.830448  9365 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:14.841981  8904 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:14.872498  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:14.874092  5949 raft_consensus.cc:2376] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8. Re-sending same reply.
I20260709 12:26:14.878886  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:14.879885  5865 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:14.872382  9315 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:14.886572  9425 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:26:14.892547  6778 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:26:14.902719  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:14.862797  9356 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:14.877171  9376 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.010s	sys 0.021s
I20260709 12:26:14.853263  9334 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:14.868532  9203 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:14.930678  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:14.920038  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:14.917266  9376 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.011s	sys 0.022s
I20260709 12:26:14.931777  5940 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:14.937975  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.939373  5949 raft_consensus.cc:2410] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:26:14.936949  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:14.953011  5865 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:14.935644  9122 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:26:14.961252  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 95142us
I20260709 12:26:14.918255  9356 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:14.970610  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.972095  9415 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:14.974468  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:14.974948  5943 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:26:14.975455  5943 pending_rounds.cc:85] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 6
I20260709 12:26:14.975725  5943 pending_rounds.cc:107] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:26:14.975977  5943 raft_consensus.cc:2889] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:14.976521  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:14.991551  6023 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:26:14.992863  5945 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:14.994472  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:15.003399  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:15.005272  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:14.944180  9203 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:15.016057  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.992871  9144 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:14.985953  9251 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:15.043677  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.043895  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 67523.9us
I20260709 12:26:15.027354  5865 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:15.047358  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:15.048678  6778 raft_consensus.cc:2410] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:15.051764  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:15.044442  6609 connection.cc:256] RPC call timeout handler was delayed by -1.0543074920001345s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
I20260709 12:26:15.070617  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:15.042874  9372 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:15.077911  5945 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:15.087208  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:15.052899  9384 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:15.099776  9340 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.100857  9340 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:15.107283  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:15.097043  5873 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:15.096801  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:15.116592  5868 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:26:15.118783  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:15.103972  9340 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:15.123138  5873 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:15.091413  9264 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:15.101383  9428 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 12:26:15.093111  9334 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:15.102833  9251 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:15.138172  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:15.096719  9408 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:15.143005  5945 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:15.144626  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:15.145550  5945 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:15.146785  5941 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:26:15.150069  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 194771us
I20260709 12:26:15.147413  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:15.123441  9372 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.152637  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:15.156473  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 86011.4us
I20260709 12:26:15.160100  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:15.145881  9408 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:15.148332  9264 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:15.168237  9431 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.172s	user 0.000s	sys 0.002s
I20260709 12:26:15.171190  9384 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:15.165948  9372 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:15.172636  9264 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:15.180012  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.183979  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:15.214577  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.184336  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.186417  9203 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:15.191386  9251 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.192126  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:15.193182  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.218640  6023 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 8: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:26:15.193881  9430 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:26:15.169098  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:15.199405  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:15.204828  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:15.208652  9261 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:15.213024  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.212354  5871 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
W20260709 12:26:15.215409  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.223234  6022 raft_consensus.cc:2393] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:15.184324  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.220247  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.232353  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:15.233435  5945 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:15.240777  5838 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:15.240091  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.244628  6020 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 9: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:15.250211  9384 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:15.250363  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:15.251997  9264 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:15.252504  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:15.253139  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.221112  9356 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:15.257157  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:15.262794  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.263442  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:15.263584  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 128132us
I20260709 12:26:15.264642  6023 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 11: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:15.265988  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:15.269495  5945 raft_consensus.cc:2410] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:15.271085  6019 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 7: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:15.270937  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.228256  9251 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:15.284262  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.261776  9435 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:15.293522  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:15.296139  6023 raft_consensus.cc:2410] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:26:15.250509  8904 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:15.297935  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.298432  5866 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:15.294864  6020 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:26:15.299893  6015 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 8: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:26:15.302064  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 97379.4us
I20260709 12:26:15.302887  6017 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 6: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:26:15.304353  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.306396  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:15.294194  9435 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.310591  5940 raft_consensus.cc:3055] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:15.310999  5940 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:15.309283  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:15.314292  9372 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:15.314910  9432 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 12:26:15.317914  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:15.320202  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:15.327251  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:15.327459  6019 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 10: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:15.330047  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.315932  9435 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:15.333129  6017 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:15.335345  5940 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:15.336488  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:15.342713  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.333181  9384 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:15.333592  9372 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:15.351410  5940 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:15.352622  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 43913us
I20260709 12:26:15.354794  9251 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:15.358127  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1288) took 1051 ms. Trace:
I20260709 12:26:15.358731  5944 rpcz_store.cc:276] 0709 12:26:14.306957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.313673 (+  6716us) service_pool.cc:224] Handling call
0709 12:26:15.357816 (+1044143us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b5400673ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:15.358001 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":944708480}
W20260709 12:26:15.362035  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.364339  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.365320  6023 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 10: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:15.366050  8904 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:15.369328  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.369767  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.370407  6018 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 10: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:15.370599  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:15.371294  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.371642  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.372323  6024 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:15.374476  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.376389  6024 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:26:15.378645  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:15.378516  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.369362  9252 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:15.381318  9058 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:15.376448  6020 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:26:15.368137  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.371435  6019 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:15.384696  5873 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
W20260709 12:26:15.387497  5869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 573) took 1960 ms (client timeout 1676 ms). Trace:
W20260709 12:26:15.387910  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 72107.8us
W20260709 12:26:15.388052  5869 rpcz_store.cc:269] 0709 12:26:13.426654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.428760 (+  2106us) service_pool.cc:224] Handling call
0709 12:26:15.387182 (+1958422us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400144bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:15.387396 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2437336320}
I20260709 12:26:15.389452  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:15.390341  5872 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:15.387501  6018 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:15.393905  6023 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:15.395141  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:26:15.396023  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 71431.5us
I20260709 12:26:15.401286  5941 raft_consensus.cc:2368] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11.
I20260709 12:26:15.401031  9372 raft_consensus.cc:2764] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:26:15.402200  5946 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:15.403894  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:15.406121  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.407500  9354 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:15.410970  8516 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:15.413308  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.413676  9404 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.616s	user 0.015s	sys 0.001s
I20260709 12:26:15.414098  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:15.414389  5913 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:15.418808  9408 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:15.419353  9408 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.405889  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:15.405751  9251 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:15.425297  5838 leader_election.cc:400] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11.
I20260709 12:26:15.426523  5838 leader_election.cc:403] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:26:15.401882  6020 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:15.428651  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:15.432451  5873 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:15.423377  9408 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:15.443147  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:15.447839  9215 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:26:15.450525  9122 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:15.451259  9122 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:15.454139  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:15.455049  6020 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:15.444991  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:15.445541  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:15.462968  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:15.414100  9404 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:15.482816  6019 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:15.435401  9140 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:15.484203  6611 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:15.443095  9214 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:15.443553  9412 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:15.499217  5940 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:15.499660  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:15.502602  6024 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:15.502902  9360 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:15.495532  8516 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.506670  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:15.507380  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 62541.1us
I20260709 12:26:15.511475  6023 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:15.511330  6611 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:15.516575  5942 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 10: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:15.513898  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.517854  6019 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
I20260709 12:26:15.510486  9272 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:15.514726  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 73529.2us
I20260709 12:26:15.499639  9144 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:15.525033  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:15.522533  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 116739us
I20260709 12:26:15.510432  9360 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.513378  9058 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:15.554728  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:15.530764  9372 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:26:15.562194  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 132186us
I20260709 12:26:15.516096  8516 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:15.542114  9360 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:15.567374  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:15.504381  9140 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:15.571700  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.555050  9058 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:15.570670  9439 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.223s	user 0.001s	sys 0.001s
I20260709 12:26:15.551857  9334 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:26:15.583828  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:15.577420  9372 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:15.588838  8516 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:15.587687  6768 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:26:15.589859  9372 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:15.593382  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.594223  6023 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
I20260709 12:26:15.595131  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:15.596534  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:15.597229  6757 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:15.597100  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:15.596028  6768 raft_consensus.cc:2368] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11.
I20260709 12:26:15.565898  9384 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:26:15.600896  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:15.601722  6756 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:15.602447  8904 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:15.606316  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:15.606711  9276 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:15.611100  9140 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:15.600232  9334 consensus_queue.cc:1243] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:15.575980  9412 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:15.622489  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:15.616446  6757 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:15.582600  9058 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:15.638165  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:15.631722  6017 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:15.643584  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 942) took 2207 ms. Trace:
I20260709 12:26:15.642381  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:15.647239  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.638636  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:15.645866  5867 rpcz_store.cc:276] 0709 12:26:13.420657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.428894 (+  8237us) service_pool.cc:224] Handling call
0709 12:26:13.429224 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:14.577455 (+1148231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.628136 (+1050681us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540043f654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:15.628366 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2543591296}
I20260709 12:26:15.644873  6762 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:15.649276  6023 raft_consensus.cc:2393] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 10: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:15.652925  6757 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:15.621083  9412 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:15.674705  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.676517  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 94761us
W20260709 12:26:15.680979  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 130292us
W20260709 12:26:15.683827  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 138642us
I20260709 12:26:15.692551  6756 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:15.648471  9144 raft_consensus.cc:697] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:15.698261  9445 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:15.679272  9412 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:15.706568  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 63214.9us
I20260709 12:26:15.707207  6020 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:15.677762  9264 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:15.707862  6024 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:15.663550  9334 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 12:26:15.719206  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 139231us
I20260709 12:26:15.722057  6024 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:26:15.735292  9417 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 1.079s	user 0.005s	sys 0.003s
I20260709 12:26:15.728193  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:15.719159  9404 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:15.751205  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:15.749402  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.755035  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.761746  5947 raft_consensus.cc:2410] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:15.744665  9203 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:15.742995  9450 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:26:15.772334  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 77899.2us
I20260709 12:26:15.725133  9445 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:15.790359  9440 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:26:15.790824  9453 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.791440  9453 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:15.801177  6768 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:15.809497  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:15.809804  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.810633  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1220) took 1289 ms. Trace:
I20260709 12:26:15.811022  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:15.811159  5872 rpcz_store.cc:276] 0709 12:26:14.523532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.526475 (+  2943us) service_pool.cc:224] Handling call
0709 12:26:15.390040 (+863565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:15.392616 (+  2576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.810360 (+417744us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b5400144bd0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:15.810526 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1930851584}
I20260709 12:26:15.815244  9452 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:15.816401  8904 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:15.816918  5990 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:15.818341  9453 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:15.818907  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 537) took 1052 ms. Trace:
I20260709 12:26:15.819149  9435 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:15.819308  5942 rpcz_store.cc:276] 0709 12:26:14.766595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.770314 (+  3719us) service_pool.cc:224] Handling call
0709 12:26:15.818641 (+1048327us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b5400602f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:15.818809 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1678335360}
I20260709 12:26:15.822027  9144 consensus_queue.cc:237] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:15.822625  6015 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:26:15.824297  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.824826  9452 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:15.829598  6778 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
W20260709 12:26:15.836611  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 77543.4us
I20260709 12:26:15.802680  5693 catalog_manager.cc:5697] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 5 to 6, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 6 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:15.838018  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.774780  9354 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:15.863399  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:15.882228  5915 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:15.863277  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:15.895761  5874 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:15.900622  5868 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:15.832597  9435 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:15.836131  9445 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:15.877784  9354 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.906076  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:15.910073  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:15.913836  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:15.907362  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 96401.3us
I20260709 12:26:15.857115  9442 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:15.933045  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 238183us
I20260709 12:26:15.914824  6021 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:15.929663  6757 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:15.948731  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:15.858762  9450 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:15.964576  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:15.971951  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:15.972944  9334 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:15.956434  9453 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:26:15.975198  5868 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:26:15.978988  5865 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:15.939258  9445 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:15.939352  9458 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:15.987192  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.989199  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 107079us
W20260709 12:26:15.989042  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 85012.6us
I20260709 12:26:15.997535  6611 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:15.999805  6611 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:15.994138  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:15.991689  5870 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:16.003037  5945 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
W20260709 12:26:16.007323  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 68562.5us
I20260709 12:26:16.010461  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:15.940469  9354 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:15.958564  9404 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
W20260709 12:26:16.017939  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 79298.5us
I20260709 12:26:16.020387  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:15.990046  9442 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:16.022037  9122 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.004s
I20260709 12:26:16.079927  9122 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:16.080451  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.083096  9122 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:15.959780  9412 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:16.087190  5949 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:15.960620  9360 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:16.091778  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:16.097200  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 86929.2us
W20260709 12:26:16.086251  9462 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:26:16.116290  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:16.120054  5874 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:16.097030  9122 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:16.102602  9251 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:16.146692  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 133815us
I20260709 12:26:16.081058  9442 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:16.154475  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:16.160238  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:16.164438  9445 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:16.164243  9376 consensus_queue.cc:1059] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:16.164641  8915 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:16.165199  8915 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.165091  9445 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:16.165737  9376 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:16.165828  8915 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:16.166633  9376 raft_consensus.cc:3055] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:16.168442  5915 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:16.167073  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:16.004634  9458 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:16.176147  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.146402  9218 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:16.142647  9435 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:16.180963  6785 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:16.167769  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:16.199754  6793 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.178243  5915 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:16.201714  6773 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:16.203210  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:16.203966  6773 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:16.185230  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:16.188685  9445 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:16.207386  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:16.207679  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:16.208225  6778 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:16.208536  5945 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:26:16.167032  9376 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:16.187920  9457 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
I20260709 12:26:16.151243  9470 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:16.122890  9356 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:26:16.219117  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:16.184005  9461 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:26:16.181933  9218 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.180191  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:16.189500  9348 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:16.181252  9360 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:16.220850  5865 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:16.222896  6768 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:16.183073  9447 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:16.234474  9315 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:16.260437  9447 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:16.244184  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:16.260859  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:16.261615  6773 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 11: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:16.261889  9447 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:16.262791  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:16.234951  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:16.247475  6766 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
W20260709 12:26:16.247579  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.252210  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:16.260569  9315 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:16.176885  9466 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:16.261922  6768 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:16.264564  9315 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.265059  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:16.263557  6773 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:16.264456  9466 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:16.265791  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:16.265782  6766 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.266870  6785 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:16.267472  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:16.268376  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:16.267082  9466 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:16.270536  9466 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.263633  9447 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:16.278890  6785 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:26:16.281412  6785 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:16.169644  9251 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:16.282889  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:16.186408  9455 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:26:16.229713  9445 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:16.225585  9470 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.306735  5865 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:16.199283  9122 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:16.253180  9218 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:16.299149  9478 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 12:26:16.290097  5990 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:16.321270  6778 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:16.323450  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:16.304854  8904 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:16.275995  9315 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:16.308048  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 308304us
I20260709 12:26:16.308390  6756 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: [8.5-10.6]
W20260709 12:26:16.294955  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 140971us
I20260709 12:26:16.185925  9459 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 12:26:16.314780  5941 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:16.252736  9348 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:16.319412  6768 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:16.305203  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:16.338161  6766 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:16.362567  6609 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:16.340402  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:16.284152  9480 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:16.342077  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 175078us
W20260709 12:26:16.348431  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.365566  5865 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:16.295749  9354 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:16.306910  9445 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.381270  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:16.366639  6778 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:16.368834  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:16.386094  6766 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:16.342379  9473 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:16.386607  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:16.396585  6778 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:16.321182  9470 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:16.363535  9428 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:26:16.307541  9144 consensus_queue.cc:1059] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:26:16.368841  9348 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:16.427598  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:16.381745  9450 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:16.395241  9473 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.449287  5865 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:16.387681  9445 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.453187  6778 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
W20260709 12:26:16.457141  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 94622.9us
I20260709 12:26:16.414613  9412 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:16.426074  9276 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:16.478592  6611 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:16.334496  9376 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:16.498512  6756 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 13. Ops: [11.6-11.6]
I20260709 12:26:16.503546  6793 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
W20260709 12:26:16.499464  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 194415us
I20260709 12:26:16.446805  9482 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:16.518339  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 46313us
I20260709 12:26:16.424925  9459 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
W20260709 12:26:16.527771  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 202959us
I20260709 12:26:16.460875  9473 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:16.478204  9450 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:16.534102  6766 raft_consensus.cc:1217] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Deduplicated request from leader. Original: 7.4->[10.5-10.5]   Dedup: 10.5->[]
I20260709 12:26:16.534705  6766 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:26:16.531987  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:16.529805  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.496397  9360 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:16.380846  9480 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:16.538314  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:16.538193  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 157007us
I20260709 12:26:16.496909  9483 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:16.514849  9482 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:16.561043  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 218898us
I20260709 12:26:16.562623  6609 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:16.478725  9455 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:26:16.550030  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:16.568929  9276 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.551067  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:16.539314  9360 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:16.602077  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 52105.5us
W20260709 12:26:16.562084  9408 consensus_meta.cc:287] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.057s	user 0.014s	sys 0.002s
I20260709 12:26:16.556301  9372 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:26:16.613667  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.547466  9442 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:16.621470  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:16.635798  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 73253.9us
I20260709 12:26:16.638002  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:16.565315  9482 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:16.583846  9483 raft_consensus.cc:697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:16.679716  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.615028  9492 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:16.647912  9251 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:16.582501  9499 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 12:26:16.652865  9500 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.630919  9408 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:16.663422  9122 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:16.659957  9458 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:16.703074  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 171151us
I20260709 12:26:16.714660  5915 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:16.717255  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 109806us
W20260709 12:26:16.696080  9214 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.187s	user 0.016s	sys 0.001s
I20260709 12:26:16.685205  9315 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:16.727941  5840 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:16.733801  5942 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 10: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:26:16.774953  9485 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.240s	user 0.001s	sys 0.001s
I20260709 12:26:16.716920  9466 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:16.743995  9450 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:16.761667  9503 raft_consensus.cc:2764] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:26:16.776222  9491 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.775607  6775 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:16.742301  9442 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.789346  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:16.791791  6768 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 6.
W20260709 12:26:16.777617  9489 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
W20260709 12:26:16.809947  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.741689  9480 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:16.734508  9214 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11.
W20260709 12:26:16.745148  9487 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.267s	user 0.002s	sys 0.000s
I20260709 12:26:16.807124  9315 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:16.737560  9458 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.744664  9479 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.198s
W20260709 12:26:16.805498  9469 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
W20260709 12:26:16.840138  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 50973us
I20260709 12:26:16.700861  9497 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.119s
W20260709 12:26:16.857246  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 219310us
I20260709 12:26:16.858685  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:16.698060  9492 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:16.867437  5990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.021s	sys 0.046s
I20260709 12:26:16.864266  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:16.869884  5990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.021s	sys 0.047s
W20260709 12:26:16.870296  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 526753us
I20260709 12:26:16.716771  9360 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:16.841019  8915 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.674s	user 0.015s	sys 0.000s
I20260709 12:26:16.874245  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:16.721400  9500 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:16.882890  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 164760us
I20260709 12:26:16.883358  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:16.884141  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:16.854152  9272 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:16.854945  9315 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:16.897133  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:16.898027  5990 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:16.891562  9480 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:26:16.891847  9251 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:16.904038  9251 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.904441  9251 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:16.904511  9218 consensus_meta.cc:287] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.000s
I20260709 12:26:16.907707  9479 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.859524  9458 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:16.908571  9479 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:16.879942  9495 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:16.879480  9058 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.233s	user 0.011s	sys 0.006s
I20260709 12:26:16.921298  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:16.849579  9442 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:16.905186  9218 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:16.928898  5943 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.021s	sys 0.046s
I20260709 12:26:16.880788  9506 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:16.915452  9504 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:16.951738  5943 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.021s	sys 0.046s
I20260709 12:26:16.909335  9214 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:16.901065  9315 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:16.961108  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 86934.4us
I20260709 12:26:16.879679  8915 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:16.830667  9483 consensus_queue.cc:237] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:16.960546  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 929) took 1062 ms. Trace:
I20260709 12:26:16.966841  6609 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:16.910006  9144 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:26:16.934464  9272 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:16.948325  9480 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:16.960453  9515 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:16.969887  6762 rpcz_store.cc:276] 0709 12:26:15.898686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.898876 (+   190us) service_pool.cc:224] Handling call
0709 12:26:15.940469 (+ 41593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:15.976237 (+ 35768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:15.976289 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:16.006205 (+ 29916us) log.cc:844] Serialized 26 byte log entry
0709 12:26:16.225077 (+218872us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:16.225144 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.225265 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:16.960230 (+734965us) raft_consensus.cc:1685] finished
0709 12:26:16.960260 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:16.960474 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37225,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":5761,"thread_start_us":112267,"threads_started":1,"wal-append.queue_time_us":358065}
I20260709 12:26:16.845335  9491 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:16.984503  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 89061us
W20260709 12:26:16.985972  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 64779.1us
I20260709 12:26:16.989255  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:16.977537  9272 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:16.982396  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:16.929075  9058 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:16.934701  9511 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.202s
I20260709 12:26:16.972522  9408 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:16.976302  9372 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:17.007261  9218 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:17.007524  9058 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:16.981033  9506 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:16.987370  9515 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:16.999233  9272 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:16.964246  9450 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:17.022660  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:17.032598  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:17.039848  5946 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:17.033069  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:17.030861  8915 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:17.042721  9492 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:17.037422  9372 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:17.013880  9480 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:17.044749  9480 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:17.005309  9511 consensus_queue.cc:1243] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:26:17.013468  9214 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:17.046844  5944 raft_consensus.cc:2410] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 12:26:17.027731  9334 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.041s	user 0.015s	sys 0.000s
I20260709 12:26:17.008090  9458 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:17.055327  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 66205.4us
I20260709 12:26:17.057777  5915 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:17.014488  9497 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:17.016441  9515 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:17.035203  9442 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:17.027619  9500 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:17.036859  9276 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:17.062600  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:17.058640  9144 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:17.056767  9122 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:16.963320  9504 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:17.060151  9334 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:17.068681  9497 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.061962  9492 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:26:17.075184  9517 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:17.075042  9458 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:17.106877  5949 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:17.084529  9508 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
W20260709 12:26:17.083496  9513 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 12:26:17.083722  9486 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.286s	user 0.000s	sys 0.001s
W20260709 12:26:17.128159  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.130256  9473 consensus_meta.cc:287] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.587s	user 0.009s	sys 0.004s
I20260709 12:26:17.130704  9479 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:17.130895  9473 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:17.133217  9473 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:17.113756  9466 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:17.101862  9497 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:17.114171  9504 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:17.114519  5942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1322) took 1285 ms (client timeout 1621 ms). Trace:
I20260709 12:26:17.117872  9276 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:26:17.119002  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56555.5us
I20260709 12:26:17.084298  9527 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:17.107060  9517 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:17.121110  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 739) took 1418 ms. Trace:
W20260709 12:26:17.121259  9445 consensus_meta.cc:287] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.622s	user 0.009s	sys 0.005s
I20260709 12:26:17.117399  5946 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
W20260709 12:26:17.084754  9481 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.308s	user 0.000s	sys 0.002s
I20260709 12:26:17.081221  9498 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:17.084998  9512 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:17.138769  5942 rpcz_store.cc:269] 0709 12:26:15.829122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.837538 (+  8416us) service_pool.cc:224] Handling call
0709 12:26:17.114303 (+1276765us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b5400632254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:17.114458 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1996162816}
I20260709 12:26:17.143877  6023 rpcz_store.cc:276] 0709 12:26:15.702692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.702872 (+   180us) service_pool.cc:224] Handling call
0709 12:26:15.714607 (+ 11735us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:15.714968 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:15.959124 (+244156us) log.cc:844] Serialized 14 byte log entry
0709 12:26:16.077069 (+117945us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.120581 (+1043512us) spinlock_profiling.cc:243] Waited 5.89 ms on lock 0x7b18000ba880. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39d282b4bc 00007f39cfb7948b 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287
0709 12:26:17.120699 (+   118us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:17.120755 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:17.120808 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.120878 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.121012 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51621,"prepare.queue_time_us":140117,"prepare.run_cpu_time_us":146,"prepare.run_wall_time_us":8663,"raft.queue_time_us":66570,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":25235,"spinlock_wait_cycles":13547904,"thread_start_us":307385,"threads_started":3}
I20260709 12:26:17.139624  5949 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:17.078887  9144 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:17.119755  5870 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:17.084055  9519 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:17.125341  9503 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:17.144809  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 58486us
I20260709 12:26:17.130278  9442 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:17.085217  9122 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:17.171060  9503 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:17.171341  9442 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:17.148558  9498 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.173280  9122 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:17.173882  9498 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:17.153473  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 931) took 1201 ms. Trace:
I20260709 12:26:17.174543  6757 rpcz_store.cc:276] 0709 12:26:15.952544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.987070 (+ 34526us) service_pool.cc:224] Handling call
0709 12:26:16.019369 (+ 32299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:16.096657 (+ 77288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:16.096705 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:16.102385 (+  5680us) log.cc:844] Serialized 26 byte log entry
0709 12:26:16.206367 (+103982us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:16.206434 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.206536 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:17.153174 (+946638us) raft_consensus.cc:1685] finished
0709 12:26:17.153202 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.153398 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":196315,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":26785,"thread_start_us":90847,"threads_started":1,"wal-append.queue_time_us":488194}
I20260709 12:26:17.155218  9491 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:17.156108  9517 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:17.178478  6785 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:26:17.178834  9517 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:17.179824  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.180712  5873 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 9: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:26:17.157434  9528 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:26:17.158371  5867 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:17.142822  9488 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.159039  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.182787  5865 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:17.182662  9488 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:17.185523  9442 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:17.186251  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:17.186453  5838 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:17.187150  5867 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:26:17.187470  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 976815us
I20260709 12:26:17.188397  9070 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:17.188959  9070 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.189342  9070 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:17.192130  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.192902  5865 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:17.149423  9445 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:17.197530  9488 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:17.198563  9415 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:17.198896  9445 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:17.138757  9458 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:17.199283  9415 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:17.200488  9348 consensus_meta.cc:287] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.751s	user 0.012s	sys 0.003s
I20260709 12:26:17.200587  9458 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:17.201073  9348 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:17.203460  9348 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:17.203697  9415 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:17.206167  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:17.164920  9144 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:17.167639  9470 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.741s	user 0.009s	sys 0.006s
I20260709 12:26:17.169725  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:17.169397  9251 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:17.206869  5871 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:17.207218  6021 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:17.207032  9470 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:17.208839  9251 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:17.200537  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:17.211509  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 813) took 1028 ms. Trace:
I20260709 12:26:17.211898  6773 rpcz_store.cc:276] 0709 12:26:16.183171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.266466 (+ 83295us) service_pool.cc:224] Handling call
0709 12:26:17.200365 (+933899us) spinlock_profiling.cc:243] Waited 7.87 ms on lock 0x7b580001f898. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:26:17.211419 (+ 11054us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18099072}
I20260709 12:26:17.150664  9276 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 9 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:17.213968  9497 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:17.214663  9276 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:17.216419  9496 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.103s	user 0.001s	sys 0.001s
I20260709 12:26:17.217057  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.218032  5867 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:17.218744  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 998) took 1008 ms. Trace:
I20260709 12:26:17.219491  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:17.220306  5865 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:17.219324  5945 rpcz_store.cc:276] 0709 12:26:16.219899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.226067 (+  6168us) service_pool.cc:224] Handling call
0709 12:26:16.226592 (+   525us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:16.257600 (+ 31008us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:16.257663 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:16.295287 (+ 37624us) log.cc:844] Serialized 26 byte log entry
0709 12:26:16.467280 (+171993us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:16.515232 (+ 47952us) log.cc:844] Serialized 14 byte log entry
0709 12:26:16.515435 (+   203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.515516 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:17.112736 (+597220us) spinlock_profiling.cc:243] Waited 6.86 ms on lock 0x7b18000cb680. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:26:17.215562 (+102826us) spinlock_profiling.cc:243] Waited 45.5 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:17.218490 (+  2928us) raft_consensus.cc:1685] finished
0709 12:26:17.218523 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.218672 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18149,"prepare.queue_time_us":71680,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":183216,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":16499,"spinlock_wait_cycles":120341248,"thread_start_us":86548,"threads_started":1,"wal-append.queue_time_us":201978}
I20260709 12:26:17.216395  9251 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:17.223894  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.224624  5869 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:26:17.224514  9251 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:17.225696  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:17.224885  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.226651  9497 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:17.228178  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:17.228428  9505 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:26:17.228757  9251 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:17.228899  5870 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
W20260709 12:26:17.231582  9516 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.068s	user 0.002s	sys 0.000s
I20260709 12:26:17.229717  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:17.246654  5873 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:17.249126  9519 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.250885  9519 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:17.252830  9471 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.954s	user 0.003s	sys 0.000s
I20260709 12:26:17.253854  9470 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:17.257094  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:17.258095  9122 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.258733  9122 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:17.259672  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:17.260077  6024 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:17.260599  6793 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 10: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:17.261180  9122 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:17.261777  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:17.262187  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:17.262707  6773 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 10: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:17.262185  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1239) took 1291 ms. Trace:
W20260709 12:26:17.264439  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.265419  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.265982  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.243029  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:17.243570  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:17.254616  9519 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:17.264067  5874 rpcz_store.cc:276] 0709 12:26:15.970537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.970775 (+   238us) service_pool.cc:224] Handling call
0709 12:26:15.971185 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:16.141429 (+170244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.261919 (+1120490us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54007f30d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:17.262110 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334061952}
I20260709 12:26:17.268131  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:17.264148  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 817) took 1079 ms. Trace:
I20260709 12:26:17.267192  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:17.269322  5948 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:17.267155  9252 raft_consensus.cc:3055] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:17.270321  9252 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:17.271291  9252 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:17.271579  5693 catalog_manager.cc:5697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 1 to 3, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 3 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.273111  9360 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:17.274202  6609 connection.cc:256] RPC call timeout handler was delayed by -2.0915441749998536s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
W20260709 12:26:17.274686  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.805380  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:17.275894  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:17.276355  9376 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:17.241869  5944 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:17.276839  8516 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.277549  8516 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:17.278234  9451 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:17.279145  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 810) took 1097 ms. Trace:
I20260709 12:26:17.278878  9451 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:17.279561  6785 rpcz_store.cc:276] 0709 12:26:16.182138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.282847 (+100709us) service_pool.cc:224] Handling call
0709 12:26:17.159899 (+877052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.179014 (+ 19115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.278353 (+ 99339us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b54002a17d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:17.278924 (+   571us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b54002a17d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:17.279080 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2026821376}
I20260709 12:26:17.280211  8516 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:17.281898  9511 raft_consensus.cc:2764] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:26:17.282109  9442 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:17.282688  9442 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:17.284191  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:17.285043  9442 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:17.269038  6756 rpcz_store.cc:276] 0709 12:26:16.184511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.378111 (+193600us) service_pool.cc:224] Handling call
0709 12:26:16.482721 (+104610us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:16.506669 (+ 23948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.263931 (+757262us) spinlock_profiling.cc:243] Waited 81.8 ms on lock 0x7b5400004450. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:17.264056 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188155264}
I20260709 12:26:17.286660  9252 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:17.287475  9499 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:17.288096  9499 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:17.290160  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:17.290838  6022 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:17.291800  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:17.292447  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:17.294116  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:17.297150  6766 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:17.292595  6768 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:17.298703  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:17.299260  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:17.301117  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:17.301769  6762 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:26:17.304200  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:17.304955  6017 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:17.281347  9451 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:17.306587  9499 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:17.306979  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:17.308054  6768 raft_consensus.cc:2368] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12.
I20260709 12:26:17.309665  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:17.312594  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:17.313364  5945 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:17.315605  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:17.318225  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:17.319626  6768 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 10: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:17.235131  5871 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:17.326618  5865 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:17.330019  5990 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:17.332981  5948 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:17.371317  5838 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:17.336535  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:17.325218  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.342105  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:17.342211  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:17.347990  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:17.359288  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:17.339850  9512 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:26:17.354624  9509 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:26:17.355609  9497 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:17.358357  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:17.359825  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:17.344821  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:17.360577  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 996) took 1201 ms. Trace:
I20260709 12:26:17.365942  5873 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:17.397816  5940 rpcz_store.cc:276] 0709 12:26:16.158763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.158942 (+   179us) service_pool.cc:224] Handling call
0709 12:26:16.209435 (+ 50493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:16.210360 (+   925us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:16.243284 (+ 32924us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.202836 (+959552us) spinlock_profiling.cc:243] Waited 7.73 ms on lock 0x7b18003545e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39d282b4bc 00007f39cfb7948b 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287
0709 12:26:17.294180 (+ 91344us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.360192 (+ 66012us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:17.360250 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:17.360313 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.360380 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.360492 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":248898,"prepare.queue_time_us":187548,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":95,"raft.queue_time_us":230135,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":34264,"spinlock_wait_cycles":17771776,"thread_start_us":193183,"threads_started":2,"wal-append.queue_time_us":459026}
I20260709 12:26:17.403627  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1002) took 1103 ms. Trace:
I20260709 12:26:17.403872  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.395471  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:17.405777  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:17.377067  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:17.407930  5944 raft_consensus.cc:2368] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12.
I20260709 12:26:17.345893  9511 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:17.425570  5987 leader_election.cc:336] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.516s (SENT)
I20260709 12:26:17.380292  6022 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:17.380743  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:17.439621  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:17.446904  6024 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:26:17.382308  6757 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:26:17.204701  9503 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:17.384466  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 54550.7us
I20260709 12:26:17.385840  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:17.381095  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:17.450006  6775 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:17.388810  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:17.450814  6768 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 10: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:26:17.391866  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 46164.2us
I20260709 12:26:17.450866  5990 leader_election.cc:400] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12.
I20260709 12:26:17.451619  5990 leader_election.cc:403] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:17.458536  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:17.459535  6016 raft_consensus.cc:2410] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:26:17.460243  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:17.461319  5949 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
W20260709 12:26:17.430281  5915 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 2.181s (SENT) [suppressed 27 similar messages]
I20260709 12:26:17.470460  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:17.403334  9447 consensus_meta.cc:287] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.124s	user 0.017s	sys 0.000s
I20260709 12:26:17.383821  9334 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:26:17.474148  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 84235.9us
I20260709 12:26:17.479570  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:17.480031  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:17.483374  6023 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: [6.4-10.6]
I20260709 12:26:17.387917  9519 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:17.485765  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:17.404345  5943 rpcz_store.cc:276] 0709 12:26:16.299773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.301749 (+  1976us) service_pool.cc:224] Handling call
0709 12:26:16.302276 (+   527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:16.302562 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:16.956806 (+654244us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.016109 (+ 59303us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.222944 (+206835us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b18003fd360. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:26:17.291653 (+ 68709us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b18003fd360. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb10c0a 00007f39d9250219 00007f39cfb98a40 00007f39cfb934e4 00007f39cfb93455 00007f39d667b3c7 00007f39d2c8d9a7 00007f39d2c9d0bf 00007f39d2c9d038
0709 12:26:17.300843 (+  9190us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.403060 (+102217us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:17.403144 (+    84us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:17.403223 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.403347 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.403538 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":252122,"prepare.queue_time_us":435448,"prepare.run_cpu_time_us":176,"prepare.run_wall_time_us":5213,"raft.queue_time_us":22296,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":190,"spinlock_wait_cycles":106820224,"thread_start_us":606784,"threads_started":2,"wal-append.queue_time_us":117401}
I20260709 12:26:17.390106  9122 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:17.391487  9453 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.459964  5913 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:17.490294  5870 raft_consensus.cc:2410] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:26:17.345865  9144 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 12:26:17.493418  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.483338  6018 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:17.495553  6611 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:17.425467  9527 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:17.515143  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:17.503891  5868 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:26:17.516623  5873 raft_consensus.cc:2410] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:26:17.519577  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:17.505317  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:17.524573  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:17.463155  9509 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.535096  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:17.426141  9521 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:17.542007  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:17.480706  9264 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.394313  9498 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:17.557281  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:17.547672  5870 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:17.403479  9470 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 12:26:17.497991  9122 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.562275  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:17.529448  9511 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:17.528684  9447 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:17.571425  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:17.573225  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 83218.3us
I20260709 12:26:17.636123  6016 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:17.514925  9334 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 9 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:17.574689  6024 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:17.582990  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.545287  9348 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:17.556236  9140 raft_consensus.cc:2764] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:26:17.592793  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.566570  9264 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:17.538046  9450 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 12:26:17.607699  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 209098us
W20260709 12:26:17.657783  5987 leader_election.cc:336] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.527s (SENT)
I20260709 12:26:17.658277  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:17.579778  9459 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:17.659735  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.596465  9272 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.004s
I20260709 12:26:17.572512  9122 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:17.615756  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.616710  6022 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:17.598098  9356 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.300s	user 0.040s	sys 0.045s
W20260709 12:26:17.701911  9356 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.404s	user 0.041s	sys 0.045s
I20260709 12:26:17.527011  9453 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:17.625181  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.637781  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:17.639087  6611 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:17.704133  9144 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:17.704347  9453 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:17.704699  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 65695.1us
I20260709 12:26:17.559415  9509 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:17.705611  9502 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:17.706116  9502 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.706476  9502 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:17.707485  9509 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:17.565886  9058 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:17.573691  9445 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 12:26:17.638988  9483 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.005s	sys 0.020s
I20260709 12:26:17.709496  9058 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:17.709910  9445 raft_consensus.cc:3055] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:17.710121  9494 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:17.710680  9445 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:17.711282  5871 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:17.711542  5943 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:17.712512  5868 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:17.711997  9445 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:17.704350  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 66712.1us
I20260709 12:26:17.715322  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:17.663170  9459 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:17.718647  9070 consensus_meta.cc:287] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.001s
I20260709 12:26:17.719345  9070 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:17.665307  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:17.720345  9459 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:17.710121  9058 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:17.721122  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 55912.1us
I20260709 12:26:17.721877  9058 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:17.722592  9473 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:17.727134  9494 consensus_queue.cc:1243] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:17.635898  9447 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.735575  9447 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:17.737898  5942 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:26:17.665773  9464 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:17.678496  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:17.738765  6024 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:17.739914  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:17.740041  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.740706  5871 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:17.741220  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:17.747072  6024 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:26:17.678520  9526 raft_consensus.cc:2764] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:26:17.685808  9272 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:17.687485  9446 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:17.747653  5867 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:26:17.748056  9446 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:17.748433  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:17.749137  9446 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:17.750274  5948 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:17.750550  9446 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:17.751823  5948 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:26:17.752154  5948 pending_rounds.cc:85] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 7
I20260709 12:26:17.752358  5948 pending_rounds.cc:107] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:26:17.752579  5948 raft_consensus.cc:2889] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:17.720289  5945 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:17.755370  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:17.756513  9497 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:17.757193  9497 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:17.758345  9497 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:17.668836  6762 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:17.688983  6793 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:17.689790  9519 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:17.765296  9519 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:17.694911  9315 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.726s	user 0.009s	sys 0.005s
I20260709 12:26:17.768551  9315 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:17.639536  9348 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:17.639494  9499 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12.
I20260709 12:26:17.799048  5945 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:17.801360  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:17.695837  9334 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:17.737577  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:17.742039  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.742267  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.654594  9511 raft_consensus.cc:697] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:17.818228  9511 consensus_queue.cc:237] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:17.818921  9264 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:17.819881  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:17.820706  5872 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:17.743897  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:17.823539  9517 consensus_meta.cc:287] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.644s	user 0.011s	sys 0.000s
W20260709 12:26:17.743950  9482 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.084s	user 0.015s	sys 0.001s
I20260709 12:26:17.744378  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:17.837110  5871 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:17.745333  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:17.682926  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:17.772845  5874 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:17.679520  9529 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 12:26:17.773223  6017 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:26:17.722621  5987 leader_election.cc:400] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12.
W20260709 12:26:17.849906  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 127433us
W20260709 12:26:17.851490  9536 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:26:17.851444  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:17.688401  9498 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:17.785929  6022 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:17.858793  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:17.860261  9376 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.003s
I20260709 12:26:17.860646  9452 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:17.860750  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:17.794351  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.771507  9315 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:17.862035  5867 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:17.847154  9482 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:17.863224  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.797613  9348 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:17.670486  9515 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.004s
I20260709 12:26:17.806027  5869 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:17.806806  6775 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:17.812240  6024 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:17.815845  5873 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:17.824322  6762 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:26:17.828033  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.823750  5947 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:17.882901  5947 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:17.807195  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:17.835330  5870 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:17.864727  5948 pending_rounds.cc:85] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 4
I20260709 12:26:17.883991  5948 pending_rounds.cc:107] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 10.5
W20260709 12:26:17.884263  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 86717.1us
I20260709 12:26:17.884419  5948 raft_consensus.cc:2889] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:17.864745  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:17.887933  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:17.888459  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:17.889156  5942 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 10: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:17.889353  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:17.889693  6016 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:17.890467  6020 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:17.891294  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:17.892481  5870 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:17.870472  6022 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:17.885941  9464 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:17.886579  9483 raft_consensus.cc:2764] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:26:17.894121  9464 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:17.894338  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.894728  9464 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:17.875296  9509 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:17.896365  9315 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:17.896586  9527 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:17.897002  9315 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:17.897351  9251 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:17.897836  9452 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:17.842121  9445 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:17.898990  5947 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:17.899042  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:17.894629  9531 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 12:26:17.848713  9360 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:17.906699  6756 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:17.905849  5840 leader_election.cc:400] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11.
I20260709 12:26:17.907212  5947 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:17.911255  5840 leader_election.cc:403] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:26:17.910775  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.913504  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.915326  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:17.916783  5945 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:17.918637  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:26:17.920215  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 489932us
I20260709 12:26:17.847427  9453 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
W20260709 12:26:17.910661  9252 consensus_meta.cc:287] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.621s	user 0.013s	sys 0.004s
I20260709 12:26:17.922681  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.923030  6016 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:26:17.917240  5987 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:17.924077  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:17.897574  9315 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:17.924510  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:17.865432  9276 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 12:26:17.902154  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:17.825785  9122 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:17.952230  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:17.954332  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:17.952888  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:17.951924  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.926268  9415 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.900682  9515 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:17.908999  9473 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:26:17.953238  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:17.930004  9070 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:17.947641  9494 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:17.915875  9251 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:26:17.916175  9360 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:17.950371  9517 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:18.075310  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:17.870472  5871 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:18.047451  5873 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:18.100061  5946 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:18.000497  9481 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:26:18.064145  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 110982us
I20260709 12:26:17.998067  9276 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11.
I20260709 12:26:18.039222  6766 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:17.992062  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:17.998445  5942 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:18.134639  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:18.051011  6768 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:26:18.074477  9415 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:18.074018  9264 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:18.047324  9458 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.009s	sys 0.017s
W20260709 12:26:18.135041  9541 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
W20260709 12:26:18.133904  9505 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.078s	user 0.000s	sys 0.009s
W20260709 12:26:18.158668  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 267630us
I20260709 12:26:18.160009  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:18.073478  9502 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:18.161770  5871 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:26:18.168676  6015 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.017s	sys 0.025s
W20260709 12:26:18.151736  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 159796us
W20260709 12:26:18.169461  6015 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.017s	sys 0.025s
I20260709 12:26:18.169891  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:18.155388  9458 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.009s	sys 0.017s
W20260709 12:26:18.203987  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.152765  9376 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:18.205435  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 45549.7us
I20260709 12:26:18.232726  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:18.155337  9488 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:18.225327  5915 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:18.150252  9144 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:18.163836  9482 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:18.256201  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:18.258383  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:18.259219  5945 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:18.259501  5872 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:26:18.261292  5838 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:18.265815  5990 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:18.268752  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:18.293216  5947 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:26:18.237599  9276 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:26:18.294514  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 58463.4us
I20260709 12:26:18.236888  9488 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.250898  9356 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:18.296042  5947 pending_rounds.cc:85] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 5
I20260709 12:26:18.315666  5947 pending_rounds.cc:107] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:26:18.316164  5947 raft_consensus.cc:2889] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:18.253399  9415 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:18.263123  9498 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:18.263751  9530 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.327220  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 554) took 1297 ms. Trace:
I20260709 12:26:18.328713  5947 rpcz_store.cc:276] 0709 12:26:17.030337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.032024 (+  1687us) service_pool.cc:224] Handling call
0709 12:26:17.032465 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:17.823632 (+791167us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b54004ceb20. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bfadc0 00007f39d2c851c0 00007f39d2c86231 00007f39d2c82d8f 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c
0709 12:26:18.326599 (+502967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.327133 (+   534us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1546646912}
W20260709 12:26:18.332383  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 181658us
I20260709 12:26:18.330159  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:18.347378  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:18.338418  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 82299.7us
I20260709 12:26:18.319304  9261 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:18.346135  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:18.263183  9144 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:18.326139  9532 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:26:18.315464  9356 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:18.350445  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:18.314942  9488 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:18.318773  9122 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:18.293835  9452 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:26:18.450493  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:18.451646  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:18.397802  9360 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:18.333025  9415 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:18.419476  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:18.479010  5947 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 12: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:18.478909  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:18.437609  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 168936us
I20260709 12:26:18.452566  6768 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:18.481073  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:18.446163  9261 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:18.334607  9530 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:18.406699  9521 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:18.387966  9498 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:18.479756  6766 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
W20260709 12:26:18.482738  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 132419us
I20260709 12:26:18.501986  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:18.432126  9511 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:18.477144  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 989) took 1271 ms. Trace:
I20260709 12:26:18.488263  6757 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 12:26:18.507488  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.438668  9356 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:18.467485  9515 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:18.432565  9251 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:18.466504  9565 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.504704  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:18.505818  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:18.507155  6023 rpcz_store.cc:276] 0709 12:26:17.205438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.205575 (+   137us) service_pool.cc:224] Handling call
0709 12:26:17.446415 (+240840us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:17.484347 (+ 37932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.476842 (+992495us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54006b9ed0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:18.477067 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242918144}
I20260709 12:26:18.509361  6022 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:26:18.492062  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:18.523808  6762 raft_consensus.cc:2393] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:18.478345  9415 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:18.492588  9547 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.359s	user 0.001s	sys 0.001s
I20260709 12:26:18.522748  6775 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:18.477357  9473 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:18.529227  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:18.491503  9122 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:18.498265  9276 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 12 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:18.514878  9546 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.368s	user 0.001s	sys 0.001s
I20260709 12:26:18.581032  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:18.507107  9511 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:18.496924  9360 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.590400  9261 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:18.541960  9415 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:18.618006  6775 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:18.598881  9511 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:18.648584  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1331) took 1307 ms. Trace:
W20260709 12:26:18.649658  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 147620us
I20260709 12:26:18.595402  9502 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:18.651217  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 164260us
I20260709 12:26:18.653422  5944 rpcz_store.cc:276] 0709 12:26:17.341610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.439367 (+ 97757us) service_pool.cc:224] Handling call
0709 12:26:17.439880 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:17.440211 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:17.490621 (+ 50410us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.407178 (+916557us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b1800023560. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:26:18.648150 (+240972us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:18.648218 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:18.648273 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.648362 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.648513 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2200,"prepare.queue_time_us":50026,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":11876,"raft.queue_time_us":10024,"spinlock_wait_cycles":25157888,"thread_start_us":230968,"threads_started":1}
I20260709 12:26:18.657092  5840 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:18.542779  9251 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:18.647755  9547 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:18.612689  9567 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.540552  9565 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:18.637367  9548 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.399s	user 0.000s	sys 0.002s
I20260709 12:26:18.674450  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:18.659951  9573 raft_consensus.cc:2764] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
W20260709 12:26:18.671279  9448 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:26:18.682436  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:18.666550  9581 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.613698  9360 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:18.671502  9452 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:18.716674  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:18.717371  9553 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.376s	user 0.001s	sys 0.001s
W20260709 12:26:18.719349  9564 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:18.680074  9334 consensus_queue.cc:1059] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 12:26:18.731819  9577 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:26:18.732343  9572 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
W20260709 12:26:18.725452  9445 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.827s	user 0.005s	sys 0.006s
I20260709 12:26:18.734778  9251 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:18.735906  9559 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.399s	user 0.000s	sys 0.002s
W20260709 12:26:18.736613  9557 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 12:26:18.736804  9360 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:18.737535  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 952) took 1419 ms. Trace:
W20260709 12:26:18.742262  9560 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.160s	user 0.001s	sys 0.001s
I20260709 12:26:18.738588  9565 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:18.739704  9563 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:26:18.740552  9530 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:18.741585  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1332) took 1196 ms. Trace:
I20260709 12:26:18.684309  9571 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:18.670563  9566 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:18.713042  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:18.714983  9561 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:18.726696  9203 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:18.679162  9567 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:18.762835  5943 rpcz_store.cc:276] 0709 12:26:17.545679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.545885 (+   206us) service_pool.cc:224] Handling call
0709 12:26:17.569014 (+ 23129us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:17.711918 (+142904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.741214 (+1029296us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400633654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:18.741507 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326755584}
I20260709 12:26:18.753201  6785 rpcz_store.cc:276] 0709 12:26:17.318003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.484767 (+166764us) service_pool.cc:224] Handling call
0709 12:26:17.526508 (+ 41741us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.543297 (+ 16789us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.543350 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.754803 (+211453us) spinlock_profiling.cc:243] Waited 4.12 ms on lock 0x7b18000191e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb642a6 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a
0709 12:26:17.756741 (+  1938us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b2c0001ffa8. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:26:17.757393 (+   652us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.129491 (+372098us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:18.145044 (+ 15553us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.149220 (+  4176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.149304 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.737266 (+587962us) raft_consensus.cc:1685] finished
0709 12:26:18.737291 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.737450 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21467,"prepare.queue_time_us":15246,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":1941,"raft.queue_time_us":31818,"raft.run_cpu_time_us":395,"raft.run_wall_time_us":17116,"spinlock_wait_cycles":76219008,"thread_start_us":343276,"threads_started":1,"wal-append.queue_time_us":895183}
I20260709 12:26:18.748363  9502 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:26:18.768847  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.682776  9276 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:18.751410  9453 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:18.775543  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 966) took 1458 ms. Trace:
I20260709 12:26:18.787614  6019 rpcz_store.cc:276] 0709 12:26:17.316999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.769452 (+452453us) service_pool.cc:224] Handling call
0709 12:26:17.769978 (+   526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.770488 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.770526 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.771203 (+   677us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.129036 (+357833us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:18.129101 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.129203 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.775195 (+645992us) raft_consensus.cc:1685] finished
0709 12:26:18.775225 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.775447 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":153,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":106,"thread_start_us":357289,"threads_started":1,"wal-append.queue_time_us":550194}
W20260709 12:26:18.774511  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 61602.2us
W20260709 12:26:18.783735  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.714855  9581 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:18.761464  9070 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:18.763590  9058 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:18.769912  9251 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.763326  9122 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:18.812736  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:18.813900  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:18.813457  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 157203us
I20260709 12:26:18.771973  9203 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:18.820204  5915 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:18.783737  9565 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:26:18.821092  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 211789us
I20260709 12:26:18.783602  9144 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:18.785172  9334 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:18.829744  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 113173us
I20260709 12:26:18.830957  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:18.831032  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:18.823230  9509 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:18.835413  6609 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:18.820900  9445 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:18.838088  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:18.793059  9453 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:18.826030  9506 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:26:18.891204  9447 consensus_meta.cc:287] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.155s	user 0.013s	sys 0.004s
I20260709 12:26:18.896644  9506 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:18.835810  9144 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.846637  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 609) took 1020 ms. Trace:
I20260709 12:26:18.881722  9579 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:26:18.849964  9452 raft_consensus.cc:3055] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:18.771719  9586 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:26:18.855670  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:18.892772  9482 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:18.912611  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:18.857666  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:18.860049  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 967) took 1542 ms. Trace:
I20260709 12:26:18.914480  6766 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:18.914762  6020 rpcz_store.cc:276] 0709 12:26:17.317329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.852373 (+535044us) service_pool.cc:224] Handling call
0709 12:26:17.890254 (+ 37881us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:17.890895 (+   641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.859770 (+968875us) spinlock_profiling.cc:243] Waited 37.4 ms on lock 0x7b54001603d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:18.859966 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86066432}
W20260709 12:26:18.860618  9348 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.960s	user 0.010s	sys 0.003s
I20260709 12:26:18.916389  9415 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:18.864923  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 608) took 1081 ms. Trace:
I20260709 12:26:18.865798  9575 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:18.837546  9058 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
W20260709 12:26:18.869323  9558 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.528s	user 0.000s	sys 0.001s
I20260709 12:26:18.868378  9511 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:18.871558  9567 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:18.786305  9561 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:18.928622  9511 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:18.929302  9561 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:18.820593  9251 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:18.931742  9251 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:18.906059  9144 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:18.874118  5943 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:18.877498  5870 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:18.922343  9482 raft_consensus.cc:697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:18.881788  9122 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:18.934688  9482 consensus_queue.cc:237] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:18.937520  9506 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:18.917877  5867 rpcz_store.cc:276] 0709 12:26:17.783912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.049824 (+265912us) service_pool.cc:224] Handling call
0709 12:26:18.062013 (+ 12189us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:18.309851 (+247838us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:18.504584 (+194733us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.759123 (+254539us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:18.772627 (+ 13504us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.772780 (+   153us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:18.792138 (+ 19358us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.812907 (+ 20769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.812995 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.864638 (+ 51643us) raft_consensus.cc:1685] finished
0709 12:26:18.864660 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.864857 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74138,"prepare.queue_time_us":370494,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":1777,"raft.queue_time_us":113243,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":19144,"spinlock_wait_cycles":4417920,"thread_start_us":567198,"threads_started":3,"wal-append.queue_time_us":243695}
W20260709 12:26:18.939394  9584 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:26:18.939693  9453 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:18.831166  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:18.885010  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 560) took 1079 ms. Trace:
W20260709 12:26:18.943202  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 112103us
I20260709 12:26:18.887579  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:18.888530  5874 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:26:18.889546  9315 consensus_meta.cc:287] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.932s	user 0.016s	sys 0.000s
I20260709 12:26:18.943528  9356 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:18.892990  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 57662.3us
I20260709 12:26:18.945017  6024 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:18.905857  9452 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:18.946620  9452 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:18.948158  9452 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:18.894111  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 990) took 1688 ms. Trace:
I20260709 12:26:18.894356  6785 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:18.823737  9565 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:18.943450  5948 rpcz_store.cc:276] 0709 12:26:17.806252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.806416 (+   164us) service_pool.cc:224] Handling call
0709 12:26:17.864478 (+ 58062us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:17.906499 (+ 42021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.906550 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:17.907384 (+   834us) log.cc:844] Serialized 52 byte log entry
0709 12:26:18.199803 (+292419us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:18.232453 (+ 32650us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.234421 (+  1968us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.234654 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.234757 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.872743 (+637986us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b1800329560. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb96522 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:26:18.882480 (+  9737us) raft_consensus.cc:1685] finished
0709 12:26:18.882530 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.884908 (+  2378us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59121,"prepare.queue_time_us":671145,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":6324,"raft.queue_time_us":16279,"raft.run_cpu_time_us":524,"raft.run_wall_time_us":41786,"spinlock_wait_cycles":29791488,"thread_start_us":218385,"threads_started":1,"wal-append.queue_time_us":665771}
I20260709 12:26:18.951053  9565 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:18.899535  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:18.950176  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:18.952880  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:18.953612  5945 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:18.848762  9480 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:18.901855  5873 rpcz_store.cc:276] 0709 12:26:17.826740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.133145 (+306405us) service_pool.cc:224] Handling call
0709 12:26:18.151005 (+ 17860us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:18.167549 (+ 16544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:18.167582 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:18.199634 (+ 32052us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.757837 (+558203us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:18.757885 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.757977 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.846358 (+ 88381us) raft_consensus.cc:1685] finished
0709 12:26:18.846384 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.846555 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7798,"raft.queue_time_us":53887,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":7498,"thread_start_us":457987,"threads_started":1,"wal-append.queue_time_us":583836}
I20260709 12:26:18.832062  9334 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:18.908416  6017 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 12: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:18.836522  9571 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:18.833240  9203 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:18.918540  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:18.986783  6018 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:18.920643  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:18.988112  6020 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:18.922923  5915 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:18.921957  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:18.926882  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:18.944155  9315 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:18.890381  9464 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.995s	user 0.009s	sys 0.007s
I20260709 12:26:18.951337  9509 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:18.952044  5942 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:18.926460  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:19.004824  9464 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:18.874136  9581 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:18.922950  9451 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:18.916617  9348 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:18.951107  9261 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:18.969758  5840 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:18.973871  9334 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:18.972163  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:18.999603  9555 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.612s	user 0.001s	sys 0.003s
I20260709 12:26:18.949047  6015 rpcz_store.cc:276] 0709 12:26:17.205916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.206052 (+   136us) service_pool.cc:224] Handling call
0709 12:26:17.306082 (+100030us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.306525 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:26:18.170019 (+863494us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.432838 (+262819us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:18.442322 (+  9484us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.442607 (+   285us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:18.469031 (+ 26424us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.469784 (+   753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.469885 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.893788 (+423903us) raft_consensus.cc:1685] finished
0709 12:26:18.893821 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.894026 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3523,"prepare.queue_time_us":599100,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":94,"raft.queue_time_us":451491,"raft.run_cpu_time_us":597,"raft.run_wall_time_us":63682,"thread_start_us":1040232,"threads_started":2,"wal-append.queue_time_us":283393}
I20260709 12:26:18.992118  6024 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
W20260709 12:26:18.992848  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 70004.7us
I20260709 12:26:18.995095  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:19.002019  5945 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
W20260709 12:26:19.003388  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 81509.8us
I20260709 12:26:19.027876  5987 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:19.007169  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:18.949345  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:19.030751  5870 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:18.966459  5944 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:26:19.030988  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 61419us
I20260709 12:26:19.017558  5941 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:18.896988  9447 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:19.018893  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:18.973079  9144 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:19.033046  5871 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:19.029567  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:19.026439  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:19.006826  9453 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:19.042716  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 955) took 1450 ms. Trace:
I20260709 12:26:19.047047  5868 rpcz_store.cc:276] 0709 12:26:17.592259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.592417 (+   158us) service_pool.cc:224] Handling call
0709 12:26:19.042468 (+1450051us) spinlock_profiling.cc:243] Waited 66.5 ms on lock 0x7b5400119754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:19.042625 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152924032}
I20260709 12:26:19.016315  9203 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:19.016232  9586 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:19.043591  5840 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:19.056295  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:19.048667  5947 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:18.970777  9582 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:26:18.924540  9575 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:19.007618  9502 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:26:19.060281  5940 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 10: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:19.004961  9509 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:19.061343  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:19.040791  9276 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:18.972790  9480 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:19.042276  9581 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:19.054297  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:19.018314  9451 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:19.050206  9453 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.068534  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:19.072152  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:19.076381  5873 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:19.078222  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:19.078665  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:19.056957  9459 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:19.083479  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:19.084676  5873 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
W20260709 12:26:19.081418  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 64136.2us
I20260709 12:26:19.064536  9579 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:26:19.085673  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.081884  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:19.013823  9561 consensus_queue.cc:1175] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" responder_term: 7 status { last_received { term: 6 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:19.059530  9483 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:26:19.087218  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:19.078805  9586 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:19.120857  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 66664.6us
I20260709 12:26:19.188603  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1327) took 1948 ms. Trace:
I20260709 12:26:19.205526  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:19.181857  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:19.177793  5948 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:19.096331  6019 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:19.204780  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:19.063073  9144 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:26:19.210237  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 123229us
I20260709 12:26:19.210912  5915 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:19.088189  9599 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:26:19.078042  9581 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:19.205441  5946 rpcz_store.cc:276] 0709 12:26:17.240710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.241029 (+   319us) service_pool.cc:224] Handling call
0709 12:26:17.241578 (+   549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.112966 (+871388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.188100 (+1075134us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b54003093d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:19.188468 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1973638656}
I20260709 12:26:19.085537  9459 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.174559  9567 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:19.095971  9480 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:19.083856  9356 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:19.224591  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:19.238973  5945 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:19.088425  9589 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.173s
I20260709 12:26:19.090449  9595 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:19.080415  9122 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:19.231323  9530 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:19.242424  9530 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.242901  9530 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:19.101821  9315 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:19.243620  9509 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:19.244303  9509 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:19.244797  9509 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:19.246321  9447 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:19.248021  9575 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:19.248452  5945 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
W20260709 12:26:19.167551  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 85745.1us
I20260709 12:26:19.132318  9261 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:19.080461  9598 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:19.087262  9506 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:19.125797  9494 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.231071  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:19.079427  9453 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:19.230966  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:19.224531  9581 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:19.218508  9579 raft_consensus.cc:697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 9 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:19.235061  9488 consensus_meta.cc:287] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.743s	user 0.014s	sys 0.002s
I20260709 12:26:19.220086  9606 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:19.221057  9470 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:19.232507  9459 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:19.236114  9603 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:26:19.291237  9470 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:19.293287  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1013) took 1556 ms. Trace:
I20260709 12:26:19.294068  5940 rpcz_store.cc:276] 0709 12:26:17.737166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.737305 (+   139us) service_pool.cc:224] Handling call
0709 12:26:19.293069 (+1555764us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54008b01d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:19.293222 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2880628992}
I20260709 12:26:19.221448  9448 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:19.297544  9448 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:19.241869  9122 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:19.184463  9252 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:19.099867  9451 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:19.280273  9488 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:19.297607  5865 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 948) took 1919 ms (client timeout 1527 ms). Trace:
I20260709 12:26:19.299870  5872 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
W20260709 12:26:19.299865  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.085959  9251 consensus_queue.cc:1059] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.143s
I20260709 12:26:19.088853  9276 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:19.215073  9144 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:19.234766  9464 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:19.260885  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:19.263417  6785 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
W20260709 12:26:19.260599  9517 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.127s	user 0.021s	sys 0.004s
I20260709 12:26:19.264842  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:19.261694  9595 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:19.270728  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.272022  9348 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:19.256461  9598 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.337950  9598 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:19.282197  9261 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:19.331189  9589 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:19.341123  9548 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.657s	user 0.000s	sys 0.002s
I20260709 12:26:19.256088  9415 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:19.288684  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1265) took 1451 ms. Trace:
I20260709 12:26:19.259121  9447 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.302376  9488 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:19.326618  5942 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:19.328088  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 67301.8us
I20260709 12:26:19.299508  9252 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:19.248324  9334 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:19.278543  9356 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:26:19.303754  5865 rpcz_store.cc:269] 0709 12:26:17.378062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.378810 (+   748us) service_pool.cc:224] Handling call
0709 12:26:19.297362 (+1918552us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540069e954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:19.297537 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2904470784}
I20260709 12:26:19.304229  9470 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:19.370210  5869 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:19.329013  9272 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:19.332393  9517 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.213s	user 0.021s	sys 0.004s
I20260709 12:26:19.345733  9606 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:19.380071  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:19.299361  9122 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:19.347481  5871 rpcz_store.cc:276] 0709 12:26:17.837848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.246471 (+408623us) service_pool.cc:224] Handling call
0709 12:26:18.246907 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.045794 (+798887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.288343 (+242549us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b5400186d54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:19.288580 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1807209856}
I20260709 12:26:19.328693  9144 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:19.361297  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:19.274200  9494 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:19.368294  9252 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:19.372929  6022 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:19.387609  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:19.359683  9573 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.374915  9334 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:19.346256  9595 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:19.369249  9565 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:19.376050  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:19.353436  9251 consensus_queue.cc:1243] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:19.383828  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:19.398824  6015 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:19.364722  9550 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 1.052s	user 0.000s	sys 0.006s
I20260709 12:26:19.390020  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:19.392230  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:19.387904  9348 raft_consensus.cc:2764] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:26:19.391880  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:19.392515  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:19.396402  6020 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:19.405305  5871 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:19.359737  9527 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:26:19.378993  9517 raft_consensus.cc:697] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:19.360090  9447 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:19.298890  9448 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:19.403462  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:19.392995  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:19.391924  9586 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:19.406069  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:19.402050  9252 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:19.425532  5870 raft_consensus.cc:2376] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14. Re-sending same reply.
I20260709 12:26:19.425786  5874 raft_consensus.cc:2368] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
I20260709 12:26:19.426920  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:19.427891  5870 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:26:19.404604  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:19.429394  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:19.432685  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:19.393765  9261 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.419636  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:19.410218  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:19.417303  6015 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:19.433064  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:19.434082  5870 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:19.442073  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:19.443113  6023 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:19.445979  9494 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:19.413950  9479 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:19.447268  9479 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:19.447722  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:19.448562  6020 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:19.434779  9348 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.422475  9589 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.395530  9573 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:19.450682  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:19.474541  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:19.453063  5871 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:19.460201  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:19.453052  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:19.505093  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 52135.1us
I20260709 12:26:19.455019  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:19.384001  9579 consensus_queue.cc:237] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:19.509235  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:19.631101  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 943) took 2335 ms. Trace:
I20260709 12:26:19.635979  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1019) took 1701 ms. Trace:
W20260709 12:26:19.580576  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 106151us
I20260709 12:26:19.658898  6756 rpcz_store.cc:276] 0709 12:26:17.295878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.296915 (+  1037us) service_pool.cc:224] Handling call
0709 12:26:17.297251 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.910193 (+612942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.630616 (+1720423us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b5400006c54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:19.630993 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1400689536}
W20260709 12:26:19.579423  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 52360.9us
I20260709 12:26:19.464085  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:19.661114  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:19.630575  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:19.661767  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 197760us
W20260709 12:26:19.577407  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 105763us
I20260709 12:26:19.658473  5949 rpcz_store.cc:276] 0709 12:26:17.934352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.934664 (+   312us) service_pool.cc:224] Handling call
0709 12:26:17.990910 (+ 56246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.136775 (+145865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:18.136810 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:18.136853 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:18.136888 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.635556 (+1498668us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54006770d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:19.635649 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.635894 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334909696}
I20260709 12:26:19.637251  5947 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:19.508435  6023 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:19.623806  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:19.659668  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:19.672502  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:19.588382  9567 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:19.674263  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:19.678918  6020 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:19.679284  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:19.671298  5946 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:19.666618  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:19.430673  9586 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.458714  9447 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:19.682092  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:19.683130  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:19.435714  9509 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:19.689337  6017 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:19.576953  9445 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:19.686990  9614 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:19.697228  6022 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:19.690510  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:19.659705  9494 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.694005  6020 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:19.467072  9488 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.220s
I20260709 12:26:19.669606  9566 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:19.709292  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:19.454005  9589 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:19.713174  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:19.685606  9442 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:19.508188  9576 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:19.707695  9452 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:26:19.697824  9586 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:19.698964  9445 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.584096  9606 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:19.697824  5944 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 7: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:19.699005  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:19.708153  9252 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:19.716228  9614 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:19.718740  9442 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:19.681284  9561 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:26:19.695626  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:19.767589  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:19.768996  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:19.770279  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:19.750234  9506 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:19.701584  5987 leader_election.cc:400] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
I20260709 12:26:19.772234  5987 leader_election.cc:403] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:26:19.767581  6021 raft_consensus.cc:2410] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:26:19.701905  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:19.702556  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:19.449546  9479 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:19.710012  6020 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:19.644279  9448 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:19.713032  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:19.714938  6785 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 12: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:19.456111  9573 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:19.463748  9276 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.236s
I20260709 12:26:19.721834  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:19.714795  9512 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.715523  9494 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:19.726511  9589 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:19.692602  9567 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:19.450639  9348 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:19.455684  9261 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:19.713246  9566 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:19.638386  9517 consensus_queue.cc:237] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:19.733470  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:19.734344  9620 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:19.811197  9473 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10.
I20260709 12:26:19.470009  9581 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:19.747843  5866 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 960) took 2007 ms (client timeout 1522 ms). Trace:
I20260709 12:26:19.733667  9576 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.756593  9598 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:19.759686  6768 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:19.741964  9459 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:19.763592  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:19.883267  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 215598us
I20260709 12:26:19.883488  6019 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:19.730479  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:19.467506  6793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:19.776871  9122 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:19.812184  5940 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: []
W20260709 12:26:19.816605  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 119950us
I20260709 12:26:19.817565  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:19.824056  6611 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:19.751567  9442 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:19.729444  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:19.814813  9453 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.000s
I20260709 12:26:19.739602  9445 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:19.844579  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:19.848459  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.754366  6017 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:19.876221  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:19.896267  6775 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:19.854604  9203 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
W20260709 12:26:19.874895  5866 rpcz_store.cc:269] 0709 12:26:17.740570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.741748 (+  1178us) service_pool.cc:224] Handling call
0709 12:26:19.181610 (+1439862us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b58002e2f98. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:26:19.747554 (+565944us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b5400388954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:19.747743 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":421640832}
I20260709 12:26:19.880846  5948 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:19.734242  9447 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.121s
I20260709 12:26:19.882860  5868 raft_consensus.cc:2410] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:26:19.885244  6793 raft_consensus.cc:606] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:26:19.904829  5945 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:26:19.905288  6793 raft_consensus.cc:614] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:19.812063  5872 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:26:19.906060  5872 pending_rounds.cc:85] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 5
I20260709 12:26:19.818697  9448 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:19.889148  5865 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:19.882457  9576 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:19.843874  9566 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:19.816705  9607 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
W20260709 12:26:19.892755  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.816767  9520 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:26:19.749413  9586 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:19.901485  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:19.903115  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:19.906335  5872 pending_rounds.cc:107] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:26:19.906765  6018 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 10: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:26:19.907868  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 83881.5us
I20260709 12:26:19.886415  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:19.878753  9452 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:26:19.844954  9573 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:26:19.910105  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:19.911921  9264 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:19.751525  9614 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:19.918718  5872 raft_consensus.cc:2889] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:19.899413  9473 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:26:19.920945  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:19.779421  9497 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 12:26:19.923378  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:19.924259  6756 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:19.924643  9252 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.970613  9252 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:19.945960  9272 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:26:19.834822  9512 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:19.973805  9519 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:19.932482  9448 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:19.975793  9448 consensus_queue.cc:1059] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:19.935248  9348 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:19.937568  9530 consensus_meta.cc:287] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.681s	user 0.012s	sys 0.000s
I20260709 12:26:19.978842  9530 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:19.979460  9348 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:19.980077  9348 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:19.981216  9530 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:19.913873  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:19.983947  9565 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.001s
I20260709 12:26:19.984793  6775 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:19.919581  9567 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:19.990094  5947 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:19.939611  6016 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:19.875417  9620 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:19.920816  6766 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:19.946851  6019 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:19.888912  9144 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:19.927991  6020 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:19.969460  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:19.990252  9261 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:19.941969  6015 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:26:19.993611  6793 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:19.904412  9453 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:20.002945  6023 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:20.013046  9497 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.002s
W20260709 12:26:20.018297  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 242695us
I20260709 12:26:19.945891  9568 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.053411  9568 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:20.056629  9568 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:20.006484  5942 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:19.946485  9452 consensus_queue.cc:1243] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:26:19.992518  9561 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:20.043774  9586 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:20.045974  9497 consensus_queue.cc:1243] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:20.046159  9144 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:20.047488  9598 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:20.048111  5946 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:20.049408  9620 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:20.052459  9581 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:20.064882  6775 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:20.122771  6775 raft_consensus.cc:606] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:26:20.123298  6775 raft_consensus.cc:614] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:20.003777  9519 raft_consensus.cc:3055] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:20.000321  9509 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:20.071416  5943 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:20.117056  9494 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:20.061028  9530 raft_consensus.cc:3055] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:20.145556  9252 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:20.056861  9459 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:20.065068  9561 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:20.167526  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:20.168081  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.173321  9512 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:20.179307  6773 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:20.186519  9561 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:20.188073  9561 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:20.190980  9453 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:20.195744  5990 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:20.207743  5915 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:20.209344  9442 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:20.143152  9519 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:20.218990  5990 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:20.215701  5838 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:20.149957  9530 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:20.223902  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 334544us
I20260709 12:26:20.235240  5942 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:26:20.248309  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 404154us
I20260709 12:26:20.150128  9509 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:20.251075  6023 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:26:20.254261  6023 pending_rounds.cc:85] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 7
I20260709 12:26:20.249825  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:20.254644  6023 pending_rounds.cc:107] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:26:20.255080  6023 raft_consensus.cc:2889] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:20.255249  9470 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:20.255980  9218 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:20.256453  9526 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:20.257020  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:20.257407  9526 raft_consensus.cc:3055] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:20.257933  9527 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:20.258285  9512 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:20.257902  9526 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:20.217990  9252 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:20.261564  5874 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:20.258401  9527 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:20.267719  5947 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:20.229555  9581 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:20.229410  9315 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:20.248384  9447 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:20.303136  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:20.304205  5865 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:20.285895  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:20.284428  9594 raft_consensus.cc:3055] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:20.229003  9579 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:26:20.270177  9571 raft_consensus.cc:3055] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:20.249332  9122 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:20.251760  9459 consensus_queue.cc:1059] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:26:20.271569  9470 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:20.343034  9509 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:20.314401  9594 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:20.287446  9315 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.252178  9261 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 12:26:20.312722  9447 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.354477  5865 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:20.317019  9571 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:20.317381  9581 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:20.391517  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:20.407847  9586 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:20.408564  9586 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:20.409072  9586 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:20.348618  9607 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.357887  9315 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:20.313090  9526 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:20.329166  9519 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:20.450402  9519 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:20.395053  9447 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:20.446264  9497 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:26:20.446938  9637 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:26:20.448619  9479 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:26:20.455590  9566 consensus_meta.cc:287] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.530s	user 0.013s	sys 0.000s
I20260709 12:26:20.439795  9614 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.346894  9527 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:20.459172  9445 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.547s	user 0.004s	sys 0.009s
I20260709 12:26:20.451545  9635 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:20.446393  9630 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 12:26:20.484750  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.491576  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:20.460304  9567 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.438647  9607 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:20.494299  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:20.462049  9608 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:20.506938  5867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 968) took 1549 ms (client timeout 1682 ms). Trace:
I20260709 12:26:20.507653  9607 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:20.485927  9566 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:20.511837  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 617) took 1548 ms. Trace:
I20260709 12:26:20.540134  5869 rpcz_store.cc:276] 0709 12:26:18.963657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.094949 (+131292us) service_pool.cc:224] Handling call
0709 12:26:19.095391 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.376534 (+281143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.511471 (+1134937us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54001446d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:20.511744 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":631494528}
I20260709 12:26:20.512192  9527 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:20.512975  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 984) took 1282 ms. Trace:
W20260709 12:26:20.514312  9631 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.185s	user 0.002s	sys 0.000s
I20260709 12:26:20.458061  9594 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:20.576627  9594 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:20.514978  9587 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:26:20.515430  9576 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.001s
I20260709 12:26:20.578850  9122 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.578752  9576 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:20.579715  9122 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:20.517212  5840 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:20.582022  6021 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 11. Current term is 12. Ops: []
W20260709 12:26:20.582687  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 65545us
I20260709 12:26:20.583911  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:20.488660  9614 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:20.477325  9488 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:20.586679  9488 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:20.588096  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:20.588913  9488 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:20.590698  9315 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:20.592581  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:20.593895  6017 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:20.595628  9447 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:20.495359  9445 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:20.598558  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.499491  9261 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:20.600000  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:20.602337  5940 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 12:26:20.529618  5867 rpcz_store.cc:269] 0709 12:26:18.957769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.058634 (+100865us) service_pool.cc:224] Handling call
0709 12:26:20.506608 (+1447974us) spinlock_profiling.cc:243] Waited 955 ms on lock 0x7b5400028354. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:20.506844 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2197260672}
I20260709 12:26:20.542538  6024 rpcz_store.cc:276] 0709 12:26:19.230650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.231537 (+   887us) service_pool.cc:224] Handling call
0709 12:26:19.231855 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.249340 (+ 17485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.249370 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:19.249410 (+    40us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:19.249437 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.512649 (+1263212us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b54003638d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:20.512721 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.512894 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39659392}
I20260709 12:26:20.606379  6611 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:20.608383  6611 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:20.609437  9272 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:20.609905  9488 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:20.610735  9585 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:20.611482  9585 raft_consensus.cc:697] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:20.614524  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:20.536211  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:20.552048  9606 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:26:20.496833  9635 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:20.492622  9581 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:20.479951  9571 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:20.478744  9479 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:20.537518  9608 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:20.575481  9598 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.585817  6023 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:20.615868  5942 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
W20260709 12:26:20.555275  9636 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:26:20.621556  5867 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:26:20.530663  9526 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:20.624788  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 985) took 1166 ms. Trace:
I20260709 12:26:20.630903  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:20.632715  5870 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:20.651144  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:20.653000  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:20.612684  9488 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:20.481590  9470 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:20.661814  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:20.652658  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:20.673764  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:20.677382  5872 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 10: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:20.627421  9561 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:20.680238  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:20.612686  9585 consensus_queue.cc:237] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:20.624516  9587 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:20.657434  6609 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:20.538715  9567 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:20.669605  6018 rpcz_store.cc:276] 0709 12:26:19.382554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.382708 (+   154us) service_pool.cc:224] Handling call
0709 12:26:20.548543 (+1165835us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b5400764654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:20.548801 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1110883712}
I20260709 12:26:20.636626  9442 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:20.675674  9562 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:20.686622  9629 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.374s	user 0.004s	sys 0.000s
I20260709 12:26:20.686705  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:20.680289  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:20.659917  9315 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:20.689437  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:20.683805  9561 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:20.658267  9483 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:26:20.682927  9488 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:20.698132  9272 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:20.711763  9315 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.712766  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:20.742403  6778 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:20.717931  9576 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:20.707736  6023 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:20.718668  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:20.671103  9598 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:20.748960  6018 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:20.725603  5990 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:20.750896  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:20.751353  5990 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:20.688997  9647 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:20.739925  9447 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:20.743630  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:20.744643  6609 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:20.746692  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:20.754117  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 239094us
I20260709 12:26:20.729403  9561 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:20.754463  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:20.756525  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.756349  9595 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:20.752578  9506 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.757319  9608 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:20.773372  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:20.775801  5870 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:26:20.761862  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:20.764609  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:20.777732  5874 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:26:20.778071  6775 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:20.766019  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:20.764756  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:20.781594  5944 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:20.781935  5874 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:20.783532  5944 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:20.767638  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:20.784700  6778 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:20.741866  9272 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.707600  9562 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:20.799190  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:20.800771  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:20.802544  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:20.795650  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:20.804395  5871 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:20.763240  9512 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:20.809969  6023 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:20.770830  9634 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:20.771392  9517 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:20.771106  9497 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:20.771770  9442 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:20.830044  6019 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:20.820127  6775 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:20.749818  9315 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:20.805961  9641 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.220s	user 0.000s	sys 0.002s
I20260709 12:26:20.834465  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:20.834723  5867 raft_consensus.cc:2410] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 12 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:20.800073  9585 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.806819  9272 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:20.752399  9647 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:20.793784  9608 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:26:20.830540  9415 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:26:20.841339  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 807) took 1127 ms. Trace:
I20260709 12:26:20.842464  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:20.779538  9506 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:20.848307  6762 rpcz_store.cc:276] 0709 12:26:19.713509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.713834 (+   325us) service_pool.cc:224] Handling call
0709 12:26:19.727904 (+ 14070us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:19.728321 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.728354 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:19.728914 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.009482 (+280568us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:20.011359 (+  1877us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.018762 (+  7403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.018877 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.653612 (+634735us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:20.840981 (+187369us) raft_consensus.cc:1685] finished
0709 12:26:20.841098 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.841255 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1109,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":3041,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":223,"spinlock_wait_cycles":23512960,"thread_start_us":272248,"threads_started":1,"wal-append.queue_time_us":501116}
W20260709 12:26:20.861507  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.843384  6023 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:20.833444  9566 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:20.822985  9634 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:20.864075  5874 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 6.
W20260709 12:26:20.864219  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 65960.9us
W20260709 12:26:20.866070  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 60612.1us
I20260709 12:26:20.867012  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:20.867481  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:20.847445  9517 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:26:20.823216  9586 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:20.875475  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:20.870071  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:20.865844  9634 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:20.828915  9512 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:20.832675  9497 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:20.881669  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:20.840387  9653 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:26:20.866247  9598 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:20.823587  9488 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:20.846634  9585 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:20.936501  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 814) took 1179 ms. Trace:
I20260709 12:26:20.883944  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:20.938982  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:20.902179  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:20.873212  9576 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:26:20.883220  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 83869.5us
I20260709 12:26:20.940625  9315 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:20.944075  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:20.870716  9479 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:20.944245  5869 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:26:20.946313  5947 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:20.943472  6793 rpcz_store.cc:276] 0709 12:26:19.756608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.924011 (+167403us) service_pool.cc:224] Handling call
0709 12:26:19.924391 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.024253 (+ 99862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.936105 (+911852us) spinlock_profiling.cc:243] Waited 68.5 ms on lock 0x7b5400008f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:20.936411 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157579136}
I20260709 12:26:20.891799  9562 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:20.901527  6016 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:20.955960  5990 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:20.901039  9635 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:20.898082  9567 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:20.894779  9506 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:20.966499  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:20.969995  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:20.946236  9415 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:20.975283  9315 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.975919  9594 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:20.976027  9348 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:20.977468  6766 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:20.978703  6018 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:20.891899  9647 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:20.902650  9586 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:20.994891  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:20.996601  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:20.997237  6023 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:20.979724  9576 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:21.001561  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:20.989507  9562 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.014488  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:21.017189  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:21.017303  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:21.018901  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:21.020222  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:20.984292  9561 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:20.998637  9415 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:21.049525  6762 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:21.038396  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1341) took 1992 ms. Trace:
I20260709 12:26:21.027256  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:21.026633  5942 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:21.051039  5943 rpcz_store.cc:276] 0709 12:26:19.045770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.045961 (+   191us) service_pool.cc:224] Handling call
0709 12:26:19.046390 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.125794 (+1079404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.038056 (+912262us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400633654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:21.038323 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2356437760}
I20260709 12:26:21.049957  6023 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:21.037211  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 846) took 2572 ms. Trace:
I20260709 12:26:21.023416  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:21.052449  6773 rpcz_store.cc:276] 0709 12:26:18.465172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.465451 (+   279us) service_pool.cc:224] Handling call
0709 12:26:18.476876 (+ 11425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.179807 (+1702931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.036781 (+856974us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b5400004454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:21.037105 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3913891200}
I20260709 12:26:21.052796  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:21.052796  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:21.058820  6019 pending_rounds.cc:85] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 3
I20260709 12:26:21.059221  6019 pending_rounds.cc:107] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:26:21.059460  6019 raft_consensus.cc:2889] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:21.059777  6019 pending_rounds.cc:107] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:26:21.059989  6019 raft_consensus.cc:2889] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:21.061102  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:21.063993  5949 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 10: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:21.066095  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:21.000455  9315 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:21.066021  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:21.067227  5949 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:21.073880  5943 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:21.021430  9647 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:21.075567  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:21.071731  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:21.039981  9530 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:21.080206  9530 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:21.079787  9576 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:21.038853  9506 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.077834  6611 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:21.035734  9571 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:21.083992  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:21.084751  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 213940us
I20260709 12:26:21.087584  5943 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:21.072273  9566 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:21.096216  9512 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:21.098366  9261 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:21.063284  9562 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:21.115911  6766 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:21.119890  6766 raft_consensus.cc:606] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Received request to transfer leadership to TS 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:21.121811  6766 raft_consensus.cc:614] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:21.088407  9526 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:21.129055  5873 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:21.093740  9594 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:21.130093  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 72321.1us
I20260709 12:26:21.101806  9634 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:21.137952  5840 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:21.140561  5949 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:21.083834  9581 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:26:21.104437  9566 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:21.089676  9506 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:21.131906  9585 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:21.109077  9479 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:26:21.130249  9526 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.120184  9512 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:26:21.161654  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:21.177330  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:21.178332  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:21.178877  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:21.179725  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:21.124962  9647 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:26:21.155166  9566 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:21.148159  9594 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:21.181890  5942 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 7: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:21.162843  9668 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:21.183092  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:21.154502  9447 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:21.207964  9517 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:21.213697  9517 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.214077  9218 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:21.214709  9218 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:21.175390  9526 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:21.215821  6766 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:21.215688  9218 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:21.216806  9218 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:21.225914  9272 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:21.218808  9509 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:21.227890  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:21.228416  9272 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:21.229656  9638 raft_consensus.cc:3055] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:21.227937  9509 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:21.230098  9638 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:21.219220  9585 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:21.208200  9595 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:21.220438  5868 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 976) took 1818 ms (client timeout 2416 ms). Trace:
I20260709 12:26:21.220582  6766 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
W20260709 12:26:21.298870  5868 rpcz_store.cc:269] 0709 12:26:19.402304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.409921 (+  7617us) service_pool.cc:224] Handling call
0709 12:26:21.220207 (+1810286us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b54007350d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:21.220363 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1018382080}
I20260709 12:26:21.299301  6766 raft_consensus.cc:606] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:26:21.214111  9517 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:21.196518  6017 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:21.234181  6756 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:21.230125  5990 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:21.150704  9667 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:26:21.186447  9581 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:21.210927  9598 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:21.389297  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:21.368911  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 264605us
I20260709 12:26:21.391947  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:21.388664  5697 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50588 (request call id 55) took 36019 ms (client timeout 15000 ms). Trace:
I20260709 12:26:21.392760  5949 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
W20260709 12:26:21.395659  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 165558us
W20260709 12:26:21.396257  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 82628.7us
I20260709 12:26:21.396816  5990 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:21.292824  9272 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:21.294834  9520 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:21.398211  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 222008us
I20260709 12:26:21.432114  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:21.432186  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:21.293352  9585 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:21.308619  9595 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.398100  9459 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:21.403673  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:21.407054  5990 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:21.395879  6017 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:21.292332  9447 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:21.405754  9561 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:21.435050  5838 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:21.433205  9664 log.cc:927] Time spent T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 12:26:21.461526  6017 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:21.400916  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:21.319067  9576 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 12:26:21.231281  9638 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:21.397902  9415 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:21.457319  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 50358.2us
I20260709 12:26:21.467818  5990 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:21.468958  9638 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:21.437300  9473 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:21.435497  9272 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:21.476457  9272 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:21.431041  9479 consensus_queue.cc:1059] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:26:21.478248  9479 raft_consensus.cc:3055] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:21.478653  9479 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:21.478947  9666 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:26:21.479538  9479 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:21.481287  9479 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:21.483067  6019 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:21.433241  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 243410us
I20260709 12:26:21.436563  9562 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:21.483928  5949 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:26:21.431286  9598 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:26:21.406344  9670 log.cc:927] Time spent T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:26:21.458222  5946 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:21.233078  9657 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.195s
I20260709 12:26:21.448727  9595 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:21.425251  9676 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:26:21.455207  9647 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:21.460011  9334 raft_consensus.cc:3055] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:21.486390  9684 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:26:21.490067  9585 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:21.506145  9526 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:21.491312  9261 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:26:21.508836  9261 consensus_queue.cc:1243] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:26:21.534827  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 50721us
I20260709 12:26:21.535950  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:21.536919  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.499177  5838 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:21.539325  9526 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:21.541098  5941 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:21.541425  9573 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:21.541998  9573 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:21.509070  9272 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:21.542884  9573 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:21.543274  9272 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:21.544170  9573 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:21.512897  9473 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:21.545769  9272 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:21.437583  9520 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:21.546320  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 582) took 1010 ms. Trace:
I20260709 12:26:21.546729  9684 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:21.546809  5940 rpcz_store.cc:276] 0709 12:26:20.536113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.536418 (+   305us) service_pool.cc:224] Handling call
0709 12:26:20.553745 (+ 17327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.602995 (+ 49250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.546043 (+943048us) spinlock_profiling.cc:243] Waited 48.2 ms on lock 0x7b540055ffd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:21.546258 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110765824}
I20260709 12:26:21.549101  9671 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.499290  9647 raft_consensus.cc:697] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:21.491644  9506 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:21.562772  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 455864us
I20260709 12:26:21.492784  9594 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:21.463568  9459 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:21.501973  9334 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:21.522142  9576 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:21.488425  9509 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:21.578096  9509 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.578521  9509 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:21.393875  5697 rpcz_store.cc:269] 0709 12:25:45.367449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.369876 (+  2427us) service_pool.cc:224] Handling call
0709 12:26:21.310686 (+35940810us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:06.494002 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:06.495022 (+  1020us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:06.495090 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:26:06.495118 (+    28us) tablet.cc:662] Decoding operations
0709 12:26:06.497510 (+  2392us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:06.523039 (+ 25529us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:06.530617 (+  7578us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:26:06.828274 (+297657us) tablet.cc:701] Row locks acquired
0709 12:26:06.828328 (+    54us) write_op.cc:260] PREPARE: finished
0709 12:26:06.828548 (+   220us) write_op.cc:270] Start()
0709 12:26:06.828767 (+   219us) write_op.cc:276] Timestamp: P: 1783599966828445 usec, L: 0
0709 12:26:06.828795 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:26:06.829600 (+   805us) log.cc:844] Serialized 7793 byte log entry
0709 12:26:07.267968 (+438368us) op_driver.cc:464] REPLICATION: finished
0709 12:26:07.730712 (+462744us) write_op.cc:301] APPLY: starting
0709 12:26:07.742797 (+ 12085us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:07.806799 (+ 64002us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:07.856877 (+ 50078us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:07.859809 (+  2932us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:07.860312 (+   503us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=20
0709 12:26:07.860374 (+    62us) write_op.cc:312] APPLY: finished
0709 12:26:07.860999 (+   625us) log.cc:844] Serialized 138 byte log entry
0709 12:26:07.861259 (+   260us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:07.861532 (+   273us) write_op.cc:454] Released schema lock
0709 12:26:07.890658 (+ 29126us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:07.963112 (+ 72454us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:07.963168 (+    56us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4813072,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":364868,"apply.run_cpu_time_us":7184,"apply.run_wall_time_us":214759,"num_ops":20,"prepare.queue_time_us":561164,"prepare.run_cpu_time_us":12750,"prepare.run_wall_time_us":400810,"raft.queue_time_us":176723,"raft.run_cpu_time_us":16478,"raft.run_wall_time_us":220423,"replication_time_us":419755,"thread_start_us":542015,"threads_started":4,"wal-append.queue_time_us":89732,"wal-append.run_cpu_time_us":10586,"wal-append.run_wall_time_us":1935287}]]}
I20260709 12:26:21.582286  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 857) took 1820 ms. Trace:
I20260709 12:26:21.494197  9634 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:21.524384  9447 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:21.556432  9681 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:21.569525  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:21.559478  9218 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.624617  9685 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:21.624864  9218 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:21.625555  9685 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:21.627502  9218 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:21.628598  9685 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:21.628823  9509 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:21.599653  9517 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:21.577466  9655 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:21.630640  9655 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:21.631511  9517 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:21.631704  9509 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:21.582854  6757 rpcz_store.cc:276] 0709 12:26:19.761725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.936767 (+175042us) service_pool.cc:224] Handling call
0709 12:26:19.937209 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:19.938516 (+  1307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.938552 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:19.995179 (+ 56627us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.375177 (+379998us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:20.375255 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.375372 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.580374 (+1205002us) spinlock_profiling.cc:243] Waited 26.8 ms on lock 0x7b2400156518. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfa1c9c4 00007f39cfa1aacb 00007f39cfbd5dc9 00007f39cfbd5a55 00007f39cfbd4fb0 00007f39d284b488 00007f39d284bbe8 00007f39d2823398 00007f39d281e76a 00007f39d281dbb6
0709 12:26:21.581917 (+  1543us) raft_consensus.cc:1685] finished
0709 12:26:21.581947 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.582211 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":858,"raft.queue_time_us":2026,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":26193,"spinlock_wait_cycles":61619456,"thread_start_us":279217,"threads_started":1,"wal-append.queue_time_us":473247}
I20260709 12:26:21.589499  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:21.590658  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:21.592108  5949 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:21.636096  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:21.636899  5941 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:21.564178  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 81004.6us
I20260709 12:26:21.638795  5990 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:21.609850  9634 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:21.641709  5946 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 13: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:21.611781  9459 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:21.584131  9667 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.614034  9334 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:21.617538  9571 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.000s
I20260709 12:26:21.618652  9122 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:21.556891  9566 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:21.623142  9527 raft_consensus.cc:3055] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:21.637370  5949 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:21.610621  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:21.648880  9517 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:21.650661  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:21.656903  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:21.633033  9681 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:21.666244  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 55715.7us
I20260709 12:26:21.631850  9655 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:21.556116  9671 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:21.663290  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 757) took 1990 ms. Trace:
I20260709 12:26:21.688005  6015 rpcz_store.cc:276] 0709 12:26:19.673287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.673762 (+   475us) service_pool.cc:224] Handling call
0709 12:26:19.696350 (+ 22588us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.024173 (+327823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.662906 (+1638733us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b540065b3d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:21.663189 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":564121984}
I20260709 12:26:21.664465  6793 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:26:21.655006  9682 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.649588  9598 consensus_queue.cc:1059] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:26:21.671614  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:21.674198  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:21.677454  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:21.682477  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:21.664757  9442 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:21.709417  9680 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:21.696338  9576 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:21.708261  9682 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:21.712553  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:21.713977  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:21.685024  9566 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:21.664882  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 769) took 1912 ms. Trace:
I20260709 12:26:21.714874  5866 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:21.655177  9473 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:21.714998  6022 rpcz_store.cc:276] 0709 12:26:19.752341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.821757 (+ 69416us) service_pool.cc:224] Handling call
0709 12:26:19.822153 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.836752 (+ 14599us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:19.887382 (+ 50630us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.536295 (+1648913us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b180025e580. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:26:21.664009 (+127714us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:21.664078 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:21.664135 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.664222 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.664800 (+   578us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10076,"prepare.queue_time_us":50414,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":100,"raft.queue_time_us":28008,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":17111,"spinlock_wait_cycles":47118080,"thread_start_us":117300,"threads_started":1,"wal-append.queue_time_us":1821172}
I20260709 12:26:21.716385  9517 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:21.716504  9473 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:21.664449  9527 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:21.688737  9647 consensus_queue.cc:237] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:21.718600  9527 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:21.719069  9473 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:21.658941  9594 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.720268  9594 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:21.720443  9527 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:21.698544  5868 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:21.698843  5874 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:21.695663  9688 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:21.714183  9682 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:21.715847  9203 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:21.698011  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:21.726004  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:21.730746  5874 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:21.732363  9070 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:21.736317  9203 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.736954  9203 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:21.699082  9479 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.738032  9479 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:21.687614  9571 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:21.678853  9681 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:21.741498  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:21.747130  5869 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:21.742810  9633 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:21.742998  9372 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:21.698429  9524 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.739465  9687 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:21.743149  9479 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:21.748360  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:21.757434  5948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1030) took 2096 ms (client timeout 1875 ms). Trace:
W20260709 12:26:21.787832  5948 rpcz_store.cc:269] 0709 12:26:19.660692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.660828 (+   136us) service_pool.cc:224] Handling call
0709 12:26:21.757097 (+2096269us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b5400675f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:21.757300 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":452267904}
I20260709 12:26:21.660692  9667 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:21.759495  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:21.793836  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:21.777226  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 45259.4us
W20260709 12:26:21.798130  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.779242  5941 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:21.782678  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:21.726217  9688 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:21.802233  9667 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:21.802453  9688 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:21.805864  9587 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:21.785557  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:21.798280  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:21.800158  5838 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:21.743999  9571 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:21.810124  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.810660  5874 raft_consensus.cc:2410] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:21.812345  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:21.819054  5869 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:21.820418  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:21.820466  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.764961  9203 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:21.828277  5987 leader_election.cc:336] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 2.416s (SENT)
I20260709 12:26:21.829286  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:21.765599  9682 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:21.824620  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:21.784242  9459 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.850199  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:21.807576  9566 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:21.854027  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:21.858891  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:21.782030  9524 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:21.818022  9571 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:21.813143  9667 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:21.855194  9598 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:21.826839  9595 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:21.859369  9488 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:21.863023  9526 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:21.879843  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1023) took 2264 ms. Trace:
I20260709 12:26:21.884032  5945 rpcz_store.cc:276] 0709 12:26:19.612016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.612138 (+   122us) service_pool.cc:224] Handling call
0709 12:26:19.612498 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:19.905588 (+293090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.875782 (+1970194us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b54006725d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:21.876037 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":671311616}
W20260709 12:26:21.880733  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 122923us
I20260709 12:26:21.867415  9372 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:21.882436  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 816) took 1865 ms. Trace:
I20260709 12:26:21.860327  9459 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:21.885383  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:21.885406  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:21.889966  5948 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:21.890636  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:21.892542  6609 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:21.851225  9682 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:21.874277  9479 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:26:21.865734  9566 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:21.806643  9587 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:21.881982  9685 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:21.886711  9667 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:21.920249  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.893553  9506 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:21.964752  9667 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:21.901608  9272 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:21.966372  9272 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.966910  9272 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:21.964387  9506 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:21.918602  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:21.968822  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:21.917713  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:21.886353  9526 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:21.887187  9372 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.895246  9594 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:21.883846  9488 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:21.952461  9655 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.896371  9598 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 12:26:21.960705  5866 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 7.
I20260709 12:26:21.938812  9524 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:21.943296  9595 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:21.969590  5948 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 11: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:21.984942  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:21.980178  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:21.970775  5869 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:21.971210  9488 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:22.019090  9698 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.086s	user 0.000s	sys 0.002s
W20260709 12:26:22.019228  9700 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.080s	user 0.000s	sys 0.002s
I20260709 12:26:21.978513  9682 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:21.970722  9372 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:22.023051  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:22.023881  5946 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:22.018846  9696 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.082s	user 0.002s	sys 0.000s
I20260709 12:26:21.970674  9526 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:21.951745  9459 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:21.972875  9691 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:22.032939  9691 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:22.018385  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:22.024994  9688 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:22.055588  5946 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:22.056766  9688 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:21.994069  9697 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.109s	user 0.003s	sys 0.000s
I20260709 12:26:22.033903  9681 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:22.034044  9691 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:22.070216  9681 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:22.063650  9459 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:22.071723  9459 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:22.072077  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:22.073390  5948 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 12: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:22.078513  9634 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:22.082645  9459 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:22.059376  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 965) took 2295 ms. Trace:
W20260709 12:26:22.019357  9694 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.137s	user 0.002s	sys 0.001s
I20260709 12:26:22.093482  9595 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.940073  9566 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:21.973340  9587 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:21.971632  9655 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:22.104348  6768 rpcz_store.cc:276] 0709 12:26:19.764228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.995512 (+231284us) service_pool.cc:224] Handling call
0709 12:26:19.996063 (+   551us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.019852 (+ 23789us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:21.549926 (+1530074us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b1800019000. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb10c0a 00007f39d9250219 00007f39cfb98a40 00007f39cfb934e4 00007f39cfb93455 00007f39d667b3c7 00007f39d2c8d9a7 00007f39d2c9d0bf 00007f39d2c9d038
0709 12:26:21.650377 (+100451us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.818955 (+168578us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.819661 (+   706us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.819862 (+   201us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:21.819897 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.819968 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.038591 (+218623us) raft_consensus.cc:1685] finished
0709 12:26:22.059013 (+ 20422us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.059277 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68396,"prepare.queue_time_us":1785826,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":22645,"raft.run_cpu_time_us":408,"raft.run_wall_time_us":14725,"spinlock_wait_cycles":17098112,"thread_start_us":250327,"threads_started":2,"wal-append.queue_time_us":223690}
I20260709 12:26:21.969909  9506 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:22.115594  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 123758us
I20260709 12:26:22.112149  9372 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:22.122433  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:22.123157  9571 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:22.123500  9567 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:22.124065  9567 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:22.125285  9571 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:22.103902  9594 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:21.973127  9685 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:22.126804  9372 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:22.140343  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:22.115633  9526 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:22.125084  9567 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:22.113135  9595 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:22.236567  5943 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:22.130511  9479 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:26:22.260779  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.139140  9687 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.218478  9372 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.139550  9594 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:22.318375  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 768) took 2566 ms. Trace:
I20260709 12:26:22.320024  9272 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:22.323426  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:22.325094  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1039) took 1292 ms. Trace:
I20260709 12:26:22.331681  5942 rpcz_store.cc:276] 0709 12:26:21.032916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.052417 (+ 19501us) service_pool.cc:224] Handling call
0709 12:26:22.324735 (+1272318us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54001bdfd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:22.324992 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271944960}
I20260709 12:26:22.331089  6021 rpcz_store.cc:276] 0709 12:26:19.751913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.811686 (+ 59773us) service_pool.cc:224] Handling call
0709 12:26:19.840803 (+ 29117us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.582651 (+741848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.318039 (+1735388us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b5400464954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:22.318264 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1703913216}
I20260709 12:26:22.326483  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:22.329020  9315 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:22.337333  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:22.307382  9479 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:22.346679  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 975) took 1377 ms. Trace:
W20260709 12:26:22.348903  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.348937  6762 rpcz_store.cc:276] 0709 12:26:20.969427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.970751 (+  1324us) service_pool.cc:224] Handling call
0709 12:26:20.975870 (+  5119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.050652 (+ 74782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.346269 (+1295617us) spinlock_profiling.cc:243] Waited 73.4 ms on lock 0x7b5400781ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:22.346558 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168755968}
I20260709 12:26:22.312894  9372 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:22.344281  9527 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:22.361358  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.313222  9687 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:22.374034  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.367731  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:22.384366  5865 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:22.368052  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.384985  5942 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 12:26:22.380828  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:22.387502  9708 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:26:22.350003  9634 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:26:22.390945  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:22.393930  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:22.388105  9479 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:22.396150  5948 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:22.398226  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:22.399040  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:22.400310  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:22.400998  5948 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:22.359622  9668 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:22.395323  9655 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:22.399058  5870 raft_consensus.cc:2410] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:22.407045  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:22.407891  5942 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:22.400205  9571 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:22.416342  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.392490  9470 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:26:22.419950  5946 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:22.426283  9594 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:22.433806  9576 raft_consensus.cc:3055] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:22.419163  9571 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.434371  9576 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:22.434584  9571 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:22.435277  9576 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:22.425392  9687 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:22.438699  6024 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:26:22.439500  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 997) took 1843 ms. Trace:
I20260709 12:26:22.439826  6024 rpcz_store.cc:276] 0709 12:26:20.595951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.606027 (+ 10076us) service_pool.cc:224] Handling call
0709 12:26:22.438050 (+1832023us) spinlock_profiling.cc:243] Waited 9.56 ms on lock 0x7b5800480098. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:26:22.438295 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.439199 (+   904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.439423 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21987200}
I20260709 12:26:21.887346  6785 rpcz_store.cc:276] 0709 12:26:20.017204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.017399 (+   195us) service_pool.cc:224] Handling call
0709 12:26:20.017942 (+   543us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.018825 (+   883us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:20.058895 (+ 40070us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.462822 (+403927us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.483702 (+ 20880us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.483896 (+   194us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:21.754370 (+1270474us) spinlock_profiling.cc:243] Waited 4.78 ms on lock 0x7b1800440020. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb10c0a 00007f39d9250219 00007f39cfb98a40 00007f39cfb934e4 00007f39cfb93455 00007f39d667b3c7 00007f39d2c8d9a7 00007f39d2c9d0bf 00007f39d2c9d038
0709 12:26:21.754641 (+   271us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.881272 (+126631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.881396 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.881598 (+   202us) raft_consensus.cc:1685] finished
0709 12:26:21.881618 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.882369 (+   751us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1299965,"prepare.queue_time_us":2858348,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":78,"raft.queue_time_us":8836,"raft.run_cpu_time_us":552,"raft.run_wall_time_us":53921,"spinlock_wait_cycles":10994048,"thread_start_us":394216,"threads_started":2,"wal-append.queue_time_us":457658,"wal-append.run_cpu_time_us":2739,"wal-append.run_wall_time_us":1004212}
I20260709 12:26:22.445091  9587 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:22.395030  9667 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:22.451637  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:22.456079  9592 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:22.456284  9445 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:22.448150  5948 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:22.393113  9488 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:22.427227  9709 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:26:22.426638  9526 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:22.461949  9488 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:22.452757  5913 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.781s (SENT) [suppressed 13 similar messages]
I20260709 12:26:22.464745  9527 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:22.473819  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.474725  9445 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.475883  9607 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:22.476476  9607 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:22.479085  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:22.475607  9445 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:22.480679  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.487419  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.488108  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:22.489154  5941 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:22.484822  9675 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:22.493185  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:22.492372  5874 raft_consensus.cc:2393] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:22.498564  5913 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:22.420567  5865 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
W20260709 12:26:22.503248  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:22.504225  5915 connection.cc:256] RPC call timeout handler was delayed by -1.7073414149999735s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
W20260709 12:26:22.504832  5915 connection.cc:256] RPC call timeout handler was delayed by -1.4840407980000236s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19210000000000002s for a response
I20260709 12:26:22.517913  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.530368  5987 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:22.507210  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:22.530256  5915 connection.cc:256] RPC call timeout handler was delayed by -1.4705407980000018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
W20260709 12:26:22.532925  5915 connection.cc:256] RPC call timeout handler was delayed by -1.2033414150000681s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20640000000000003s for a response
I20260709 12:26:22.534003  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:22.535215  5870 raft_consensus.cc:2393] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:22.537191  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.538563  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:22.539427  5865 raft_consensus.cc:2410] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:22.540246  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:22.495280  9675 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:22.507216  9506 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:22.493023  9576 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:22.420158  9668 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:22.492587  9592 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:22.491168  9667 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:22.553452  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:22.552861  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.557365  5865 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:22.552084  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 150958us
I20260709 12:26:22.477432  9607 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:22.563879  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:22.564780  5948 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:22.568794  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:22.573252  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:22.574337  5948 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:26:22.584013  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:22.585214  5946 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
W20260709 12:26:22.598145  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 214366us
I20260709 12:26:22.543956  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.548131  9445 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:22.549922  9272 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:22.614787  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.615478  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.616568  5874 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:26:22.591995  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:22.607743  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:22.554032  9655 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:22.618495  6022 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 10: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:22.619390  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:26:22.619606  5867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 987) took 1956 ms (client timeout 1839 ms). Trace:
W20260709 12:26:22.623692  5867 rpcz_store.cc:269] 0709 12:26:20.663254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.673381 (+ 10127us) service_pool.cc:224] Handling call
0709 12:26:22.619342 (+1945961us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54003890d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:22.619528 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361515776}
I20260709 12:26:22.625110  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.625828  5867 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:22.618675  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:22.630288  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:22.632732  6024 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:22.626933  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.634953  5867 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:26:22.636246  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.628924  6609 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:22.630721  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:22.641937  6021 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 11: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:22.643286  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.643803  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:22.644783  5867 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:22.646760  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:22.640486  5948 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:22.647818  5865 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 7.
W20260709 12:26:22.645907  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 115614us
I20260709 12:26:22.554929  9571 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:22.651662  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:22.677546  5987 leader_election.cc:336] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.682s (SENT)
I20260709 12:26:22.652065  5870 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:22.657402  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:22.678634  6022 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 12: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:22.661679  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:22.665676  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:22.679935  6020 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
W20260709 12:26:22.667141  9711 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 12:26:22.684597  5870 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:22.685952  6024 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:22.667354  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:22.667449  9667 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:22.667976  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:22.669775  9668 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:22.612572  9252 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:22.618942  9445 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:22.678071  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:22.620622  9655 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.691723  5870 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:22.704535  5870 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:22.624643  9470 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:22.693747  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 159839us
I20260709 12:26:22.618906  9524 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 12:26:22.707686  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:22.632090  9372 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:22.612931  9488 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:26:22.684002  9272 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:22.713837  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:22.718537  5915 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:22.722448  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:22.723618  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:22.726583  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:22.707557  9655 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:22.698887  9714 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:22.737073  5865 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 9 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:22.706758  9470 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:22.627499  9675 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:22.739287  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 61758us
I20260709 12:26:22.743959  9714 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.715911  9445 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:22.696635  9717 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:26:22.757877  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:22.715359  9272 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.759596  9272 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:22.706615  9252 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:22.734426  9459 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:22.758114  9445 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:22.758895  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 781) took 1704 ms. Trace:
I20260709 12:26:22.767311  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:22.774179  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:22.731922  9668 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:26:22.756374  9712 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:26:22.782542  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:22.783565  6778 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:22.784821  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:22.764573  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:22.787274  6015 raft_consensus.cc:2376] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13. Re-sending same reply.
I20260709 12:26:22.765228  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:22.743695  9470 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:22.766474  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:22.761235  9585 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:22.790979  6785 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 7: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:22.770509  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:22.771473  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.764777  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:22.772673  6023 rpcz_store.cc:276] 0709 12:26:21.054101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.432761 (+378660us) service_pool.cc:224] Handling call
0709 12:26:21.437569 (+  4808us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.437986 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.438016 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.465729 (+ 27713us) log.cc:844] Serialized 26 byte log entry
0709 12:26:22.444476 (+978747us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b1800030be0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:26:22.683066 (+238590us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:22.683128 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.683221 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.758504 (+ 75283us) raft_consensus.cc:1685] finished
0709 12:26:22.758532 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.758815 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48570,"raft.queue_time_us":29241,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":15925,"spinlock_wait_cycles":11642752,"thread_start_us":238319,"threads_started":1,"wal-append.queue_time_us":1202239}
I20260709 12:26:22.793759  6021 raft_consensus.cc:2410] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:26:22.733295  9506 raft_consensus.cc:697] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:22.764401  5872 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:22.789736  6768 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 14: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:22.800235  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:22.795676  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:22.796075  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1364) took 1895 ms. Trace:
I20260709 12:26:22.700281  9592 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:22.798389  6019 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:26:22.808018  9687 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.808755  9687 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:22.811844  9725 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:22.812856  9479 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:22.813063  9725 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:22.813416  9479 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:22.814689  9479 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:22.816067  9479 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:22.818149  9726 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:22.803557  9445 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:22.820878  9687 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:22.804351  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:22.846709  6766 raft_consensus.cc:2393] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:22.788826  9668 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:22.822028  9218 raft_consensus.cc:3055] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:22.821457  9445 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:22.824927  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1034) took 1800 ms. Trace:
I20260709 12:26:22.825896  9571 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:22.828460  9720 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:26:22.829478  9252 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:22.833745  9727 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:22.837292  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:22.840968  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:22.856904  5947 rpcz_store.cc:276] 0709 12:26:21.024698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.025781 (+  1083us) service_pool.cc:224] Handling call
0709 12:26:21.040367 (+ 14586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.040623 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:21.040995 (+   372us) log.cc:844] Serialized 14 byte log entry
0709 12:26:22.824345 (+1783350us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b18003d5940. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39d282b4bc 00007f39cfb7948b 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287
0709 12:26:22.824473 (+   128us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.824533 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:22.824587 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.824693 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.824855 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4136,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":42,"raft.queue_time_us":31875,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":7354,"spinlock_wait_cycles":11122432,"thread_start_us":109920,"threads_started":2,"wal-append.queue_time_us":140723,"wal-append.run_cpu_time_us":2559,"wal-append.run_wall_time_us":1288697}
I20260709 12:26:22.801874  5944 rpcz_store.cc:276] 0709 12:26:20.901160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.907743 (+  6583us) service_pool.cc:224] Handling call
0709 12:26:20.908222 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.908692 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.908731 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.909341 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 12:26:22.317336 (+1407995us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b1800327d60. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:26:22.494219 (+176883us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:22.494260 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.494331 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.785342 (+291011us) raft_consensus.cc:1685] finished
0709 12:26:22.785365 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.795994 (+ 10629us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":5474,"spinlock_wait_cycles":56076032,"thread_start_us":176595,"threads_started":1,"wal-append.queue_time_us":1710215}
I20260709 12:26:22.749347  9714 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:22.772413  9459 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:22.833065  9598 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:22.851045  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:22.852866  6778 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:22.872231  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:22.853745  5867 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:22.855365  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:22.873708  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:22.874506  6019 raft_consensus.cc:2410] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:22.875355  5874 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 12:26:22.861562  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:22.857580  5942 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:26:22.863325  6775 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:22.862526  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:22.864760  6768 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:22.867053  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:22.877923  6021 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:26:22.821463  9520 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:22.878154  6756 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:22.874464  6778 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:22.866048  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:22.879460  5872 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:26:22.882836  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:22.882762  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:22.884093  6778 raft_consensus.cc:2368] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13.
I20260709 12:26:22.886708  6757 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 11: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:22.886904  6773 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:26:22.856812  9334 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:22.890730  6766 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:22.859359  9561 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.858090  9058 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:22.911482  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:22.888229  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:22.890847  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:22.851042  9218 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:22.875628  9459 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:22.909127  6021 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:26:22.890805  6785 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:22.909884  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:22.934406  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.943311  9734 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:22.870908  9687 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.917085  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:22.964823  6768 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:22.907672  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:22.945953  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:22.946791  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:22.949954  9735 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:22.963632  6019 raft_consensus.cc:2393] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 10: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:22.967222  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:22.967415  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:22.956930  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 209925us
I20260709 12:26:22.907608  9598 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:22.951046  9719 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:26:22.944324  9252 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:22.968842  6015 raft_consensus.cc:2393] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 11: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:22.969379  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:22.967947  9715 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:22.958173  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 231637us
I20260709 12:26:22.968124  6775 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:22.971527  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:22.952486  9729 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:22.964068  9734 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:22.980896  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.987069  6768 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:22.945043  9561 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:22.992758  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:22.951397  9058 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:22.970590  9687 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:22.975514  9731 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:23.006585  9058 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:22.990207  9218 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:23.008384  9218 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:22.991775  9729 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:23.012720  9725 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:22.986873  9733 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:22.951742  9506 consensus_queue.cc:237] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:22.988507  9734 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:22.982558  9252 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:23.031499  9687 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:23.017524  9731 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:23.033708  9739 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:26:23.046746  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.048300  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:22.976722  9598 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:23.049252  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 58354.1us
I20260709 12:26:23.074971  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:23.077258  9745 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:26:23.088493  5915 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:23.039348  9459 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:23.097520  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 130193us
I20260709 12:26:23.097903  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:23.097394  9733 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:23.099598  9733 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:23.100077  9733 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:23.069495  9742 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:23.099054  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:23.082545  9614 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:23.102041  5915 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:23.090422  9743 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:23.105535  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:23.108769  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:23.110204  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:23.096732  9668 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:23.112608  9729 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:23.114580  9731 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:23.114384  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:23.117558  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:23.118734  6775 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:23.119108  9252 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:23.120579  9561 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:23.122458  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:23.123879  6020 raft_consensus.cc:2410] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:26:23.123994  5869 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:26:23.124571  5869 pending_rounds.cc:85] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 4
I20260709 12:26:23.124894  5869 pending_rounds.cc:107] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:26:23.125128  5869 raft_consensus.cc:2889] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:23.125202  6775 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 12:26:23.130065  6768 raft_consensus.cc:2410] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:26:23.105185  9742 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:23.131937  9742 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:23.135895  9470 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:23.137490  6778 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:23.141829  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.177060  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 184407us
I20260709 12:26:23.122346  9727 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:23.102890  9725 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:26:23.180665  5913 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:23.108511  9445 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:26:23.185400  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 80920us
I20260709 12:26:23.191576  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:23.205125  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.209969  9751 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:23.210570  9751 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.210793  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 639) took 2403 ms. Trace:
I20260709 12:26:23.155012  9747 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.211675  9747 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:23.210940  9751 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:23.178673  9252 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:23.178848  9735 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:23.122267  9272 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:23.157214  9743 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:23.211416  5871 rpcz_store.cc:276] 0709 12:26:20.807871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.808011 (+   140us) service_pool.cc:224] Handling call
0709 12:26:22.620870 (+1812859us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.621306 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:22.621333 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:22.624904 (+  3571us) log.cc:844] Serialized 26 byte log entry
0709 12:26:22.717198 (+ 92294us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:22.717240 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.717321 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.824817 (+107496us) raft_consensus.cc:1685] finished
0709 12:26:22.824841 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.210450 (+385609us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54003890d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:23.210716 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":16567,"spinlock_wait_cycles":4167998848,"thread_start_us":91945,"threads_started":1,"wal-append.queue_time_us":149605}
W20260709 12:26:23.221277  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 79871.6us
I20260709 12:26:23.223809  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:23.187110  9727 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:23.227453  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 46906.2us
I20260709 12:26:23.225839  5915 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:23.233548  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:23.236581  5913 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:23.241283  6773 raft_consensus.cc:2368] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 14. Current term is 15.
I20260709 12:26:23.226933  9732 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:23.237967  9727 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:23.245739  9470 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:23.250365  9747 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:23.252219  9576 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.649s	user 0.009s	sys 0.005s
I20260709 12:26:23.252599  9754 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:23.253052  9754 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.253407  9754 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:23.250566  9579 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:23.229944  9445 consensus_queue.cc:1059] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:26:23.270272  6024 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:23.270952  9445 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:23.255836  9598 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:23.227259  9655 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:23.258014  9579 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:23.277367  9579 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:23.279157  9334 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:23.279824  9334 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:23.283720  9334 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:23.273324  9742 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:23.264430  9758 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:23.285547  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:23.287261  9758 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:23.288090  9758 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:23.270179  9459 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:23.289448  9459 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:23.290366  9459 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:23.264720  9732 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:23.301340  9459 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:23.301822  9732 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:23.302600  9761 raft_consensus.cc:3055] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:23.303236  9761 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:23.286072  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:23.256860  9715 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:26:23.286767  9742 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:23.304425  9761 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:23.311712  9761 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:23.269680  9762 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:23.316462  9272 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:23.336026  5866 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1001) took 1577 ms (client timeout 1763 ms). Trace:
I20260709 12:26:23.334879  6609 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:23.346817  5866 rpcz_store.cc:269] 0709 12:26:21.758937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.760350 (+  1413us) service_pool.cc:224] Handling call
0709 12:26:23.335800 (+1575450us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54007f3354. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:23.335951 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370466048}
W20260709 12:26:23.352895  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 127218us
I20260709 12:26:23.354738  5915 leader_election.cc:400] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13.
I20260709 12:26:23.355304  5915 leader_election.cc:403] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:23.329489  9691 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:23.342000  9762 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:23.314247  9732 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:23.369083  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:23.356437  9526 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:23.354820  9459 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:23.377120  9459 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:23.378363  9497 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:23.378772  9459 consensus_queue.cc:1059] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:23.380020  9459 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:23.381028  9459 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:23.381790  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:23.381901  9459 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:23.382548  6773 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:23.383111  6022 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:26:23.383204  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:23.384294  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:23.384711  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:23.385926  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:23.386123  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:23.386642  6022 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:23.368757  9762 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:23.386905  9459 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:23.389369  5873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 996) took 2461 ms (client timeout 2204 ms). Trace:
I20260709 12:26:23.386161  9567 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:23.389999  5873 rpcz_store.cc:269] 0709 12:26:20.928120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.946222 (+ 18102us) service_pool.cc:224] Handling call
0709 12:26:23.389080 (+2442858us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54007f30d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:23.389250 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330990976}
I20260709 12:26:23.390417  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:23.394585  9526 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:23.395514  5840 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:23.395709  9526 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:23.396551  9680 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:23.397275  9680 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:23.366302  9742 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13.
I20260709 12:26:23.398379  9742 raft_consensus.cc:3055] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:23.398785  9742 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:23.399091  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:23.399999  9566 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:23.390327  9459 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:23.400962  6016 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:23.401077  9459 raft_consensus.cc:3055] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:23.401620  9459 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:23.402455  9459 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:23.404211  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:23.399590  9742 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:23.408100  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 986) took 2401 ms. Trace:
I20260709 12:26:23.413407  5945 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:23.413872  6793 rpcz_store.cc:276] 0709 12:26:21.006526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.086619 (+ 80093us) service_pool.cc:224] Handling call
0709 12:26:21.086982 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.689745 (+602763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.391169 (+1701424us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b54002c24d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:23.408023 (+ 16854us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1322139008}
I20260709 12:26:23.412915  6611 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:23.415488  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:23.389482  5868 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:23.415949  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:23.416517  6611 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:23.386128  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:23.420889  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:23.410704  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:23.411327  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:23.418674  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 182139us
I20260709 12:26:23.422583  5948 raft_consensus.cc:2393] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:23.423039  6019 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:23.429077  5840 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:23.429950  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:23.430714  6021 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:23.434101  5948 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:23.436894  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:23.438125  5913 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:23.448949  6023 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:23.448186  6793 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:23.452497  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:23.449539  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:23.454483  5948 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:23.453253  6023 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:23.453469  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:23.456753  5949 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:23.403650  9459 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:23.444736  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:23.469939  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:23.465312  5948 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:23.471828  5840 leader_election.cc:400] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 14. Current term is 15.
I20260709 12:26:23.467597  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:23.472185  5840 leader_election.cc:403] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:23.448066  6019 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:23.472961  5945 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 11. Current term is 12. Ops: []
W20260709 12:26:23.476799  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.477675  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:23.443622  9566 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:23.476660  6019 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:23.443713  9272 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:23.443168  9497 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:23.449663  9743 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:23.482542  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:23.476624  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:23.454020  9334 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:23.362076  9734 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:23.486240  6023 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:23.468653  9506 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:23.411921  9733 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:23.451423  9680 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:26:23.522213  6793 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
W20260709 12:26:23.531586  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.559423  6020 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 9: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:23.559999  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:23.495877  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:23.564473  6793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:23.565653  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:23.566183  6793 raft_consensus.cc:606] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:23.474928  9561 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:26:23.569708  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 73975.1us
I20260709 12:26:23.480825  9667 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:23.480646  9668 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:23.596390  6773 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:23.595789  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:23.571676  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:23.486428  9718 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:23.485729  9742 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:23.578610  5949 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:23.481299  9723 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.603772  6611 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:23.606375  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.544536  9714 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.658s	user 0.009s	sys 0.005s
W20260709 12:26:23.623416  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.549685  9479 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.553841  9506 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:23.597680  9561 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:23.484246  9731 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.126s
I20260709 12:26:23.598097  9497 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:23.560557  9488 raft_consensus.cc:3055] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:23.488458  9606 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.138s
I20260709 12:26:23.484854  9729 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.126s
I20260709 12:26:23.601042  9724 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.486692  9764 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.145s
I20260709 12:26:23.608960  9272 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:23.609961  9334 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.608350  9732 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:23.611094  9668 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:23.637609  9561 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:23.678462  9668 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:23.679328  9671 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:23.653240  9766 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:23.681502  9766 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.558414  9743 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:23.681689  9762 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:23.682003  9727 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:23.684345  9762 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:23.649149  9724 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:23.657759  9765 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.664835  9655 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:23.630375  9723 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:23.665653  9497 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:23.666266  9334 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:23.671900  9732 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:23.672432  9595 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:23.674161  9764 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:23.674744  9733 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:23.676462  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:23.676950  9734 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:23.548914  9775 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 12:26:23.628249  9714 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:23.641819  9479 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:23.641064  9488 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:23.713335  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:23.643178  9751 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:23.755666  6018 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:23.855302  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:23.856838  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:23.682050  9766 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:23.827785  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 351210us
I20260709 12:26:23.837037  6768 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:23.839022  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.866444  6019 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:23.735261  9497 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:23.883383  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:23.884459  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 280767us
I20260709 12:26:23.885283  6611 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:23.886343  6611 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:23.886529  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:23.887310  5871 raft_consensus.cc:2393] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:23.891206  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:23.711441  9765 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:23.903053  6611 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:23.909139  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:23.930025  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:23.931042  5873 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:23.934312  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.850278  9733 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:23.946605  5865 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:26:23.947147  5865 pending_rounds.cc:85] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 5
I20260709 12:26:23.947425  5865 pending_rounds.cc:107] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:26:23.947669  5865 raft_consensus.cc:2889] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:23.843586  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 412042us
I20260709 12:26:23.893467  9734 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.955614  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:23.948686  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:23.957813  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:23.816515  9667 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 14. Current term is 15.
I20260709 12:26:23.968044  5867 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:23.968045  5948 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:23.973189  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:23.973956  5867 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:23.977109  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:23.995513  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:24.005712  9592 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.156s	user 0.011s	sys 0.019s
I20260709 12:26:23.911327  9731 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:23.893553  9272 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:23.962239  5913 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:23.916656  9653 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 12:26:24.018585  5867 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:23.959257  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:24.019565  5873 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:23.939544  9727 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:24.023679  5871 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:23.894045  9762 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:26:23.941167  9691 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.938309  9776 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.962850  9779 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.030108  9334 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:24.031601  9731 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:24.038198  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 76049.2us
I20260709 12:26:24.014568  6768 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:24.082593  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:24.083756  5948 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 13: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:23.943392  9680 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:23.993310  9445 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.722s	user 0.012s	sys 0.003s
I20260709 12:26:23.980288  9734 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:24.049432  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 90291.6us
W20260709 12:26:24.032697  9592 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.179s	user 0.011s	sys 0.019s
I20260709 12:26:24.009378  9769 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:24.048405  9782 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:24.078640  9779 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:24.104852  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:24.105758  5940 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:23.909266  9729 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 12:26:24.106523  9779 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:24.107012  9729 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:24.088124  9334 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:24.107909  9729 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:24.108377  9334 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:23.969292  9733 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:24.112089  9733 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:24.112916  9733 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:24.040212  9667 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:24.089596  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:24.091486  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:23.963256  9718 raft_consensus.cc:697] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:24.097002  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:24.095079  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:24.081836  9768 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:24.115061  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 1006) took 1456 ms. Trace:
I20260709 12:26:24.116277  6021 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:24.118520  9479 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:24.125238  9758 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.836s	user 0.011s	sys 0.005s
W20260709 12:26:24.156172  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.127218  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1277) took 1768 ms. Trace:
I20260709 12:26:24.109326  9729 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:24.090616  9731 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:24.072688  9780 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:24.084275  5873 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:24.071218  9691 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:24.071451  9776 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:24.151713  9777 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:26:23.964883  9488 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:24.164973  5941 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 8.
I20260709 12:26:24.169690  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:24.134724  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1297) took 1706 ms. Trace:
I20260709 12:26:24.266188  5874 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 11: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:24.173141  5948 raft_consensus.cc:2410] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:26:24.180440  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1004) took 2046 ms. Trace:
I20260709 12:26:24.267315  5941 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:24.267526  5868 rpcz_store.cc:276] 0709 12:26:22.133929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.134174 (+   245us) service_pool.cc:224] Handling call
0709 12:26:23.389250 (+1255076us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.428540 (+ 39290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.180116 (+751576us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54007f30d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:24.180343 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2884985088}
I20260709 12:26:24.187387  5915 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:24.269594  5941 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:24.270015  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 82781.1us
I20260709 12:26:24.271417  5915 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:24.272047  6773 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:24.145962  6015 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:24.187644  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:24.168718  5867 raft_consensus.cc:2410] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
W20260709 12:26:24.208781  5945 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.061s	sys 0.095s
W20260709 12:26:24.277673  5945 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.061s	sys 0.095s
I20260709 12:26:24.087422  9606 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:24.171521  6016 rpcz_store.cc:276] 0709 12:26:22.658823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.661217 (+  2394us) service_pool.cc:224] Handling call
0709 12:26:24.114825 (+1453608us) spinlock_profiling.cc:243] Waited 721 ms on lock 0x7b5400160b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:24.114975 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1658592512}
I20260709 12:26:24.172313  9768 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:24.201280  5869 rpcz_store.cc:276] 0709 12:26:22.358906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.359676 (+   770us) service_pool.cc:224] Handling call
0709 12:26:22.365257 (+  5581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.125708 (+760451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.126884 (+1001176us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b5400653e54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:24.127136 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1713883520}
W20260709 12:26:24.206368  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 268829us
W20260709 12:26:24.290700  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.216549  5943 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:26:24.293011  9527 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.185770  9723 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:24.295203  6775 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:26:24.296094  6611 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:24.220638  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:24.100802  9445 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:24.058557  9598 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:26:24.129563  9742 consensus_meta.cc:287] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.506s	user 0.004s	sys 0.007s
I20260709 12:26:24.243517  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:24.310163  6768 raft_consensus.cc:2410] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:26:24.311272  6757 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:24.263401  9372 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.164559  9667 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:24.172426  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 78398.3us
I20260709 12:26:24.176607  9727 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:24.151943  9276 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:24.211392  9479 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:24.227624  9561 consensus_meta.cc:287] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.549s	user 0.008s	sys 0.004s
I20260709 12:26:24.287576  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:24.069449  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 383819us
I20260709 12:26:24.296495  6793 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:24.297035  6785 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:24.298270  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:24.100301  9680 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:24.059854  9497 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:24.225682  9579 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:24.320376  5944 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:24.198572  9769 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:24.288911  9775 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:24.289606  9729 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:24.283061  9724 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:24.210501  9758 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:24.310287  9598 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:24.347049  6019 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 14: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:24.387035  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:24.389796  6024 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:26:24.387403  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:24.387691  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.341504  9459 consensus_meta.cc:287] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.868s	user 0.011s	sys 0.003s
I20260709 12:26:24.330799  9727 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:24.392329  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:24.385766  9792 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:26:24.349071  9479 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:24.399326  6773 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:24.400455  9479 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:24.401598  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:24.319113  9334 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:24.167891  9718 consensus_queue.cc:237] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:24.345111  9276 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:24.338148  9788 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:24.184433  9506 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:24.293759  9527 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:24.347142  9754 consensus_meta.cc:287] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.033s	user 0.014s	sys 0.000s
I20260709 12:26:24.326440  9768 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:24.400733  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:24.431692  6785 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:24.331426  9372 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:24.395234  9598 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:24.404800  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:24.401270  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:24.346591  9714 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:24.407323  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:24.263177  9780 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:24.448887  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:24.450534  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:24.374090  9561 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:24.384734  9743 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.002s	sys 0.026s
W20260709 12:26:24.386363  9786 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:26:24.392827  9566 consensus_meta.cc:287] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.906s	user 0.009s	sys 0.005s
I20260709 12:26:24.470175  9566 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:24.341310  9488 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:24.394500  9775 raft_consensus.cc:697] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:24.412195  9691 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:24.199188  9782 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:24.460548  9743 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.002s	sys 0.027s
I20260709 12:26:24.490345  9782 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:24.499347  9769 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:24.459380  9791 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:24.435154  9754 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:24.423359  9765 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:24.424850  9766 consensus_meta.cc:287] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.003s
I20260709 12:26:24.430675  9276 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:24.508996  9734 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:24.520555  9751 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:24.443836  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:24.529317  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 85602.6us
I20260709 12:26:24.403034  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:24.445125  9527 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:24.538128  9667 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:24.541234  9607 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:24.543403  9561 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:24.544772  9656 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:24.545509  9656 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:24.445540  9776 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:24.452811  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:24.454632  9780 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:24.401134  9479 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:24.576016  9479 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:24.578444  9479 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:24.456563  9768 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:24.466732  9587 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:24.581470  9587 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:24.582515  9587 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:24.583796  9587 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:24.467952  9732 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:24.469175  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 830) took 1584 ms. Trace:
I20260709 12:26:24.585035  6778 rpcz_store.cc:276] 0709 12:26:22.884997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.885231 (+   234us) service_pool.cc:224] Handling call
0709 12:26:22.885570 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.138378 (+252808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.468825 (+1330447us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b540081b5d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:24.469092 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":578627456}
I20260709 12:26:24.436650  9685 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:24.441895  9506 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:24.474638  9775 consensus_queue.cc:237] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:24.420186  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:24.590983  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:24.531430  6793 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:24.525756  9766 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:24.593549  6793 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:24.549126  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:24.558071  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:24.560262  6766 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:24.562638  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:24.565869  9585 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.564236  9791 raft_consensus.cc:697] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:24.595379  9799 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.571125  9758 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:24.568471  9675 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:24.572831  9680 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:24.574237  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:24.481078  9727 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:24.428665  9788 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:24.587792  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:24.605472  6778 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:24.596138  6785 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:24.596802  6773 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:26:24.597990  6756 raft_consensus.cc:2368] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12.
W20260709 12:26:24.603389  6766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 834) took 1630 ms (client timeout 2131 ms). Trace:
I20260709 12:26:24.601583  9585 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:24.611987  6766 rpcz_store.cc:269] 0709 12:26:22.970978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.971213 (+   235us) service_pool.cc:224] Handling call
0709 12:26:24.600974 (+1629761us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54002a12d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:24.601118 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3634520320}
I20260709 12:26:24.614164  6768 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:24.600203  9791 consensus_queue.cc:237] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:24.607298  9788 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:24.599691  9799 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:24.546727  9656 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:24.635551  9731 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:24.642571  9731 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:24.645448  9585 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:24.650359  9766 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:24.626488  9754 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:24.643421  9372 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:24.640960  9727 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:24.692703  9799 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:24.751967  9736 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.752904  9736 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:24.756847  9736 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:24.741380  9598 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:24.736721  9788 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:24.776780  9788 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:24.814119  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:24.266022  5870 rpcz_store.cc:276] 0709 12:26:22.428381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.710226 (+281845us) service_pool.cc:224] Handling call
0709 12:26:23.338730 (+628504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.338984 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.339024 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.339092 (+    68us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:23.339139 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.339206 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.134446 (+795240us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b54007f3350. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:24.134636 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1428276608}
I20260709 12:26:24.170423  5838 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:24.824580  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 653933us
I20260709 12:26:24.257970  5940 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:24.880332  5949 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:24.880941  6023 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:24.886915  5946 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:26:24.887866  6775 raft_consensus.cc:2410] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:26:24.889021  9655 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.890147  9655 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:24.898296  9723 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:24.887025  6757 raft_consensus.cc:2410] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:24.948169  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:24.956759  9655 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:24.962815  9723 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:24.974956  9723 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:24.981117  9729 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.982348  9729 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:24.995615  6017 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:25.002691  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1010) took 1475 ms. Trace:
I20260709 12:26:25.003449  6023 rpcz_store.cc:276] 0709 12:26:23.527468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.870227 (+342759us) service_pool.cc:224] Handling call
0709 12:26:23.870553 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.881894 (+1011341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.002241 (+120347us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b54001072d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:25.002569 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2279993216}
I20260709 12:26:25.019142  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 666) took 1037 ms. Trace:
I20260709 12:26:25.019843  5865 rpcz_store.cc:276] 0709 12:26:23.981431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.022356 (+ 40925us) service_pool.cc:224] Handling call
0709 12:26:24.869098 (+846742us) spinlock_profiling.cc:243] Waited 6.11 ms on lock 0x7b58002e2f98. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:26:24.961956 (+ 92858us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.018271 (+ 56315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.018318 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:25.018407 (+    89us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:25.018494 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.018588 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.018814 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14061312}
W20260709 12:26:25.049815  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 106457us
I20260709 12:26:25.066573  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1063) took 1109 ms. Trace:
I20260709 12:26:25.073752  6018 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:25.070520  5949 rpcz_store.cc:276] 0709 12:26:23.953788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.953962 (+   174us) service_pool.cc:224] Handling call
0709 12:26:23.954351 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.881532 (+927181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.001380 (+119848us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b54001bdfd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:25.062825 (+ 61445us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2073978240}
I20260709 12:26:25.112775  9758 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.113942  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:25.114816  9598 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:25.119995  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 802) took 1648 ms. Trace:
I20260709 12:26:25.114817  9680 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.114786  6020 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:25.121539  9483 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.160404  9483 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:25.127624  6015 rpcz_store.cc:276] 0709 12:26:23.470936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.605824 (+134888us) service_pool.cc:224] Handling call
0709 12:26:24.145604 (+539780us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.274625 (+129021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.119336 (+844711us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b5400160650. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:25.119690 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1240319232}
I20260709 12:26:25.110435  9655 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:25.186406  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 53568.4us
I20260709 12:26:25.113791  9758 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:25.184585  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:25.182952  9729 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:25.245013  5840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.288s	user 0.026s	sys 0.096s
W20260709 12:26:25.265304  5872 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.195s	user 0.050s	sys 0.109s
W20260709 12:26:25.266033  5872 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.229s	user 0.051s	sys 0.109s
I20260709 12:26:25.242293  9598 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:25.271904  5840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.336s	user 0.027s	sys 0.096s
I20260709 12:26:25.240237  9655 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:25.278139  6762 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:25.239439  9680 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:25.297006  9519 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.229s	user 0.061s	sys 0.101s
W20260709 12:26:25.302278  9757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.977s	user 0.071s	sys 0.032s
W20260709 12:26:25.323669  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.947s	user 0.035s	sys 0.059s
W20260709 12:26:25.309495  9497 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.022s	sys 0.041s
W20260709 12:26:25.333438  9497 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.022s	sys 0.042s
W20260709 12:26:25.334619  9442 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.720s	user 0.023s	sys 0.096s
I20260709 12:26:25.324801  9598 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:25.342068  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.068s	user 0.040s	sys 0.060s
I20260709 12:26:25.339676  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1388) took 1018 ms. Trace:
I20260709 12:26:25.342001  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:25.319191  9738 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.171s	user 0.055s	sys 0.111s
W20260709 12:26:25.321655  9656 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.010s	sys 0.016s
W20260709 12:26:25.323625  9668 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.029s	user 0.036s	sys 0.064s
I20260709 12:26:25.338876  9598 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:25.347136  9519 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.373s	user 0.061s	sys 0.103s
W20260709 12:26:25.342904  9757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.096s	user 0.071s	sys 0.033s
W20260709 12:26:25.369509  6016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.929s	user 0.013s	sys 0.006s
W20260709 12:26:25.372035  5914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.967s	user 0.006s	sys 0.014s
W20260709 12:26:25.376744  5942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.970s	user 0.012s	sys 0.031s
I20260709 12:26:25.296375  9587 consensus_queue.cc:1059] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.150s
W20260709 12:26:25.373502  5912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.008s	sys 0.012s
I20260709 12:26:25.344823  5944 rpcz_store.cc:276] 0709 12:26:24.210535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.210693 (+   158us) service_pool.cc:224] Handling call
0709 12:26:24.211067 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.375333 (+164266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.189958 (+814625us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b5400673ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:25.229337 (+ 39379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":250855168}
W20260709 12:26:25.377094  9656 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.012s	sys 0.016s
W20260709 12:26:25.375479  9738 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.329s	user 0.055s	sys 0.113s
W20260709 12:26:25.402014  5914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.128s	user 0.007s	sys 0.015s
W20260709 12:26:25.372840  9442 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.759s	user 0.024s	sys 0.097s
W20260709 12:26:25.386808  5912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.079s	user 0.009s	sys 0.012s
W20260709 12:26:25.387096  9775 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.014s	sys 0.024s
I20260709 12:26:25.387486  6022 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
W20260709 12:26:25.390452  9808 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:26:25.381767  9497 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:25.382227  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 1.10804e+06us
I20260709 12:26:25.287678  9732 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.188s
I20260709 12:26:25.369773  9731 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:25.397562  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 45999.7us
W20260709 12:26:25.402089  5942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.099s	user 0.012s	sys 0.031s
W20260709 12:26:25.401918  6016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.089s	user 0.014s	sys 0.007s
I20260709 12:26:25.427935  9598 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:25.429323  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:25.404856  5869 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:25.429378  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 998) took 1030 ms. Trace:
W20260709 12:26:25.383544  9668 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.295s	user 0.036s	sys 0.073s
I20260709 12:26:25.435408  6775 rpcz_store.cc:276] 0709 12:26:24.398958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.400985 (+  2027us) service_pool.cc:224] Handling call
0709 12:26:25.428961 (+1027976us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b54001e00d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:25.429255 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":971392768}
W20260709 12:26:25.432056  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.442018  9598 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:25.397424  9680 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:25.410545  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:25.395643  9655 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:25.427948  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:25.407387  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 59931.2us
W20260709 12:26:25.438436  9775 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.014s	sys 0.024s
W20260709 12:26:25.387535  9718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.006s	sys 0.028s
I20260709 12:26:25.441010  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:25.452431  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:25.453188  9724 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:25.454152  9732 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:25.455420  9497 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:25.442138  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:25.411051  9519 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:26:25.464239  9718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.050s	user 0.007s	sys 0.031s
I20260709 12:26:25.486635  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:25.482424  9757 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:25.489094  9757 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:25.490576  9803 consensus_queue.cc:1059] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:26:25.501513  6018 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:25.505885  6017 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:25.507782  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:25.508664  6017 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
W20260709 12:26:25.510138  9791 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.026s	sys 0.059s
I20260709 12:26:25.511021  6023 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
W20260709 12:26:25.516533  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.517733  9724 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:25.518998  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1070) took 1428 ms. Trace:
I20260709 12:26:25.519483  5948 rpcz_store.cc:276] 0709 12:26:24.090845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.091109 (+   264us) service_pool.cc:224] Handling call
0709 12:26:25.518465 (+1427356us) spinlock_profiling.cc:243] Waited 5.73 ms on lock 0x7b5400467dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:25.518772 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13183744}
I20260709 12:26:25.519938  9758 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:25.520603  9724 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:25.521420  9732 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:25.517443  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:25.511149  9791 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.026s	sys 0.059s
I20260709 12:26:25.526170  6022 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:25.468611  9519 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:25.501783  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1386) took 1292 ms. Trace:
I20260709 12:26:25.536060  5942 rpcz_store.cc:276] 0709 12:26:24.209269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.209407 (+   138us) service_pool.cc:224] Handling call
0709 12:26:24.270911 (+ 61504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:24.271343 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:24.271374 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:24.302381 (+ 31007us) log.cc:844] Serialized 26 byte log entry
0709 12:26:25.454962 (+1152581us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:25.455083 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.455224 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:25.501218 (+ 45994us) raft_consensus.cc:1685] finished
0709 12:26:25.501261 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.501670 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6157,"raft.queue_time_us":4321,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":13567,"thread_start_us":1131733,"threads_started":1,"wal-append.queue_time_us":1195566}
I20260709 12:26:25.561928  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:25.480861  9479 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.032s	sys 0.049s
I20260709 12:26:25.576076  6020 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:25.577880  6018 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:25.523716  9655 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:25.523197  9731 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:25.586180  9811 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:25.587692  9731 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:25.589831  9586 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.590541  9586 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:25.593066  9809 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:25.598455  6020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 1018) took 1380 ms (client timeout 1795 ms). Trace:
W20260709 12:26:25.577265  9479 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.033s	sys 0.050s
I20260709 12:26:25.603458  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:25.606523  6793 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.973s	user 0.005s	sys 0.023s
I20260709 12:26:25.607875  6793 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:25.608927  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1002) took 1208 ms. Trace:
I20260709 12:26:25.609440  9809 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:25.612545  9809 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:25.569514  9758 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:25.619585  9758 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:25.568399  9668 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:25.601547  6020 rpcz_store.cc:269] 0709 12:26:24.203801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.113333 (+909532us) service_pool.cc:224] Handling call
0709 12:26:25.583868 (+470535us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:25.609352  6793 rpcz_store.cc:276] 0709 12:26:24.400146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.592435 (+192289us) service_pool.cc:224] Handling call
0709 12:26:24.593412 (+   977us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.608403 (+1014991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.608807 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:25.628222  6017 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
W20260709 12:26:25.622902  5915 connection.cc:256] RPC call timeout handler was delayed by -1.0930068309999115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2135s for a response
I20260709 12:26:25.729149  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:25.733479  5915 connection.cc:256] RPC call timeout handler was delayed by -1.0930068309999115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2135s for a response
I20260709 12:26:25.730334  6019 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
W20260709 12:26:25.740775  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 105476us
I20260709 12:26:25.638854  9809 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.731015  9775 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.761904  9731 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.764225  9497 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:25.766057  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:25.767141  5947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.320s	user 0.022s	sys 0.078s
I20260709 12:26:25.637729  9655 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:25.796412  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:25.797156  5947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.493s	user 0.023s	sys 0.078s
W20260709 12:26:25.809015  6024 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.354s	user 0.015s	sys 0.007s
I20260709 12:26:25.809558  5987 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:25.811627  5871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 661) took 1839 ms (client timeout 1754 ms). Trace:
I20260709 12:26:25.774960  9668 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:25.812140  5871 rpcz_store.cc:269] 0709 12:26:23.947635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.948316 (+   681us) service_pool.cc:224] Handling call
0709 12:26:25.739858 (+1791542us) spinlock_profiling.cc:243] Waited 55.6 ms on lock 0x7b54004a0454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:25.787205 (+ 47347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127887616}
I20260709 12:26:25.812886  9758 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:25.770550  9718 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.807847  9809 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:25.823442  9655 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:25.824465  9655 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:25.824966  9655 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.825142  9758 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:25.826292  9758 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:25.816362  6024 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:25.825376  9655 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:25.845021  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:25.846271  6015 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
W20260709 12:26:25.846849  9815 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.244s	user 0.001s	sys 0.001s
I20260709 12:26:25.845283  6611 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:25.848748  9315 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.004s	sys 0.148s
W20260709 12:26:25.849321  9315 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.005s	sys 0.148s
I20260709 12:26:25.849512  6611 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:25.850530  9315 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.850893  9656 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:25.850941  6611 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:25.851307  9479 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:25.814420  9811 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.810591  9731 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:25.851701  9675 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:25.851649  9656 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:25.871965  6024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 809) took 1517 ms (client timeout 1948 ms). Trace:
W20260709 12:26:25.872606  6024 rpcz_store.cc:269] 0709 12:26:24.344199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.344393 (+   194us) service_pool.cc:224] Handling call
0709 12:26:25.854686 (+1510293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:25.857172  9479 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.882050  9788 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:25.863163  9598 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:25.798900  9802 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.312s
I20260709 12:26:25.866889  9809 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:25.877988  9656 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:25.852147  9497 consensus_queue.cc:1059] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:26:25.851290  9315 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:25.855309  9775 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:25.911971  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1385) took 1703 ms. Trace:
I20260709 12:26:25.925303  5947 rpcz_store.cc:276] 0709 12:26:24.208847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.209035 (+   188us) service_pool.cc:224] Handling call
0709 12:26:24.276315 (+ 67280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.276603 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:24.303879 (+ 27276us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.865730 (+1561851us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:25.888132 (+ 22402us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:25.911463 (+ 23331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.911627 (+   164us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.911893 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1992,"prepare.queue_time_us":32548,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"thread_start_us":1494039,"threads_started":1}
I20260709 12:26:25.882663  9788 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.911923  9800 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:25.814628  9757 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:25.883599  9479 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:25.931918  9315 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:25.933604  9823 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.870183  9718 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:25.892858  9811 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:25.937224  9768 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.225s	user 0.026s	sys 0.052s
I20260709 12:26:25.917193  6611 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:25.928854  9668 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:25.924710  9809 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.944051  9788 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:25.991609  9768 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.408s	user 0.027s	sys 0.052s
I20260709 12:26:26.014566  9731 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:26.015810  9731 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:26.016522  9731 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:26.017915  9823 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:25.918790  9675 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:26.011053  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 137575us
I20260709 12:26:25.892458  9519 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:26.004846  9315 raft_consensus.cc:3055] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:26.033085  9757 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:26.024384  9668 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.067112  9668 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:26.068213  5840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.006s	sys 0.007s
W20260709 12:26:26.068728  5840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.006s	sys 0.007s
W20260709 12:26:26.073712  5872 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.005s	sys 0.011s
W20260709 12:26:26.074259  5872 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.005s	sys 0.011s
I20260709 12:26:26.075549  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 663) took 2127 ms. Trace:
W20260709 12:26:26.077912  5838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.005s	sys 0.008s
W20260709 12:26:26.078680  5838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.005s	sys 0.008s
W20260709 12:26:26.083324  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 794793us
W20260709 12:26:26.095715  5865 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.003s	sys 0.011s
W20260709 12:26:26.096182  5865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.004s	sys 0.011s
I20260709 12:26:26.097178  6611 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:26.034871  9315 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:26.082557  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 273144us
I20260709 12:26:26.106437  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 794) took 2721 ms. Trace:
I20260709 12:26:26.107262  6021 rpcz_store.cc:276] 0709 12:26:23.385183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.474109 (+ 88926us) service_pool.cc:224] Handling call
0709 12:26:24.115797 (+641688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.186826 (+ 71029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.104480 (+1917654us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b5400160b50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:26.106092 (+  1612us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1441716480}
I20260709 12:26:26.046000  9809 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:26.086561  9587 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.000s	sys 0.013s
I20260709 12:26:26.096503  9729 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.026615  9758 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:26.123978  6023 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.010s	sys 0.006s
W20260709 12:26:26.133060  6023 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.010s	sys 0.006s
W20260709 12:26:26.122010  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 93969.8us
W20260709 12:26:26.115074  9738 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.057s	sys 0.064s
W20260709 12:26:26.139863  5913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.015s	sys 0.014s
W20260709 12:26:26.141463  5913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.015s	sys 0.014s
W20260709 12:26:26.134266  9442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.000s	sys 0.016s
I20260709 12:26:26.118728  9680 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:26.126631  9587 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.000s	sys 0.014s
I20260709 12:26:26.143357  6611 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:26.127635  9802 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:26.147851  9442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.000s	sys 0.017s
W20260709 12:26:26.141289  9738 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.057s	sys 0.065s
I20260709 12:26:26.140599  9768 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.076210  5872 rpcz_store.cc:276] 0709 12:26:23.948275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.970910 (+ 22635us) service_pool.cc:224] Handling call
0709 12:26:23.971301 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.971568 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:24.037143 (+ 65575us) log.cc:844] Serialized 14 byte log entry
0709 12:26:26.074762 (+2037619us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:26.074878 (+   116us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:26.075058 (+   180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.075189 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.075458 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":49473,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":186921,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":20077,"thread_start_us":2030289,"threads_started":2,"wal-append.queue_time_us":1367807}
I20260709 12:26:26.066118  9731 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:26.158775  9802 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.268283  9768 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:26.274129  9768 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:26.279238  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 144833us
I20260709 12:26:26.142858  9598 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.110s
I20260709 12:26:26.280200  6611 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:26.289004  9775 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:26.290128  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.173568  9729 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:26.294329  9315 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:26.275679  9802 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:26.295846  9729 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:26.296631  9729 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:26.297765  9823 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:26.298484  9668 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:26.279412  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 111653us
I20260709 12:26:26.169446  9723 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.133s
I20260709 12:26:26.250422  9680 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:26.306062  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.325008  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.332010  5990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.008s	sys 0.008s
W20260709 12:26:26.332540  5990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.008s	sys 0.008s
I20260709 12:26:26.305892  9519 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:26.304987  9598 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:26.295893  9315 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:26.332260  5839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.010s	sys 0.064s
I20260709 12:26:26.297155  9729 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:26.354442  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:26.355628  9835 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.356034  6023 raft_consensus.cc:2410] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:26:26.306433  9718 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:26.355875  9729 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:26.362319  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:26.362573  9656 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:26.363886  6023 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:26.334898  9775 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:26.341534  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 53176.6us
W20260709 12:26:26.344398  5839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.011s	sys 0.064s
I20260709 12:26:26.305720  9809 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:26.346035  9842 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:26.347026  5942 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:26.329195  9823 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:26.348256  9840 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.349932  9738 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.352834  9831 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:26.353855  9839 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.354651  5948 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:26.370757  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:26.367682  5840 connection.cc:256] RPC call timeout handler was delayed by -1.936923305000164s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
I20260709 12:26:26.366719  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:26.395520  6021 raft_consensus.cc:2368] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12.
W20260709 12:26:26.392774  5840 connection.cc:256] RPC call timeout handler was delayed by -1.7704233049998948s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
I20260709 12:26:26.386344  9738 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:26.399132  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.398321  9831 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:26.406603  9831 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:26.398229  9723 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.407444  9723 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:26.408105  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.410391  9723 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:26.401759  5987 leader_election.cc:336] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.605s (SENT)
I20260709 12:26:26.416008  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:26.410629  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:26.343981  9598 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:26.365516  9656 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:26.368027  9675 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:26.422602  5840 connection.cc:256] RPC call timeout handler was delayed by -1.7605233050003335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
W20260709 12:26:26.426200  5840 connection.cc:256] RPC call timeout handler was delayed by -1.7605233050003335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
W20260709 12:26:26.426553  5840 connection.cc:256] RPC call timeout handler was delayed by -1.6795233050002025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
W20260709 12:26:26.426923  5840 connection.cc:256] RPC call timeout handler was delayed by -1.6183233050001036s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
W20260709 12:26:26.427328  5840 connection.cc:256] RPC call timeout handler was delayed by -1.5427233049999813s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1966s for a response
W20260709 12:26:26.427635  5840 connection.cc:256] RPC call timeout handler was delayed by -1.4689233050003168s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2048s for a response
W20260709 12:26:26.427977  5840 connection.cc:256] RPC call timeout handler was delayed by -1.3966702640000221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1544s for a response
W20260709 12:26:26.428325  5840 connection.cc:256] RPC call timeout handler was delayed by -1.394223305000196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
W20260709 12:26:26.428699  5840 connection.cc:256] RPC call timeout handler was delayed by -1.394223305000196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
I20260709 12:26:26.421013  6023 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 12: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:26.387207  9842 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.366140  9775 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:26.434600  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:26.438509  9809 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.438584  9843 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:26.439719  9768 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:26.356314  9835 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:26.441231  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.442482  5870 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 12:26:26.402609  9718 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.444773  5872 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:26.445254  5840 connection.cc:256] RPC call timeout handler was delayed by -1.3546233050001319s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2175s for a response
I20260709 12:26:26.422765  9668 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.432319  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:26.442442  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:26.446841  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:26.454663  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.454150  6775 raft_consensus.cc:2393] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:26.460726  6793 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
W20260709 12:26:26.455120  5840 connection.cc:256] RPC call timeout handler was delayed by -1.0741505229998438s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
I20260709 12:26:26.449472  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.463665  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.465142  5945 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:26.469578  9656 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.470546  9656 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:26.471669  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.472404  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.469745  9675 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.473663  9675 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:26.474175  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.474638  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:26.475241  5945 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 13: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:26.475646  6773 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:26.476227  9675 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:26.476521  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:26.477850  9656 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:26.482095  6766 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 14: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:26.485862  6773 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:26.487577  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.488016  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.488953  6785 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:26.488883  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.490787  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.491537  9718 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:26.491982  9768 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:26.493741  9838 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:26.444643  9723 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:26:26.494334  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.495158  5941 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:26.495256  9723 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:26.495887  9723 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:26.496338  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.497798  5871 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:26.496929  9723 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:26.498960  9723 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:26.425235  9840 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:26.503553  6021 raft_consensus.cc:2393] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 10: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:26.505878  9832 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.506529  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:26.505760  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:26.495375  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 89918.2us
I20260709 12:26:26.459239  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:26.434294  9842 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:26.510461  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.515635  5945 raft_consensus.cc:2393] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:26.516878  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.517812  6775 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 13: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:26.521972  5940 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 9: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:26.508759  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.502466  9668 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:26.401839  9839 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:26.429421  9823 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:26.532101  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.521485  9586 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:26.501539  9843 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:26.501380  9768 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:26.533174  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:26.533675  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1315) took 1132 ms. Trace:
I20260709 12:26:26.536451  5946 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:26.536502  5871 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:26.543349  5941 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:26.501396  9835 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:26.550079  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.551527  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.551784  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.552398  5940 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:26.556751  6775 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:26:26.513353  9840 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:26.563957  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:26.566169  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:26.566493  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 54785.2us
I20260709 12:26:26.567253  6021 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 9: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:26.538825  5865 rpcz_store.cc:276] 0709 12:26:25.398037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.400201 (+  2164us) service_pool.cc:224] Handling call
0709 12:26:25.400936 (+   735us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.096595 (+695659us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.096625 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.157482 (+ 60857us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.419787 (+262305us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:26.419826 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.419912 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:26.523291 (+103379us) raft_consensus.cc:1685] finished
0709 12:26:26.523320 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.529898 (+  6578us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13746,"raft.queue_time_us":448595,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":131,"thread_start_us":801277,"threads_started":2,"wal-append.queue_time_us":342288}
I20260709 12:26:26.568140  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.506698  9832 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:26.537591  9802 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:26.569192  5915 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:26.582101  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:26.582383  9802 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:26.582794  9835 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.516794  9598 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:26.583472  9835 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:26.584424  9598 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:26.585825  9835 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:26.586633  9835 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:26.587915  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:26.593590  9846 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:26.594219  9846 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:26.595518  9803 raft_consensus.cc:3055] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:26.596006  9803 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:26.598366  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:26.588800  5865 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:26.590097  6773 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:26.590302  9835 raft_consensus.cc:697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:26.570402  5873 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:26.570446  6762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 827) took 3770 ms (client timeout 1724 ms). Trace:
I20260709 12:26:26.571786  9832 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:26.572191  9718 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:26.573174  9731 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.570016  5940 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 6.
I20260709 12:26:26.595409  9846 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:26.588608  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1393) took 2293 ms. Trace:
I20260709 12:26:26.575088  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.660385  5941 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:26.579047  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.579504  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.580332  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.580295  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.577896  5944 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
W20260709 12:26:26.609269  6762 rpcz_store.cc:269] 0709 12:26:22.799735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.799946 (+   211us) service_pool.cc:224] Handling call
0709 12:26:26.570134 (+3770188us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b540021fa54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:26.570337 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4724840192}
I20260709 12:26:26.614269  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.575074  5990 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:26.576727  9788 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.564s	user 0.012s	sys 0.002s
I20260709 12:26:26.527433  9809 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:26.578490  9479 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.568s	user 0.006s	sys 0.006s
I20260709 12:26:26.658824  5943 rpcz_store.cc:276] 0709 12:26:24.294771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.296394 (+  1623us) service_pool.cc:224] Handling call
0709 12:26:24.296706 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.941698 (+644992us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:24.941804 (+   106us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:24.941909 (+   105us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:24.942008 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.588103 (+1646095us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b5400633154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:26.588160 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.588367 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1234763648}
I20260709 12:26:26.663000  6757 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
W20260709 12:26:26.693038  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 77733.6us
I20260709 12:26:26.694823  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:26.696171  5872 raft_consensus.cc:2410] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:26.698004  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.573412  9668 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:26.577673  9839 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:26.575682  9843 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:26.597044  9803 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:26.701490  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.708976  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.705826  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:26.706828  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:26.699844  9655 consensus_meta.cc:287] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.855s	user 0.008s	sys 0.008s
I20260709 12:26:26.714748  9655 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:26.718505  5949 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:26.712363  5865 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:26:26.576045  9831 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.126s
I20260709 12:26:26.721907  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:26.724529  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.736655  6762 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:26.576231  9586 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.129s
W20260709 12:26:26.739974  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 141727us
I20260709 12:26:26.739243  9768 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:26.736284  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:26.734612  9732 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:26.705003  9680 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:26.745129  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 332666us
I20260709 12:26:26.741206  5941 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:26.743782  5940 raft_consensus.cc:2410] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:26.750262  9586 raft_consensus.cc:3055] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:26.750859  9586 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:26.751823  9586 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:26.757148  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.753508  5946 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:26.738216  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:26.768893  5872 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
W20260709 12:26:26.769503  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.723280  9758 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:26.698509  9788 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:26.698843  9479 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:26.787848  5865 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:26.782235  5942 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:26.797035  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 261965us
I20260709 12:26:26.798897  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:26.788722  5866 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:26.802264  5869 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:26.802440  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:26.704985  9731 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:26.803328  6762 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:26.772085  6756 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:26.755319  9768 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:26.814778  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.897449  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 106633us
W20260709 12:26:26.912849  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 129578us
I20260709 12:26:26.917092  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:26.943447  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 368525us
I20260709 12:26:26.945386  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:26.962716  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 163485us
I20260709 12:26:26.841681  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 807) took 2544 ms. Trace:
I20260709 12:26:26.801474  5873 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:26.913089  9803 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:26.907380  9811 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:26.920586  9775 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:26.816704  6609 leader_election.cc:336] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.733s (SENT)
I20260709 12:26:27.090713  5945 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:27.091146  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.091493  9809 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:27.091856  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:26.881815  9758 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:26:27.093338  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.093611  6768 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:27.094453  5990 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:27.096487  9479 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:26.739269  9835 consensus_queue.cc:237] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:27.101822  9788 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:27.106657  6611 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:27.162266  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 55710.3us
I20260709 12:26:26.814095  9841 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:27.108011  9655 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:26.961598  5947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.034s	sys 0.052s
I20260709 12:26:27.165771  5872 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
W20260709 12:26:27.166291  5947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.040s	sys 0.053s
I20260709 12:26:27.143612  5942 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:27.141057  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:27.144917  5867 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:27.167577  5941 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 6. (term mismatch)
I20260709 12:26:27.130863  5949 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:27.114394  9823 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:27.094492  9738 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:27.165911  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 73946.4us
I20260709 12:26:27.093380  5870 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:27.169483  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:27.133044  6016 rpcz_store.cc:276] 0709 12:26:24.297259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.297988 (+   729us) service_pool.cc:224] Handling call
0709 12:26:24.298434 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:24.306538 (+  8104us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:24.306591 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:24.312980 (+  6389us) log.cc:844] Serialized 26 byte log entry
0709 12:26:25.568306 (+1255326us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:25.582400 (+ 14094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.841068 (+1258668us) spinlock_profiling.cc:243] Waited 7.55 ms on lock 0x7b540041d554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:26.841135 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:26.841203 (+    68us) raft_consensus.cc:1685] finished
0709 12:26:26.841217 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.841568 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":12894,"spinlock_wait_cycles":17365376,"thread_start_us":1121397,"threads_started":1,"wal-append.queue_time_us":1165871,"wal-append.run_cpu_time_us":4538,"wal-append.run_wall_time_us":1366533}
W20260709 12:26:27.159981  9842 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.612s	user 0.020s	sys 0.001s
I20260709 12:26:27.168119  5941 pending_rounds.cc:85] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 5
I20260709 12:26:27.165236  9479 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:27.176944  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:27.176559  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 82929us
I20260709 12:26:27.115193  9831 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
W20260709 12:26:27.180569  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.181483  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:27.182276  5874 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:26:27.183036  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.175304  5941 pending_rounds.cc:107] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:26:27.186307  5941 raft_consensus.cc:2889] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:27.186924  5941 pending_rounds.cc:107] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:26:27.187587  5941 raft_consensus.cc:2889] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:27.169066  9586 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:27.186865  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:27.193089  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.177021  9519 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:27.177511  9479 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.195881  5867 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:27.197829  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:27.200892  5913 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:27.201413  5942 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:27.202922  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:27.204087  5942 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:27.174865  9841 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:27.181173  9788 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:26:27.178783  9842 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:27.185493  9847 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:26:27.247875  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.234385  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:27.214255  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:27.272768  5942 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
W20260709 12:26:27.272965  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 118172us
I20260709 12:26:27.215276  9768 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:27.274443  5942 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:27.221740  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:27.218338  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.276544  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 63973.4us
I20260709 12:26:27.277418  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:27.278637  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:27.206707  9479 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:27.280551  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:27.235925  9840 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:27.258486  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:27.297590  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:27.235316  9732 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:27.221529  9831 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:27.302397  5874 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:27.199636  9811 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:26:27.302634  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.302148  5867 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:27.305275  6775 raft_consensus.cc:2368] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 12. Current term is 13.
I20260709 12:26:27.299806  9839 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:27.307525  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 490843us
I20260709 12:26:27.315285  5870 raft_consensus.cc:2376] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12. Re-sending same reply.
I20260709 12:26:27.302038  9838 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:27.258750  9823 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:26:27.322451  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.323267  5872 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:27.327276  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.327629  9483 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:27.327358  9860 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.118s
I20260709 12:26:27.314554  9866 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:27.333741  9866 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:27.334219  9866 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:27.335027  9867 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:26:27.336795  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.338085  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.338521  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:27.339030  5870 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:27.339329  6021 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:27.340420  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.340965  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:27.341254  5870 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:27.342545  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:27.346206  6021 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:26:27.321414  9598 consensus_meta.cc:287] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.736s	user 0.012s	sys 0.005s
W20260709 12:26:27.321151  9757 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.235s	user 0.012s	sys 0.008s
I20260709 12:26:27.344473  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:27.352082  6017 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:26:27.340535  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:27.313127  9742 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:27.317559  9315 consensus_meta.cc:287] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.938s	user 0.010s	sys 0.006s
I20260709 12:26:27.342651  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:27.321355  9723 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.822s	user 0.015s	sys 0.000s
I20260709 12:26:27.354990  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:27.363340  5987 leader_election.cc:336] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 2.204s (SENT)
I20260709 12:26:27.317577  9831 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:27.366586  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:27.361577  5869 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:27.328820  9809 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:26:27.378032  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:27.373620  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.373931  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:27.382453  5872 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:27.387588  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 129232us
I20260709 12:26:27.394048  5913 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:27.395323  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:27.396319  6020 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:27.400024  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:27.400583  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:27.401450  6018 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:27.400588  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 120125us
I20260709 12:26:27.419109  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:27.420466  6016 raft_consensus.cc:2410] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:26:27.421952  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1071) took 1139 ms. Trace:
I20260709 12:26:27.427270  5867 raft_consensus.cc:2410] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 12:26:27.428875  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:27.397790  9839 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:27.422338  5947 rpcz_store.cc:276] 0709 12:26:26.321504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.329120 (+  7616us) service_pool.cc:224] Handling call
0709 12:26:26.346713 (+ 17593us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.347163 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:26.393958 (+ 46795us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.387373 (+993415us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.421501 (+ 34128us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:27.421566 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:27.421613 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.421706 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.421880 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33839,"prepare.queue_time_us":103530,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":2400,"raft.queue_time_us":18938,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":6940,"thread_start_us":1025881,"threads_started":4,"wal-append.queue_time_us":811026}
W20260709 12:26:27.439087  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 75799.2us
I20260709 12:26:27.421800  9813 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:27.448058  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:27.400116  9838 raft_consensus.cc:697] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:27.451936  6024 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:26:27.452085  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.375618  9742 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:27.457150  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 362716us
I20260709 12:26:27.373193  9870 consensus_queue.cc:1059] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:26:27.458611  5942 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:27.453591  6609 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 2.234s (SENT) [suppressed 32 similar messages]
I20260709 12:26:27.461378  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:27.461994  5874 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:27.462723  5867 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 11: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:26:27.464331  9859 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
W20260709 12:26:27.466805  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 72872.8us
W20260709 12:26:27.469966  9856 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:26:27.471328  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:27.472266  5943 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 13: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:27.473207  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:27.474126  9848 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:27.474573  9848 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.474929  9848 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:27.478252  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:27.442771  9839 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:27.479403  9839 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:27.480182  5941 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:26:27.485287  6017 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:27.489569  9842 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:27.490221  9869 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:26:27.491071  9841 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:27.497296  9865 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:26:27.435382  9840 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:27.498775  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:27.535884  6021 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:27.499816  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:27.536841  5874 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:27.481498  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:27.443809  9873 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.462213  6016 raft_consensus.cc:2376] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12. Re-sending same reply.
W20260709 12:26:27.503175  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 49610.7us
I20260709 12:26:27.508069  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:27.525940  6018 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:27.542335  5867 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:27.533758  6020 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:27.481575  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:27.543818  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 62334.4us
I20260709 12:26:27.531333  5872 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:27.544687  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:27.538842  9866 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:27.542974  5947 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
W20260709 12:26:27.549748  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.550601  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:27.551510  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:27.543394  6611 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:27.550455  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 761721us
I20260709 12:26:27.555389  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.556452  5872 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:26:27.536037  9442 raft_consensus.cc:3055] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:26:27.566926  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 118952us
I20260709 12:26:27.444427  9831 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:27.558250  9479 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:27.574122  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:27.580284  5941 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:27.558763  9867 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:27.546505  9742 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:27.590287  5874 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:27.559453  9723 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:27.564198  9860 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:27.565954  9788 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.541144  9841 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
W20260709 12:26:27.602617  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 115905us
I20260709 12:26:27.599828  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:27.565583  9866 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:27.581368  9832 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:27.610204  9878 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.610977  9878 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:27.571200  9442 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:27.605762  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.614575  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:27.615322  5872 raft_consensus.cc:2376] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12. Re-sending same reply.
W20260709 12:26:27.616309  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 73057us
I20260709 12:26:27.586474  9837 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:27.557888  9848 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:27.578281  9598 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:26:27.592118  9867 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:27.552099  9873 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:27.715214  5873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 1324) took 1267 ms (client timeout 1652 ms). Trace:
W20260709 12:26:27.714208  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 143763us
I20260709 12:26:27.720625  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 843) took 1160 ms. Trace:
W20260709 12:26:27.657025  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.624862  5867 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:27.745395  6775 rpcz_store.cc:276] 0709 12:26:26.560521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.560745 (+   224us) service_pool.cc:224] Handling call
0709 12:26:27.720324 (+1159579us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b5400008554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:27.720504 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1460819328}
I20260709 12:26:27.711341  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.747174  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:27.668041  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 115369us
I20260709 12:26:27.733001  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:27.747829  5867 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:27.668020  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:27.743599  5873 rpcz_store.cc:269] 0709 12:26:26.448343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.448500 (+   157us) service_pool.cc:224] Handling call
0709 12:26:27.714964 (+1266464us) spinlock_profiling.cc:243] Waited 58.6 ms on lock 0x7b5400186d54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:27.715117 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134754560}
I20260709 12:26:27.747745  5874 raft_consensus.cc:2410] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:27.721930  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1074) took 1333 ms. Trace:
I20260709 12:26:27.598709  9723 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:27.701550  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.751700  5944 rpcz_store.cc:276] 0709 12:26:26.394391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.398766 (+  4375us) service_pool.cc:224] Handling call
0709 12:26:27.721720 (+1322954us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b5400675f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:27.721864 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398982016}
W20260709 12:26:27.754714  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 137392us
I20260709 12:26:27.757542  6019 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:27.761921  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1023) took 2021 ms. Trace:
W20260709 12:26:27.665488  9865 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:26:27.756453  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.763718  5873 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:27.762301  6019 rpcz_store.cc:276] 0709 12:26:25.740471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.737513 (+1997042us) service_pool.cc:224] Handling call
0709 12:26:27.757337 (+ 19824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:27.761478 (+  4141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.761855 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:27.769497  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 181284us
I20260709 12:26:27.768733  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.771361  5867 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 7.
I20260709 12:26:27.731383  9497 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:27.749722  9867 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:27.776942  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.638883  9878 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:27.719306  9813 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:27.603886  9860 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:27.775377  5872 raft_consensus.cc:2410] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:27.772500  5868 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:27.777616  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.773586  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.717729  9872 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:27.788434  5867 raft_consensus.cc:2410] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 12:26:27.788563  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.754647  9723 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:27.757962  9586 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.808199  5867 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:26:27.791210  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:27.809811  5873 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:27.793448  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:27.794340  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:26:27.776388  9803 consensus_meta.cc:287] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.638s	user 0.004s	sys 0.009s
I20260709 12:26:27.762960  9837 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:27.826690  5867 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:27.829200  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:26:27.835244  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 47645.3us
I20260709 12:26:27.776572  9497 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.637910  9315 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:27.790968  9860 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:27.610221  9832 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:27.665350  9788 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:27.848044  5872 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 12:26:27.824122  9831 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:27.794967  9872 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:27.878463  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:27.813606  9813 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:27.833350  9837 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:27.849100  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 891) took 1378 ms. Trace:
I20260709 12:26:27.880024  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:27.880292  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:27.881351  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:27.879518  6757 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:27.845019  9497 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:27.890514  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:27.700641  9840 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:27.899441  6756 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:27.880334  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:27.614058  9838 consensus_queue.cc:237] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:27.830334  9586 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:27.901932  6793 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:27.768597  9442 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:27.890661  6766 rpcz_store.cc:276] 0709 12:26:26.471100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.471299 (+   199us) service_pool.cc:224] Handling call
0709 12:26:27.848849 (+1377550us) spinlock_profiling.cc:243] Waited 8.72 ms on lock 0x7b5400005354. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:27.849029 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20056448}
W20260709 12:26:27.893402  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 47829.9us
I20260709 12:26:27.919077  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:27.917124  6785 raft_consensus.cc:2410] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:27.924085  9731 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:27.924846  9881 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 12:26:27.925949  9791 raft_consensus.cc:3055] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:27.926514  9791 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:27.927529  9791 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:27.928714  9791 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:27.929792  9586 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:27.932499  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:27.933858  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:27.934887  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 625) took 1525 ms. Trace:
I20260709 12:26:27.935222  9788 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:27.891006  9479 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:27.881855  9872 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:27.935321  5948 rpcz_store.cc:276] 0709 12:26:26.411688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.412268 (+   580us) service_pool.cc:224] Handling call
0709 12:26:27.722147 (+1309879us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.722631 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:27.722666 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.723382 (+   716us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.821989 (+ 98607us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:27.822060 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.822173 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.933077 (+110904us) raft_consensus.cc:1685] finished
0709 12:26:27.933096 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.934627 (+  1531us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400675f50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:27.934831 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2529,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":118,"spinlock_wait_cycles":3010987264,"thread_start_us":98018,"threads_started":1,"wal-append.queue_time_us":206152}
I20260709 12:26:27.936686  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:27.938741  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:27.939427  6757 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:27.939594  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:27.940295  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:27.940692  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:27.941035  6757 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:27.934903  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:27.942487  6785 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:27.942929  9832 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:27.954244  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 125889us
I20260709 12:26:27.848973  9315 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:27.977861  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:27.979400  9873 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:27.980347  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 185457us
I20260709 12:26:27.981681  5947 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:26:27.981973  5949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 626) took 1568 ms (client timeout 1948 ms). Trace:
W20260709 12:26:27.992054  5949 rpcz_store.cc:269] 0709 12:26:26.413391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.413535 (+   144us) service_pool.cc:224] Handling call
0709 12:26:27.981765 (+1568230us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b5400672d54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:27.981906 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":629326080}
I20260709 12:26:27.985459  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:27.949522  9803 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:28.005291  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:28.041880  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 200031us
I20260709 12:26:28.047115  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:28.049101  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 44056.7us
I20260709 12:26:28.013837  9878 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:28.054706  9893 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:28.061931  9731 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:28.063895  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 49709us
I20260709 12:26:28.064819  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:28.066497  9848 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:28.011560  9803 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:28.067767  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:28.068261  9803 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:28.069034  9878 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:28.055183  9497 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:28.069217  9832 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:28.070416  9832 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:28.070896  9832 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:28.071373  9832 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:28.013770  9873 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:28.076049  9873 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.076716  9873 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:28.049562  9813 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:28.079103  9315 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:28.082221  9497 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:28.084295  5915 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:28.087579  9835 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:28.088083  9835 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.088474  9835 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:28.075026  6022 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:28.017390  5870 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:28.090808  5915 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:28.094461  9848 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:28.096877  5915 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:28.098381  5913 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:28.116596  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:28.061360  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:28.108956  5840 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:28.124707  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:28.125663  9497 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:28.130040  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 68744.6us
I20260709 12:26:28.130913  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:28.131734  9497 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:28.132304  9497 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:28.133230  9497 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:28.138121  9680 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:28.140391  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:28.141749  9898 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:28.142277  9898 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.142637  9898 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:28.097227  9775 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:26:28.178797  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:28.185431  5867 raft_consensus.cc:2376] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9. Re-sending same reply.
W20260709 12:26:28.120715  9839 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.641s	user 0.010s	sys 0.003s
I20260709 12:26:28.150465  9890 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:28.165899  9655 raft_consensus.cc:3055] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:28.117992  9848 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:26.571573  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:28.170251  9768 raft_consensus.cc:3055] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:28.171645  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 673) took 1321 ms. Trace:
I20260709 12:26:28.119869  9846 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:26:28.174392  9889 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:26:28.161342  9882 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.551s	user 0.002s	sys 0.000s
I20260709 12:26:28.218897  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:28.219906  5874 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:28.190667  9811 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:28.156133  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 892) took 1682 ms. Trace:
W20260709 12:26:28.187129  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 46062us
I20260709 12:26:28.149782  6611 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:28.226120  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 76425us
I20260709 12:26:28.224269  6768 rpcz_store.cc:276] 0709 12:26:26.473656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.473800 (+   144us) service_pool.cc:224] Handling call
0709 12:26:26.474199 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:27.157589 (+683390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.155777 (+998188us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b5400005d54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:28.156007 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":935388928}
I20260709 12:26:28.227129  6611 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:28.210906  9867 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:28.207333  9848 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.238014  6611 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:28.119451  9876 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:28.210253  9768 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:28.216255  9846 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:28.194986  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 70334.9us
I20260709 12:26:28.210457  5869 rpcz_store.cc:276] 0709 12:26:26.850811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.090804 (+239993us) service_pool.cc:224] Handling call
0709 12:26:28.171406 (+1080602us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b5400026f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:28.171571 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502217984}
I20260709 12:26:28.198678  6609 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:28.183564  9775 consensus_queue.cc:1243] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:28.234622  5990 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:28.205428  9868 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:28.203065  9655 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:28.177071  5838 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:28.150683  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.269737  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.268632  5874 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:28.244719  9848 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:28.247234  9871 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:26:28.254377  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 55795.1us
I20260709 12:26:28.283345  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:28.235450  9841 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:26:28.289412  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:28.254842  5913 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:28.290343  6024 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:28.239574  9890 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:26:28.257616  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.242537  9483 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:28.205251  9839 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:28.277848  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:28.257395  9791 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:28.279466  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:28.261965  9519 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:28.300472  6021 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:28.280071  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 103114us
I20260709 12:26:28.245736  9876 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.252444  9846 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:28.250870  6611 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:28.292572  6609 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:28.262524  9868 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:28.295019  6020 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:28.279076  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:28.302983  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:28.293546  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:28.242240  9837 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:28.305682  6023 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:28.306305  5990 leader_election.cc:400] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12.
I20260709 12:26:28.306432  6015 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:28.281102  9883 raft_consensus.cc:2764] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:26:28.306895  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:28.307971  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 57224.4us
W20260709 12:26:28.268567  9842 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.011s	sys 0.018s
I20260709 12:26:28.310606  6019 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:28.317605  6611 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:28.315548  5913 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:28.316329  9868 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:28.312173  5990 leader_election.cc:403] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:28.304792  5838 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:28.251020  9723 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:28.316807  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1401) took 1217 ms. Trace:
W20260709 12:26:28.320777  5693 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40818 (request call id 147) took 55429 ms (client timeout 15000 ms). Trace:
I20260709 12:26:28.328573  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:28.300413  9731 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:28.299573  9866 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:28.284839  9871 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:28.326151  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:28.355283  6023 raft_consensus.cc:2410] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:26:28.333125  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:28.333696  9840 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:28.356014  5940 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:28.364080  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:28.364955  6016 raft_consensus.cc:2410] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 12:26:28.322618  9842 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.012s	sys 0.018s
I20260709 12:26:28.321012  9479 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:28.338142  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:28.320987  9791 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:28.340718  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 48280us
I20260709 12:26:28.303364  9876 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:28.307538  9890 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:28.341491  9519 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:28.342217  9899 raft_consensus.cc:2764] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:26:28.343570  9867 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:28.344517  9903 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:28.294835  9483 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:28.346298  9853 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:28.348032  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:28.417243  5944 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:28.321990  9860 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:28.341446  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.419885  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:28.352797  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:28.328828  5693 rpcz_store.cc:269] 0709 12:25:32.862504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.863217 (+   713us) service_pool.cc:224] Handling call
0709 12:26:28.281271 (+55418054us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:18.477332 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:18.669151 (+191819us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:18.678424 (+  9273us) write_op.cc:435] Acquired schema lock
0709 12:26:18.684907 (+  6483us) tablet.cc:662] Decoding operations
0709 12:26:18.717703 (+ 32796us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:18.717796 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:18.717827 (+    31us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:26:18.718330 (+   503us) tablet.cc:701] Row locks acquired
0709 12:26:18.718382 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:26:18.718597 (+   215us) write_op.cc:270] Start()
0709 12:26:18.718807 (+   210us) write_op.cc:276] Timestamp: P: 1783599978718513 usec, L: 0
0709 12:26:18.718837 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:26:18.719725 (+   888us) log.cc:844] Serialized 916 byte log entry
0709 12:26:19.398136 (+678411us) op_driver.cc:464] REPLICATION: finished
0709 12:26:19.556393 (+158257us) write_op.cc:301] APPLY: starting
0709 12:26:19.556521 (+   128us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:19.556717 (+   196us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:19.556896 (+   179us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:19.557466 (+   570us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:19.557845 (+   379us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:26:19.557911 (+    66us) write_op.cc:312] APPLY: finished
0709 12:26:19.558277 (+   366us) log.cc:844] Serialized 28 byte log entry
0709 12:26:19.558515 (+   238us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:19.558640 (+   125us) write_op.cc:454] Released schema lock
0709 12:26:19.558717 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:19.560253 (+  1536us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:19.560300 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5347414,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":137443,"apply.run_cpu_time_us":2751,"apply.run_wall_time_us":2756,"num_ops":2,"prepare.queue_time_us":573856,"prepare.run_cpu_time_us":22852,"prepare.run_wall_time_us":472364,"raft.queue_time_us":299199,"raft.run_cpu_time_us":42405,"raft.run_wall_time_us":191199,"replication_time_us":665949,"thread_start_us":808590,"threads_started":4,"wal-append.queue_time_us":173955,"wal-append.run_cpu_time_us":42234,"wal-append.run_wall_time_us":1662764}]]}
I20260709 12:26:28.368506  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:28.369553  6024 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:28.422952  6021 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
W20260709 12:26:28.374444  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 50231.7us
I20260709 12:26:28.328157  5946 rpcz_store.cc:276] 0709 12:26:27.099659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.308183 (+208524us) service_pool.cc:224] Handling call
0709 12:26:27.319292 (+ 11109us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.423641 (+104349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:27.423686 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.440012 (+ 16326us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.551089 (+111077us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:27.551147 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.551242 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.160500 (+609258us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:28.302637 (+142137us) raft_consensus.cc:1685] finished
0709 12:26:28.302662 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.316726 (+ 14064us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43734,"raft.queue_time_us":76032,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":9255,"spinlock_wait_cycles":9663872,"thread_start_us":144963,"threads_started":2,"wal-append.queue_time_us":121596}
I20260709 12:26:28.300818  9872 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:28.329387  9803 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:28.354506  9866 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.352211  9731 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.413081  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:28.414175  6020 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:28.412178  9479 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:28.375881  9883 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12.
I20260709 12:26:28.375351  9840 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:28.442854  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:28.439225  5869 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:28.378402  9519 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:26:28.441234  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 103217us
I20260709 12:26:28.412501  9586 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.467993  5987 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:28.479095  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:28.445947  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:28.327543  9768 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:28.493300  5940 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:28.492291  6024 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:28.438560  9598 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:28.545226  9598 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.545859  9598 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:28.430182  9803 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.442181  9866 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:28.548269  9803 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:28.520340  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:28.479414  9519 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:28.436444  9731 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:28.527577  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:28.491623  5873 pending_rounds.cc:85] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 3
I20260709 12:26:28.514544  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:28.536939  6018 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:28.363137  9655 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:28.541553  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:28.551863  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 270254us
I20260709 12:26:28.552881  5947 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:28.521806  6016 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:28.556263  5873 pending_rounds.cc:107] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:26:28.567742  5873 raft_consensus.cc:2889] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:28.515522  9871 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:28.558494  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.572896  6021 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:28.587903  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:28.597725  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 107431us
I20260709 12:26:28.490724  9586 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:28.618359  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 206795us
I20260709 12:26:28.575248  9867 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:28.624029  9900 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:28.625758  9905 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 12:26:28.626704  9909 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 12:26:28.627849  9902 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:26:28.625555  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:28.620743  9903 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:28.631529  9811 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:28.632321  9811 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:28.631995  9903 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:28.644770  6019 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:28.551362  9878 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:26:28.569077  9853 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:26:28.661065  6019 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:28.613149  9871 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:28.635134  9655 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:28.620254  9723 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:28.342427  5692 catalog_manager.cc:5697] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 3 to 9, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 9 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:28.720866  5692 catalog_manager.cc:5697] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:28.589886  9890 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:28.682094  9910 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 12:26:28.648044  9742 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:28.681679  5871 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 12:26:28.682158  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.682871  9907 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
W20260709 12:26:28.691987  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 590677us
I20260709 12:26:28.699195  5990 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:28.699903  9831 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:28.599447  9483 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:28.706278  9860 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:28.710884  9872 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:28.601824  9791 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:28.613802  9479 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:28.715068  9898 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.007s
I20260709 12:26:28.718542  6611 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:28.758778  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 89752.7us
W20260709 12:26:28.761938  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 112543us
I20260709 12:26:28.736346  9868 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:28.786641  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:28.787904  6024 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 11: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:26:28.735299  9873 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.658s	user 0.014s	sys 0.001s
W20260709 12:26:28.744637  9835 consensus_meta.cc:287] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.656s	user 0.009s	sys 0.002s
I20260709 12:26:28.795620  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:28.805500  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:28.810307  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:28.816892  5946 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:28.648046  9840 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:28.830058  6768 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:26:28.821303  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 122195us
W20260709 12:26:28.826164  5987 leader_election.cc:336] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.626s (SENT)
W20260709 12:26:28.833463  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 115003us
I20260709 12:26:28.787139  9315 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:28.834769  5990 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:28.829051  9868 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:28.839254  5990 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:28.806599  9870 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:28.729600  9598 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:28.829909  9907 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:26:28.764079  9742 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:28.861428  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:28.779575  9848 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:28.723003  5692 catalog_manager.cc:5697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 10, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 10 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:28.880576  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 54456.3us
I20260709 12:26:28.880580  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:28.805935  9873 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:28.798123  9898 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:28.892997  6022 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:26:28.820284  9835 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:28.889046  6611 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:28.898800  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:28.894344  9846 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.000s
I20260709 12:26:28.908140  9912 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:28.891841  9841 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:28.893801  9868 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:28.878979  9872 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
W20260709 12:26:28.904711  9832 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.833s	user 0.013s	sys 0.002s
W20260709 12:26:28.923784  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 84774.6us
I20260709 12:26:28.927692  6611 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:28.928442  6609 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:28.928921  6611 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:28.930511  9921 raft_consensus.cc:2764] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:26:28.931936  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1078) took 2389 ms. Trace:
I20260709 12:26:28.871363  9870 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:28.915724  9912 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.932490  5942 rpcz_store.cc:276] 0709 12:26:26.542134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.275303 (+733169us) service_pool.cc:224] Handling call
0709 12:26:27.275624 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:27.459266 (+183642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.931361 (+1472095us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b5400459cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:28.931672 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":420323584}
I20260709 12:26:28.930804  9832 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:28.959105  9925 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:26:28.959471  9917 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:28.960664  9832 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:28.904230  9811 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:28.963001  6609 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:28.963289  9811 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:28.938454  9842 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:28.912211  9853 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:26:28.968823  9921 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:28.974298  5692 catalog_manager.cc:5697] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:28.976392  9853 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:28.976959  9853 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:28.918318  9846 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:28.958530  9731 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:28.940382  9788 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:28.931066  9927 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 12:26:28.954175  9912 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:28.956285  9890 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:26:28.958186  9924 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 12:26:28.982388  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 137677us
I20260709 12:26:28.977974  6017 raft_consensus.cc:2410] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 12:26:29.009362  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.938833  9903 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 12:26:29.014691  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 87079.3us
I20260709 12:26:28.985276  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:29.016578  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:26:28.982975  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 78024us
I20260709 12:26:29.026845  5944 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:29.027786  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:29.028012  9442 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:29.029209  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:28.957233  9923 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:28.954838  9803 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:28.977521  9853 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:29.045678  6778 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:29.047339  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:28.984112  9842 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:28.999984  9598 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:28.981484  9731 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:29.055116  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:29.056118  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:29.068697  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:29.069489  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.069918  6757 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:29.073737  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:29.077220  9938 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:29.078227  9938 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:29.079413  9938 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:29.079509  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:29.047132  9742 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:29.086809  9842 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:28.973378  9917 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.095s
W20260709 12:26:29.063247  9866 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.510s	user 0.013s	sys 0.000s
I20260709 12:26:29.096778  5944 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:29.097990  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:29.098868  6793 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:29.093989  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:29.095466  6019 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:29.102365  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:29.104554  5865 raft_consensus.cc:2410] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:29.105232  9768 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:29.105767  9930 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:26:29.109833  9860 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.110577  9860 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:29.037945  9811 consensus_queue.cc:1059] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:26:29.023568  9826 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.244s
I20260709 12:26:29.062589  9867 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:29.122349  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:29.119426  5944 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:29.075472  9872 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:29.103951  9876 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.687s	user 0.006s	sys 0.007s
I20260709 12:26:29.096078  9846 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:29.093364  9873 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:29.137159  5870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 690) took 1802 ms (client timeout 1856 ms). Trace:
I20260709 12:26:29.129740  6021 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:29.136981  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:29.094358  9938 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:29.143987  5870 rpcz_store.cc:269] 0709 12:26:27.334581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.374892 (+ 40311us) service_pool.cc:224] Handling call
0709 12:26:29.136860 (+1761968us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54002f99d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:29.137050 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":987540352}
I20260709 12:26:29.098409  9887 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:29.103741  9923 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:29.126207  9867 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:28.977506  5692 catalog_manager.cc:5697] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 5 to 10, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 10 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:29.047272  9911 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.243s
I20260709 12:26:29.100939  9866 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:29.128484  9788 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:29.159797  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:29.161451  9939 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:29.167505  9940 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.159030  9742 raft_consensus.cc:3055] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:26:29.174470  9932 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 12:26:29.174225  9938 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:29.143275  9835 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:29.167639  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:29.153170  9898 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:29.151880  9923 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:29.143525  9876 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:29.178175  9872 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:29.187861  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:29.180239  9519 consensus_meta.cc:287] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.585s	user 0.007s	sys 0.007s
I20260709 12:26:29.186728  9742 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:29.176476  9890 raft_consensus.cc:697] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:29.211107  5871 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:29.218463  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:29.222095  5874 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:29.227823  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 263345us
I20260709 12:26:29.209556  9923 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:29.230204  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:29.209733  9803 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:29.219848  9860 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:29.249698  5874 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:26:29.254549  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:29.254272  9742 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:29.255219  5874 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:26:29.255800  9742 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:29.235229  9938 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:29.218331  9519 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:29.228739  9870 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:29.274915  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:29.287657  5872 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:29.280192  9839 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:29.283365  9876 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:29.193917  9940 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:29.308475  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:29.311913  5949 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:29.312950  9519 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:29.313987  6762 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:29.314527  9940 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:29.317255  9868 raft_consensus.cc:3055] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:29.317386  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:29.317869  9868 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:29.306977  9890 consensus_queue.cc:237] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:29.319432  9908 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.749s	user 0.000s	sys 0.004s
I20260709 12:26:29.321484  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:29.250432  9788 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:29.323511  9868 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:29.325001  9868 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:29.325242  9866 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:29.325899  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:29.327668  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:29.328325  5869 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:29.292680  5874 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
W20260709 12:26:29.298936  9935 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.324s	user 0.002s	sys 0.002s
I20260709 12:26:29.311225  5871 raft_consensus.cc:2368] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13.
I20260709 12:26:29.331763  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:29.333012  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:29.337458  5941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1088) took 1719 ms (client timeout 1793 ms). Trace:
W20260709 12:26:29.338014  5941 rpcz_store.cc:269] 0709 12:26:27.618112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.772717 (+154605us) service_pool.cc:224] Handling call
0709 12:26:29.337175 (+1564458us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54006a3bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:29.337343 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3307074816}
I20260709 12:26:29.285403  5692 catalog_manager.cc:5697] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:29.339564  9837 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:29.342026  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:29.342775  5872 raft_consensus.cc:2376] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13. Re-sending same reply.
I20260709 12:26:29.343086  9912 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:29.340193  5692 catalog_manager.cc:5697] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 11, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 11 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:29.343868  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:29.348222  9442 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:29.348347  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:29.341341  9921 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:29.351423  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:29.384938  5871 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:29.385385  5947 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:29.295027  9860 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:29.353242  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 188401us
I20260709 12:26:29.353976  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:29.391302  5874 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:29.362691  6609 connection.cc:256] RPC call timeout handler was delayed by -1.8782115559997692s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19390000000000002s for a response
W20260709 12:26:29.392746  6609 connection.cc:256] RPC call timeout handler was delayed by -1.8476115559997197s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19730000000000003s for a response
W20260709 12:26:29.393342  6609 leader_election.cc:336] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.799s (SENT)
I20260709 12:26:29.394065  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:29.401504  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.434110  6021 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:29.275763  9918 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
W20260709 12:26:29.435281  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 44598.2us
I20260709 12:26:29.352406  9940 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:26:29.442551  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:29.443816  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 50527.6us
W20260709 12:26:29.444620  6609 leader_election.cc:336] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.856s (SENT)
I20260709 12:26:29.446063  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:29.446509  9768 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:29.447494  9912 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:29.448194  9912 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:29.450030  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:29.450935  5871 raft_consensus.cc:2368] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12.
I20260709 12:26:29.452139  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:29.446873  5872 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:29.453783  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:29.454464  5872 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:29.434167  9860 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:29.461844  9928 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.494s	user 0.001s	sys 0.001s
I20260709 12:26:29.463195  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:29.463583  9933 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.489s	user 0.005s	sys 0.000s
I20260709 12:26:29.464051  5945 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:29.465615  5987 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:29.467466  9871 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:29.467931  9871 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.468241  9871 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:29.470798  5692 catalog_manager.cc:5697] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 11, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 11 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:29.445518  6609 connection.cc:256] RPC call timeout handler was delayed by -1.3922115559998929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2479s for a response
W20260709 12:26:29.474009  6609 connection.cc:256] RPC call timeout handler was delayed by -1.2129191409999294s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
W20260709 12:26:29.474303  6609 connection.cc:256] RPC call timeout handler was delayed by -1.1763147189999472s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1824s for a response
I20260709 12:26:29.475514  9953 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.476060  9953 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:29.478876  9953 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:29.414633  9442 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:29.480677  9442 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:29.481431  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:29.482924  5872 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:29.495570  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:29.496573  6778 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:26:29.499635  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:29.501232  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:29.511122  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:29.512850  5990 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:29.515652  9923 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:29.469004  9867 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:29.526055  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:29.527189  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:29.529045  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:29.529721  6768 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:29.519502  9923 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:29.531754  5947 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:29.532594  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:29.533234  6766 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:29.537278  6793 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:29.535089  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:29.557725  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 45140us
I20260709 12:26:29.503692  9876 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:29.522476  9655 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:29.533033  9598 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:29.578887  9655 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:29.521468  9921 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:26:29.541953  9940 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:29.578930  6793 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:29.601735  9768 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:26:29.611351  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 130019us
W20260709 12:26:29.591116  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.614594  6768 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:29.615190  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:29.566244  9775 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:29.584209  9598 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.629016  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:29.581799  9860 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:29.636029  6775 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:29.581339  9866 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:26:29.642633  9848 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:29.633198  9887 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:29.571413  9853 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:29.662335  6757 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:29.662858  9586 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:29.664768  9867 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:29.642746  9866 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:29.666708  9866 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:29.581406  9876 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:29.667817  9866 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:29.669267  9866 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:29.633366  9598 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:29.631026  9655 raft_consensus.cc:697] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:29.677521  9655 consensus_queue.cc:237] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:29.626513  9921 consensus_queue.cc:1243] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:29.720183  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1350) took 1287 ms. Trace:
I20260709 12:26:29.720811  5873 rpcz_store.cc:276] 0709 12:26:28.432511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.441743 (+  9232us) service_pool.cc:224] Handling call
0709 12:26:28.491386 (+ 49643us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:28.584298 (+ 92912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:28.584329 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:28.587386 (+  3057us) log.cc:844] Serialized 52 byte log entry
0709 12:26:29.271184 (+683798us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b18002cd4c0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:26:29.648639 (+377455us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:29.650300 (+  1661us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.650595 (+   295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.650710 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:29.719765 (+ 69055us) raft_consensus.cc:1685] finished
0709 12:26:29.719880 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.720108 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":796,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":9298,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":16387,"spinlock_wait_cycles":24109440,"thread_start_us":377174,"threads_started":1,"wal-append.queue_time_us":1015746}
I20260709 12:26:29.727514  9868 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:29.728569  9868 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:29.729010  9853 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:29.735561  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 120402us
I20260709 12:26:29.736402  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:29.736773  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:29.738734  9876 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:29.739530  9842 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:29.739735  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:29.743361  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:29.631922  9775 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:29.746608  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:29.750312  9842 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:29.747607  5941 raft_consensus.cc:2376] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12. Re-sending same reply.
I20260709 12:26:29.756826  5942 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: [14.6-14.6]
I20260709 12:26:29.757332  6778 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:29.760263  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:29.763245  6766 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:29.763621  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:29.765697  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:29.772225  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:29.774245  5942 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:29.775166  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:29.777184  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:29.781451  6609 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:29.767768  9890 raft_consensus.cc:2764] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 12:26:29.788894  5946 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:29.789273  5947 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:29.770445  9953 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:29.793066  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:29.797703  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:29.807698  5946 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:26:29.787700  9842 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:29.819314  5944 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 11. Current term is 12. Ops: [10.8-11.9]
I20260709 12:26:29.790524  9922 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:29.831290  5944 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:29.832857  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:29.833711  5944 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:29.834223  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:29.834549  9598 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:29.838157  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:29.838466  9598 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:29.848619  9871 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:29.850339  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.800063  9788 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:26:29.823262  9775 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:29.854863  5942 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:29.804714  9960 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:26:29.801203  9853 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:26:29.863173  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:29.867350  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:29.857190  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:29.858538  9950 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:29.868945  5940 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:29.869513  5871 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
W20260709 12:26:29.871178  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 336202us
I20260709 12:26:29.873714  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:29.874001  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:29.865726  9842 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:29.887252  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.887629  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:29.888721  5942 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:29.889109  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:26:29.889946  5987 leader_election.cc:336] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.532s (SENT)
I20260709 12:26:29.891645  5940 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:29.871147  9950 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:29.893051  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:29.890904  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:29.901372  5940 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 12:26:29.941519  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51622.1us
I20260709 12:26:29.954965  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:29.955870  5871 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:29.930749  9868 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:29.960687  5942 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:29.910667  9950 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:29.941435  9973 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:29.978004  5987 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:29.932291  9894 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:26:29.983470  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:29.990828  5940 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:29.992074  5946 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
W20260709 12:26:30.002923  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.008893  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:30.013587  5871 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:30.029280  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:30.031666  5987 leader_election.cc:336] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.573s (SENT)
I20260709 12:26:30.032212  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:29.982784  9972 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:30.036877  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 160218us
I20260709 12:26:30.037933  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:30.040697  9972 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:30.047065  5871 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:30.037287  9871 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:30.037281  9973 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:30.045113  9979 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:26:30.058344  5987 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.838s (SENT)
I20260709 12:26:30.058915  5987 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:30.050226  9975 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:26:30.047693  9843 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:26:30.060403  9871 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:30.062085  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.062938  5872 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:30.066571  5987 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.925s (SENT)
I20260709 12:26:30.067214  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:30.068779  9871 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:30.069016  9972 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:30.070166  5945 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:30.070487  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.048506  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.083877  5871 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:30.084601  9479 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.088481  9853 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:30.088688  9479 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:30.089514  9788 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:30.089967  9788 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:30.090044  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:30.091209  9479 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:30.095516  6609 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:30.091122  9788 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:30.097080  9788 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:30.087527  9980 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:26:30.101521  5872 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:30.101795  9980 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:30.102829  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.103549  5872 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:26:30.103806  5940 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:26:30.082743  9979 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:26:30.100155  9479 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:26:30.106688  9960 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:30.091141  9853 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:30.105486  5987 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:30.104645  9980 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:30.114259  5871 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:30.095949  9843 consensus_queue.cc:1243] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:30.115367  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:30.115907  5987 leader_election.cc:336] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.710s (SENT)
I20260709 12:26:30.121991  5871 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:30.121452  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:30.108753  9979 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:30.109836  9479 consensus_queue.cc:1243] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:30.127256  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.121649  9853 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:30.132659  9479 consensus_queue.cc:1059] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:30.132375  9980 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:30.132967  9853 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:30.133920  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.134261  9980 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:30.134601  5871 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:30.117858  9973 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.128810  9921 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:30.129892  5987 leader_election.cc:336] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.790s (SENT)
I20260709 12:26:30.138072  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:30.122051  9871 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:30.138813  5940 pending_rounds.cc:85] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 6
I20260709 12:26:30.139196  5940 pending_rounds.cc:107] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:26:30.139276  9921 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:30.139451  5940 raft_consensus.cc:2889] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:30.141955  9871 raft_consensus.cc:697] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:30.143097  9871 consensus_queue.cc:237] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:30.144199  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:30.146116  5987 leader_election.cc:336] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.845s (SENT)
I20260709 12:26:30.130793  5872 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:30.137805  9973 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:30.147436  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.145004  5866 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:30.136782  5944 raft_consensus.cc:2410] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 12:26:30.150067  5987 leader_election.cc:336] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.805s (SENT)
W20260709 12:26:30.152525  5874 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.761s	user 0.014s	sys 0.002s
W20260709 12:26:30.153414  5987 leader_election.cc:336] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 2.052s (SENT)
I20260709 12:26:30.154412  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:30.157303  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:30.153147  5874 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:26:30.158360  9655 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.160455  5913 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:30.170543  5871 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:30.172189  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.172760  5913 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:30.158232  9442 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.156086  9768 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:26:30.161679  9866 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:30.176082  5913 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:30.168435  9655 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:30.185593  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.163446  9867 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:30.189234  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.189042  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:30.171780  9847 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:30.194291  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.194936  5872 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:30.185956  9944 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:30.191784  9868 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:30.191982  9923 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:30.203692  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.204188  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.205201  5872 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 10: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:30.207331  5866 raft_consensus.cc:2410] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:26:30.202911  9944 raft_consensus.cc:697] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:30.177287  9442 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:30.196687  9847 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:30.212152  9847 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:30.220626  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.209046  9944 consensus_queue.cc:237] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:30.224814  9921 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:30.225013  9963 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.578s	user 0.000s	sys 0.002s
I20260709 12:26:30.227516  5866 raft_consensus.cc:2393] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:30.232076  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.233296  5866 raft_consensus.cc:2410] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:30.234647  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.236931  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.237864  5872 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:30.238296  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.239404  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.241254  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.241770  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.242466  5866 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 13: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:30.251446  5866 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:30.255087  5866 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:30.252477  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.258180  5872 raft_consensus.cc:2410] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:26:30.261344  5872 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:30.266599  5872 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:26:30.277141  5944 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: [14.6-14.6]
W20260709 12:26:30.278199  5987 leader_election.cc:336] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.943s (SENT)
I20260709 12:26:30.279067  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:30.279592  5940 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:30.288357  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.280858  9921 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:30.294837  5871 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:30.295603  5987 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 2.187s (SENT)
I20260709 12:26:30.297070  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:30.290169  9867 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:30.308228  9867 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:30.310946  9867 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:30.313906  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:30.314661  9841 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:30.318485  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:30.319244  6766 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:30.323194  6778 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:26:30.321741  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:30.323627  6778 pending_rounds.cc:85] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 8
I20260709 12:26:30.323887  6757 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:30.323896  6778 pending_rounds.cc:107] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:26:30.324883  6778 raft_consensus.cc:2889] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:30.325224  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:30.325762  6768 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
W20260709 12:26:30.327925  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.315553  9972 consensus_queue.cc:1059] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:26:30.332262  9886 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:30.322427  9731 raft_consensus.cc:3055] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:30.332867  9886 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:30.333079  9731 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:30.333328  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:30.334024  9886 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:30.335217  9886 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:30.335664  9867 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:30.351466  5990 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:30.334314  9731 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:30.352833  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:30.358114  5940 raft_consensus.cc:2410] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:26:30.352113  6766 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:26:30.356415  5944 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:30.375782  5871 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:30.383708  5871 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:30.391548  6757 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:30.394464  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:30.393419  9731 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:30.401213  9994 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:30.419217  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1356) took 1120 ms. Trace:
I20260709 12:26:30.419062  9972 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:26:30.420630  9988 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:26:30.408303  9993 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:26:30.425770  9917 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:30.426477  9917 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:30.433727  5987 leader_election.cc:336] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.516s (SENT)
I20260709 12:26:30.434294  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:30.427547  9917 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:30.438624  9917 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:30.419734  5866 rpcz_store.cc:276] 0709 12:26:29.298432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.256337 (+957905us) service_pool.cc:224] Handling call
0709 12:26:30.256690 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.322660 (+ 65970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.322706 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.336159 (+ 13453us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.402158 (+ 65999us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:30.402203 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.402273 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.418621 (+ 16348us) raft_consensus.cc:1685] finished
0709 12:26:30.418673 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.418871 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90711,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":128,"thread_start_us":131019,"threads_started":2,"wal-append.queue_time_us":77616}
I20260709 12:26:30.447314  9831 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:30.459792  5869 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.131s	user 0.019s	sys 0.000s
I20260709 12:26:30.460305  5869 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:30.460888  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 1360) took 1161 ms. Trace:
I20260709 12:26:30.461189  5869 rpcz_store.cc:276] 0709 12:26:29.299692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.327365 (+ 27673us) service_pool.cc:224] Handling call
0709 12:26:30.460807 (+1133442us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.425846  9972 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:30.471385  9972 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:30.472342  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.471930  9962 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.826s	user 0.005s	sys 0.000s
W20260709 12:26:30.483268  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.494333  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1011) took 1346 ms. Trace:
I20260709 12:26:30.494933  5867 rpcz_store.cc:276] 0709 12:26:29.147559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.147966 (+   407us) service_pool.cc:224] Handling call
0709 12:26:29.148414 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:29.157708 (+  9294us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:29.157760 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:29.185616 (+ 27856us) log.cc:844] Serialized 26 byte log entry
0709 12:26:29.331554 (+145938us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:29.331618 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.331737 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.494011 (+1162274us) raft_consensus.cc:1685] finished
0709 12:26:30.494040 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.494247 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2421,"raft.queue_time_us":8778,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":4176,"thread_start_us":110374,"threads_started":1,"wal-append.queue_time_us":352286}
W20260709 12:26:30.497833  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.502912  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1019) took 1462 ms. Trace:
I20260709 12:26:30.503535  6762 rpcz_store.cc:276] 0709 12:26:29.040500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.040726 (+   226us) service_pool.cc:224] Handling call
0709 12:26:29.054622 (+ 13896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.314443 (+259821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.502616 (+1188173us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54002a17d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:30.502836 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":595854208}
W20260709 12:26:30.507702  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.508231  5943 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:26:30.511386  5865 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.206s	user 0.015s	sys 0.022s
W20260709 12:26:30.511839  5865 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.207s	user 0.015s	sys 0.022s
I20260709 12:26:30.513904  9942 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:30.516475  9942 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:30.516896  9942 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:30.520385  9942 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:30.524330  6022 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:30.526217  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:30.522006  9831 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.526964  6016 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:30.527489  6019 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:30.527227  9831 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:30.529731  6019 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:30.529488  9999 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:30.530186  9831 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:30.522555  9921 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.531622  9921 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:30.521520  6762 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:26:30.531998  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:30.532454  9873 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:30.525321  9993 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.534946  9993 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:30.535795  9921 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:30.536442  9994 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:30.537314  9994 consensus_queue.cc:1243] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:30.539194  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:30.539913  6756 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:30.540838  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.541531  5866 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:30.542280  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.543078  5872 raft_consensus.cc:2368] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13.
I20260709 12:26:30.545725  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.547539  9841 raft_consensus.cc:3055] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:30.548224  9841 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:30.549321  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:30.550200  5943 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
W20260709 12:26:30.552253  6017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 1025) took 2273 ms (client timeout 2361 ms). Trace:
I20260709 12:26:30.552600  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:30.552778  6017 rpcz_store.cc:269] 0709 12:26:28.278640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.278808 (+   168us) service_pool.cc:224] Handling call
0709 12:26:30.551939 (+2273131us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b54003647d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:30.552110 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1535030016}
I20260709 12:26:30.553388  5866 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:30.547493  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.556174  9718 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:30.557098  5871 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:30.549260  9841 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:30.563259  9841 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:30.567879  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.568706  5872 raft_consensus.cc:2376] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9. Re-sending same reply.
I20260709 12:26:30.569779  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.570678  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1039) took 1125 ms. Trace:
I20260709 12:26:30.570832  6019 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:30.571071  5869 rpcz_store.cc:276] 0709 12:26:29.445612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.569480 (+1123868us) service_pool.cc:224] Handling call
0709 12:26:30.570601 (+  1121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.572546  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:30.574528  9973 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:30.576150  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:30.577279  9973 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:30.577864  9973 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.578385  9973 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:30.585616  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:30.587741  6756 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:30.600723  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.602073  5870 raft_consensus.cc:2376] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9. Re-sending same reply.
I20260709 12:26:30.537503  9993 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:30.605403  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.587689  9718 consensus_queue.cc:1243] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:30.607100  5869 raft_consensus.cc:2368] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12.
I20260709 12:26:30.609119  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.610132  5869 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 10: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:30.611353  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.582273  9655 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:30.613147  5867 raft_consensus.cc:2393] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:30.610185  9942 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:30.622917  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.624015  5867 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:30.631165  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:29.473062  5692 catalog_manager.cc:5697] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 4 to 7, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 7 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:26:30.635093  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.636879  5692 catalog_manager.cc:5697] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 13, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 13 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:30.646390  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.647490  5867 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:30.647642  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.647861  5692 catalog_manager.cc:5697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 2 to 13, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 13 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:30.640915  9973 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:30.653208  9973 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:30.653990  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.654673  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.655125  5871 raft_consensus.cc:2410] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:26:30.655843  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.657052  5692 catalog_manager.cc:5697] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 7 to 11, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 11 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:30.659754  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.660535  5866 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:30.661432  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.662305  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.661484  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.654107  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.665474  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.572472  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:30.669085  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.659289  5692 catalog_manager.cc:5697] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:30.671775  5692 catalog_manager.cc:5697] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:30.673425  5692 catalog_manager.cc:5697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 3 to 10, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 10 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:30.676091  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.675642  5692 catalog_manager.cc:5697] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 8, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 8 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:30.677915  5692 catalog_manager.cc:5697] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 9, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 9 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:26:30.679775  5692 catalog_manager.cc:5697] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 1 to 8, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 8 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:26:30.681506  5692 catalog_manager.cc:5697] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 3 to 4, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 4 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:26:30.686625  5692 catalog_manager.cc:5697] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 1 to 9, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 9 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:30.688977  5692 catalog_manager.cc:5697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:30.694293  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1010) took 1567 ms. Trace:
I20260709 12:26:30.694836  5865 rpcz_store.cc:276] 0709 12:26:29.127196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.127355 (+   159us) service_pool.cc:224] Handling call
0709 12:26:29.146493 (+ 19138us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.146740 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:29.162275 (+ 15535us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.304952 (+142677us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b18000adf80. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:26:30.471168 (+1166216us) spinlock_profiling.cc:243] Waited 5.89 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:30.693858 (+222690us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:30.693930 (+    72us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:30.693991 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.694071 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.694219 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":181446,"prepare.queue_time_us":15531,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":3627,"spinlock_wait_cycles":19236864,"thread_start_us":1207032,"threads_started":1,"wal-append.queue_time_us":455243}
I20260709 12:26:30.699843  9990 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:30.700623  9990 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:30.704236  9990 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:30.712469  9990 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:30.713375  9990 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:30.716614  9990 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:30.725687  5872 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:30.726246  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.728597  5865 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:30.730629  5865 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: [14.6-14.6]
I20260709 12:26:30.735980  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:30.737601 10009 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:30.739923 10009 raft_consensus.cc:2764] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 12:26:30.753168  9964 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 1.111s	user 0.005s	sys 0.000s
I20260709 12:26:30.759214  9990 consensus_queue.cc:1059] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:30.760350  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:30.764072 10012 raft_consensus.cc:3055] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:30.764838  9990 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:30.764788 10012 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:30.784771 10012 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:30.786041  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.786273 10012 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:30.786796  5865 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:30.788033  5865 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:26:30.789386  5865 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 12:26:30.791419  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.796641  9990 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:30.798067  9990 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:30.799511  9976 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.758s	user 0.000s	sys 0.005s
I20260709 12:26:30.801838  6756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:30.802863  6756 raft_consensus.cc:606] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
W20260709 12:26:30.813210  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.823730 10013 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:30.824501 10013 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:30.826366  9943 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:30.827044  9943 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:30.827747 10013 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:30.831115  6016 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:30.831130  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:30.833938  6756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:30.835314  6756 raft_consensus.cc:606] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:26:30.834628  9943 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:30.840596  9943 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:30.840974  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.842718  9973 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:30.843327  9973 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:30.845093  9973 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:30.847671 10003 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.391s	user 0.002s	sys 0.000s
I20260709 12:26:30.851176  9442 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:30.853790  9971 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.845s	user 0.005s	sys 0.000s
I20260709 12:26:30.857710  5867 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:26:30.861997  9977 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.820s	user 0.000s	sys 0.002s
W20260709 12:26:30.896791  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.897780  5990 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:30.899802  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:30.901968 10001 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:30.903542  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:30.906369  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:30.901561  9942 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:30.907338  9942 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.907763  9942 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:30.910496  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.912184  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:30.914053  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.915549  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.917716  9942 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:30.920179  9942 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:30.921294  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:30.922298  6762 raft_consensus.cc:2410] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 15 because replica has last-logged OpId of term: 14 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:30.926194  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:30.926482  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:30.927115  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 856) took 1744 ms. Trace:
I20260709 12:26:30.927281  5865 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:30.927448  6756 raft_consensus.cc:2410] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:26:30.928778  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:30.929790  6766 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:30.928782  5866 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:30.932741  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:30.933175  9950 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:30.933391  6766 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:30.933604  9990 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.934316  9990 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:30.935578  6016 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:30.936949  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:30.937685  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:30.938645  6019 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:26:30.939079  6019 pending_rounds.cc:85] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 5
I20260709 12:26:30.939360  6019 pending_rounds.cc:107] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:26:30.939213  5869 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:30.939870  6019 raft_consensus.cc:2889] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:30.940793  6016 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:30.941061  6017 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:30.941915  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:30.942525  6016 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:30.942625  6017 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:26:30.944039  6019 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:30.927520  6023 rpcz_store.cc:276] 0709 12:26:29.183277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.183527 (+   250us) service_pool.cc:224] Handling call
0709 12:26:29.191900 (+  8373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:29.315332 (+123432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.315362 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:29.328414 (+ 13052us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b2c001ddd28. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:26:29.328937 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 12:26:29.530926 (+201989us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:29.530975 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.752450 (+1221475us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:30.926774 (+174324us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b540041da54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:30.926833 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.926884 (+    51us) raft_consensus.cc:1685] finished
0709 12:26:30.926897 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.927054 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":418,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"spinlock_wait_cycles":344561152,"thread_start_us":201408,"threads_started":1,"wal-append.queue_time_us":313133}
W20260709 12:26:30.952966  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.962492  9990 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:30.977774  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:30.978056  5869 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:30.983505  6766 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:30.987214  6017 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:30.988544  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:30.988863  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:30.989483  6023 raft_consensus.cc:2368] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12.
I20260709 12:26:30.989914  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:30.990319  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.991262  6019 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 8: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:30.990715  5945 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:30.999260  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:30.992559  5990 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:31.009181  5838 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:31.013939  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:31.015283  6018 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 9: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:31.012787  6024 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:31.017545  6019 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:31.018868  6021 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:31.020246  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:31.026903  6023 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:31.029132  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:31.033246  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:31.018433 10013 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:31.032778  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:31.041744  6018 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:31.046701  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:31.029412  9442 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:31.028488  9973 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.042835  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:31.051338  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.037070  9866 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:31.055910  9866 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:31.057847  9840 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:31.058380  9840 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:31.058802  9840 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:31.064620  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:31.038653 10013 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.065243  5913 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:31.063958  9442 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:31.052295  9846 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:31.071729 10013 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:31.057126  9866 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:31.085090  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:31.066195  9800 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:26:31.063663  9973 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:31.100545  6023 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:31.126871  6015 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:26:31.127463  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:31.131426  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:31.132045  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 692) took 3796 ms. Trace:
I20260709 12:26:31.114202  9442 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:31.132469  5868 rpcz_store.cc:276] 0709 12:26:27.335225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.808391 (+473166us) service_pool.cc:224] Handling call
0709 12:26:29.137904 (+1329513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.153893 (+ 15989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.153948 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:29.154015 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:29.154078 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.154182 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.131843 (+1977661us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54002f99d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:31.131975 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4409,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":9717,"spinlock_wait_cycles":3056449920}
I20260709 12:26:31.127561  6015 pending_rounds.cc:85] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 3
I20260709 12:26:31.145190  6015 pending_rounds.cc:107] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:26:31.146732  6015 raft_consensus.cc:2889] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:31.147439  6766 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 12:26:31.148658  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.149501  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.153122  6756 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:26:31.150503  5915 connection.cc:256] RPC call timeout handler was delayed by -1.1883810120002636s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23610000000000003s for a response
I20260709 12:26:31.157392  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:31.157897  6768 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:31.151436  9942 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.158795  9973 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:31.159461  9942 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:31.164394  9800 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:31.165133  9800 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:31.139599  9853 raft_consensus.cc:3055] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:31.165762  9853 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:31.166774  9853 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:31.166985  9800 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:31.168263  9853 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:31.176054  6023 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:31.177582  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:31.190150  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:31.191120  6768 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:31.208411  6016 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:31.228385  6768 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:31.245638  9846 raft_consensus.cc:697] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:31.247754  9846 consensus_queue.cc:237] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:31.254693  9866 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:26:31.288430  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 269104us
I20260709 12:26:31.292266  5915 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:31.352526  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.369475  9942 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:31.400136  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 328651us
I20260709 12:26:31.403121  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:31.428440  5940 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:26:31.400149  9840 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:31.440371  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.444334  5947 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:31.445720  5868 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:31.454051  6756 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:26:31.454612  6756 pending_rounds.cc:85] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 7
I20260709 12:26:31.454864  6756 pending_rounds.cc:107] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:26:31.455124  6756 raft_consensus.cc:2889] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:31.458328  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:31.439824 10042 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.459542  5945 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:31.459996 10042 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:31.461259 10044 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:31.462524 10042 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:31.430658 10036 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:31.433574  9800 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:26:31.466188  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:31.471258 10030 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:26:31.474114 10035 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:31.463858 10027 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:31.469681 10039 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:26:31.498797  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 220270us
W20260709 12:26:31.503127  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 100113us
I20260709 12:26:31.509543  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:31.500733 10029 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.078s	user 0.001s	sys 0.001s
I20260709 12:26:31.513446  5915 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:31.469295  9853 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:31.514492  5874 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:31.515317  5868 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:31.500478  9894 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:31.469905 10036 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:31.526763  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:31.533488  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.544902  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.545303  5915 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:31.545555  5840 connection.cc:256] RPC call timeout handler was delayed by -3.4925496350001595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
W20260709 12:26:31.545953  5840 connection.cc:256] RPC call timeout handler was delayed by -3.361149634999947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1652s for a response
W20260709 12:26:31.546355  5840 connection.cc:256] RPC call timeout handler was delayed by -3.341349634999915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
W20260709 12:26:31.546767  5840 connection.cc:256] RPC call timeout handler was delayed by -3.341349634999915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
I20260709 12:26:31.510201 10039 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:31.547161  5840 connection.cc:256] RPC call timeout handler was delayed by -3.2342496350001966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
W20260709 12:26:31.551559  5840 connection.cc:256] RPC call timeout handler was delayed by -3.2207496350001747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
I20260709 12:26:31.549890  5945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60934281a651439a87cd8c11702f67d2"
dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:48330
I20260709 12:26:31.552467  5945 raft_consensus.cc:606] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:26:31.552922  5945 raft_consensus.cc:614] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:31.554092  5840 connection.cc:256] RPC call timeout handler was delayed by -3.2207496350001747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
W20260709 12:26:31.554859  5840 connection.cc:256] RPC call timeout handler was delayed by -3.1694496350000918s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
W20260709 12:26:31.555267  5840 connection.cc:256] RPC call timeout handler was delayed by -3.1694496350000918s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
W20260709 12:26:31.555656  5840 connection.cc:256] RPC call timeout handler was delayed by -3.1676496350000889s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
W20260709 12:26:31.556028  5840 connection.cc:256] RPC call timeout handler was delayed by -3.1676496350000889s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
W20260709 12:26:31.556373  5840 connection.cc:256] RPC call timeout handler was delayed by -3.1667496350000874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 12:26:31.556740  5840 connection.cc:256] RPC call timeout handler was delayed by -3.1667496350000874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 12:26:31.557153  5840 connection.cc:256] RPC call timeout handler was delayed by -3.1649496350000845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18700000000000003s for a response
W20260709 12:26:31.557458  5840 connection.cc:256] RPC call timeout handler was delayed by -3.1649496350000845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18700000000000003s for a response
W20260709 12:26:31.556835 10034 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 12:26:31.558491  5840 connection.cc:256] RPC call timeout handler was delayed by -3.1271496350000234s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1912s for a response
W20260709 12:26:31.558861  5840 connection.cc:256] RPC call timeout handler was delayed by -2.8877496350000911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2178s for a response
W20260709 12:26:31.559172  5840 connection.cc:256] RPC call timeout handler was delayed by -2.4494496350002919s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2665s for a response
W20260709 12:26:31.559583  5840 connection.cc:256] RPC call timeout handler was delayed by -2.4494496350002919s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2665s for a response
I20260709 12:26:31.548391  9846 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:31.498509 10035 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:31.560679  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:31.559335 10037 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:26:31.563046 10036 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:31.568163  6609 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:31.582671 10051 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:31.584378 10051 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:31.584990 10051 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:31.586987 10051 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:31.587301 10051 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.571132  9800 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:31.588775  9840 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:31.588466 10051 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:31.613495  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:31.588377 10027 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:31.569823 10048 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:31.619549 10035 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:31.620071 10048 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:31.620676 10048 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:31.626042  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 80817us
I20260709 12:26:31.573766 10032 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:26:31.653684  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:31.628641  5896 debug-util.cc:398] Leaking SignalData structure 0x7b0800298e80 after lost signal to thread 2070
I20260709 12:26:31.575645 10047 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.082s
W20260709 12:26:31.653118 10013 consensus_meta.cc:287] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.008s
W20260709 12:26:31.675944  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 85600us
I20260709 12:26:31.606134  9990 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:26:31.688593 10009 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:31.689941  5840 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:31.691552  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:31.697073  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:31.697443  6015 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:31.678454  9800 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:31.605667 10039 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:31.689795  9840 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:26:31.715965  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:31.716902  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:31.726420  6021 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:31.678854  9846 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:26:31.712189 10047 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:31.680352 10013 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:31.719530 10043 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:31.697734 10036 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:26:31.738077  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:31.720568  9800 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:31.735862 10045 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:31.733273 10047 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:26:31.745136  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 55292.8us
I20260709 12:26:31.750120  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:31.752048  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:31.753031  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:31.752805 10051 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:31.748288 10047 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:31.753911  9990 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:31.754519  9990 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:31.754930  9990 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:31.755491 10051 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:31.756291 10051 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:31.757187 10032 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.757988 10013 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:31.757843 10032 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:31.758324 10051 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:31.760365 10032 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:31.742621  9846 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:31.772414  5840 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:31.769826  9846 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.774351  9846 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:31.782838  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:31.746284 10045 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:31.738932 10056 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:26:31.784101  6021 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:31.743824  9853 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
W20260709 12:26:31.807385  5946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.247s	user 0.005s	sys 0.015s
I20260709 12:26:31.794595  5840 leader_election.cc:400] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 12. Current term is 13.
W20260709 12:26:31.792939  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 170394us
I20260709 12:26:31.867094  5840 leader_election.cc:403] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:31.862150 10036 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:31.868708  5946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.309s	user 0.012s	sys 0.016s
I20260709 12:26:31.871114  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1123) took 1323 ms. Trace:
W20260709 12:26:31.794144 10057 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:26:31.873315  5686 maintenance_manager.cc:419] P 03f70467f9c34de887520aec76068d6f: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033339
I20260709 12:26:31.788280 10032 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.871701  5946 rpcz_store.cc:276] 0709 12:26:30.548043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.558055 (+ 10012us) service_pool.cc:224] Handling call
0709 12:26:30.558504 (+   449us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:30.558973 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.559006 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:30.559770 (+   764us) log.cc:844] Serialized 52 byte log entry
0709 12:26:31.870700 (+1310930us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:31.870747 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.870831 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.870878 (+    47us) raft_consensus.cc:1685] finished
0709 12:26:31.870885 (+     7us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.871046 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":126,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"thread_start_us":1310199,"threads_started":1,"wal-append.queue_time_us":104149,"wal-append.run_cpu_time_us":2479,"wal-append.run_wall_time_us":1006693}
I20260709 12:26:31.805879 10013 raft_consensus.cc:2764] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:26:31.874754  5915 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:31.862541 10059 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:31.818419 10061 raft_consensus.cc:2764] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
W20260709 12:26:31.795050 10058 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:31.848453 10009 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:31.797894 10043 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:31.884595  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.869302 10056 consensus_queue.cc:1243] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:31.904759  9926 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.905452  9926 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:31.787427 10051 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:26:31.907874  9926 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:31.908123 10051 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.884908 10059 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:31.914716 10059 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:31.915477 10059 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 12. Current term is 13.
I20260709 12:26:31.915289 10048 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:31.873380  9894 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:31.921446  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 127122us
I20260709 12:26:31.917109  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:31.893046  9840 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:31.871901 10036 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:31.899942 10047 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:31.958555  5867 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:31.924207  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:31.926874  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:31.876430 10066 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:26:31.909019 10051 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:31.958683  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 58403.6us
W20260709 12:26:31.940183  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.974694  5896 debug-util.cc:398] Leaking SignalData structure 0x7b08002e0b40 after lost signal to thread 5731
I20260709 12:26:31.965772  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:31.975769  5865 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:31.963131  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:31.977972  5874 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:31.962397  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:31.883867 10032 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:31.808643  9800 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:31.932837  9840 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.982754  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:31.983536  9831 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.958479  9846 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:31.927767 10039 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:31.933982  9926 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:31.979143 10061 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:31.993546  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:31.993486 10067 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:32.012811  9980 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.982225 10059 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:32.002386 10047 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:32.005184  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:32.005505  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:32.004901 10036 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:32.001929  5874 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:32.006611 10032 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:32.011860 10039 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:32.008176 10048 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:32.031495 10039 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:31.921000 10009 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:32.011232  9990 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:32.014086 10067 raft_consensus.cc:2764] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:26:32.011595  9831 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:32.014762 10051 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:31.913295 10045 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:32.015952  5840 leader_election.cc:400] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12.
I20260709 12:26:32.054334  5840 leader_election.cc:403] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:31.991053  9840 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:32.026885  6015 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:26:32.026227  6021 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:32.028880  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:32.034819  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:32.035254 10062 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 12:26:32.062042  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:32.043747  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:32.072003  5915 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:32.025317 10059 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:32.073930  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.061280  6017 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:32.026865  9980 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:32.079869  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 64056us
I20260709 12:26:32.084792  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:32.075342 10072 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:32.045946  9926 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:32.089962  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:32.075614 10039 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:32.075284 10059 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:32.079241 10045 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.086423 10032 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:32.097965 10032 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.098520 10032 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:32.083978 10071 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.088721 10061 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12.
I20260709 12:26:32.108043 10071 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:32.114231 10071 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:32.099701 10047 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:32.116001 10047 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:32.116501 10047 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.118666  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:32.102963 10048 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:32.119709  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1420) took 1142 ms. Trace:
I20260709 12:26:32.120128  5948 rpcz_store.cc:276] 0709 12:26:30.977163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.118269 (+1141106us) service_pool.cc:224] Handling call
0709 12:26:32.119639 (+  1370us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:32.122784  6611 connection.cc:256] RPC call timeout handler was delayed by -1.0176977020000777s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
I20260709 12:26:32.123435  9990 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:32.125090  6766 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:32.126315  6766 raft_consensus.cc:606] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:32.126879 10009 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:32.127537  9980 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:32.128257  9846 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:32.130267  9831 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:32.132961  6773 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.658s	user 0.004s	sys 0.007s
I20260709 12:26:32.092924 10039 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:32.137295  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 47471us
W20260709 12:26:32.142769  6773 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.668s	user 0.006s	sys 0.008s
I20260709 12:26:32.144212  9980 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:32.145195  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:32.144412  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:32.148593  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:32.148985  5868 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:32.149345  5943 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:32.145438  9926 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:32.149528  5840 leader_election.cc:400] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12.
I20260709 12:26:32.148651  6023 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:32.151517  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:32.154646  9990 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:32.155633  9990 consensus_queue.cc:1243] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:32.156198  9990 raft_consensus.cc:3055] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:32.156569  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 924) took 2607 ms. Trace:
I20260709 12:26:32.154673  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:32.147562  9918 raft_consensus.cc:2764] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:26:32.158118  5868 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 13: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:32.158586 10051 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:32.160970  6611 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:32.153398  5840 leader_election.cc:403] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:26:32.166224  6023 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:32.157055  6773 rpcz_store.cc:276] 0709 12:26:29.546108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.557972 (+ 11864us) service_pool.cc:224] Handling call
0709 12:26:29.558418 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:29.592013 (+ 33595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.592069 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:29.593284 (+  1215us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.475039 (+881755us) spinlock_profiling.cc:243] Waited 7.88 ms on lock 0x7b1800337300. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:26:32.143370 (+1668331us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:32.143416 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.143505 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.143546 (+    41us) raft_consensus.cc:1685] finished
0709 12:26:32.143556 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.152875 (+  9319us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23286,"raft.queue_time_us":30114,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":18130304,"thread_start_us":1668080,"threads_started":1,"wal-append.queue_time_us":910732,"wal-append.run_cpu_time_us":2399,"wal-append.run_wall_time_us":1022854}
I20260709 12:26:32.174906  9840 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:32.149773 10039 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:32.156591  9990 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:32.190208 10043 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:26:32.093050 10072 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:32.202865 10072 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:32.182713  9894 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:32.191270  9980 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:32.167373 10009 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:26:32.177147  9917 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:26:32.197676 10051 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:26:32.183233  9926 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:26:32.211097  9980 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.240576 10047 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:32.246869 10047 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:32.235348  9980 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:32.258272  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:32.259410  9840 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:32.259591  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1051) took 1323 ms. Trace:
I20260709 12:26:32.260170  6020 rpcz_store.cc:276] 0709 12:26:30.935733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.257810 (+1322077us) service_pool.cc:224] Handling call
0709 12:26:32.259503 (+  1693us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.229182  9990 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:32.269013  9990 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:32.269733  5896 debug-util.cc:398] Leaking SignalData structure 0x7b0800321800 after lost signal to thread 5899
I20260709 12:26:32.270808  5870 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:32.271631  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 724) took 1547 ms. Trace:
I20260709 12:26:32.272027  5870 rpcz_store.cc:276] 0709 12:26:30.724439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.269319 (+1544880us) service_pool.cc:224] Handling call
0709 12:26:32.270341 (+  1022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.271313 (+   972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.271561 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.277354 10059 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:32.279904 10059 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:32.282483  9871 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.283181  9871 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:32.233897  9841 consensus_meta.cc:287] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.670s	user 0.015s	sys 0.008s
I20260709 12:26:32.286046  9871 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:32.288751  9980 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:32.296674  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 662) took 2517 ms. Trace:
I20260709 12:26:32.296831 10032 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:32.308302  5871 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:26:32.297289  5941 rpcz_store.cc:276] 0709 12:26:29.779054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.779269 (+   215us) service_pool.cc:224] Handling call
0709 12:26:29.793092 (+ 13823us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:29.794302 (+  1210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.794356 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:29.803390 (+  9034us) log.cc:844] Serialized 26 byte log entry
0709 12:26:29.958933 (+155543us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:29.982381 (+ 23448us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.982628 (+   247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.982713 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.673518 (+690805us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b18001313e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:26:30.789384 (+115866us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:32.296386 (+1507002us) raft_consensus.cc:1685] finished
0709 12:26:32.296418 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.296596 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":602,"prepare.queue_time_us":23680,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":5167,"raft.queue_time_us":4291,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":7254,"spinlock_wait_cycles":39974400,"thread_start_us":150309,"threads_started":1,"wal-append.queue_time_us":233968,"wal-append.run_cpu_time_us":7744,"wal-append.run_wall_time_us":1761026}
W20260709 12:26:32.297464  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:32.317221 10064 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.419s	user 0.001s	sys 0.001s
I20260709 12:26:32.309576  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 725) took 1584 ms. Trace:
I20260709 12:26:32.318225  5871 rpcz_store.cc:276] 0709 12:26:30.724740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.307298 (+1582558us) service_pool.cc:224] Handling call
0709 12:26:32.307921 (+   623us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.309120 (+  1199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.309488 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.318768 10032 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:32.319370  9860 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:32.319886  9860 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:32.320866  9860 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:32.324052  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 932) took 1143 ms. Trace:
I20260709 12:26:32.326218 10039 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:32.314486  6022 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
W20260709 12:26:32.327862  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 113927us
I20260709 12:26:32.297369  9980 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:32.331993  9842 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.757s	user 0.016s	sys 0.002s
I20260709 12:26:32.334227  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:32.337083  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 125361us
I20260709 12:26:32.387567  6611 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:32.337603  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:32.388635  9917 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:32.388764  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:32.389130  6023 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.389551  5867 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:32.387586  5942 raft_consensus.cc:1217] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Deduplicated request from leader. Original: 11.6->[12.7-12.7]   Dedup: 12.7->[]
I20260709 12:26:32.390868  5942 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:26:32.391487  9917 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:32.391959  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:32.392107  9917 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:32.392601  5942 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:32.338536  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:32.394667  9917 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:32.395308  9917 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:32.396373  9917 raft_consensus.cc:697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:32.339143  5940 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:32.339505  5948 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:26:32.397873  9917 consensus_queue.cc:237] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:32.399058  5948 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:32.339262  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:32.399689  9990 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:32.400345  6015 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:32.340636  5943 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:32.400664 10061 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:32.400461 10059 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:32.401254 10061 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:32.402546  5867 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:32.341691 10085 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.345345  5946 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:32.346382  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:32.412796  5945 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:32.347478  6785 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.348394  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:32.413453  6023 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
W20260709 12:26:32.350389  6831 heartbeater.cc:646] Failed to heartbeat to 127.2.4.254:41049 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.2.4.254:41049 timed out after 15.000s (SENT)
I20260709 12:26:32.358297  9912 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.359485 10084 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.360186 10082 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.361441  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:32.361886  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:32.362174  9853 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.361683  6016 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:32.410537  9860 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:32.365067  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.365523  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:32.366824  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:32.368002  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:32.367965  5840 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:32.368422  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:32.375051  6756 rpcz_store.cc:276] 0709 12:26:31.180359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.180604 (+   245us) service_pool.cc:224] Handling call
0709 12:26:31.181056 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.455637 (+274581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.323728 (+868091us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b5400006754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:32.323984 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626617728}
I20260709 12:26:32.406219  5942 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:32.384150 10051 consensus_queue.cc:1059] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.019s
W20260709 12:26:32.403239  9842 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.828s	user 0.017s	sys 0.003s
I20260709 12:26:32.443660  6021 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:32.374595  9866 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:32.383056  9883 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.401784  9990 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:32.460050  6019 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:32.460464  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 854) took 1539 ms. Trace:
I20260709 12:26:32.364318  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:32.384976 10032 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:26:32.498756  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:32.499797  6017 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:32.503491  6766 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:32.502832  6757 rpcz_store.cc:276] 0709 12:26:30.921478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.361576 (+1440098us) service_pool.cc:224] Handling call
0709 12:26:32.460351 (+ 98775us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.506331  6021 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:32.441406 10027 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:32.410715  9911 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:26:32.508278  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:32.508641 10027 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:32.478615  6762 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:32.510737  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:32.511442  6021 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:32.512133 10027 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:32.512425 10043 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:32.516312  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 128012us
I20260709 12:26:32.424885 10059 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:26:32.438082  9980 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:26:32.522617  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1058) took 1501 ms. Trace:
I20260709 12:26:32.522639  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:32.525300  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:32.527119  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:32.525233  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:32.527741  6017 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:32.531243  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:32.531955  6018 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:32.496973  9894 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:32.533308  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:32.534819  6768 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.533504  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:32.536115  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:32.536811  6018 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:32.458062  9912 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:32.538908  6021 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:26:32.504247  9841 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:32.540215  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 152719us
I20260709 12:26:32.504518  9871 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:32.505053  9990 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:32.545290  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:32.547502  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.546442  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:32.519320 10051 consensus_queue.cc:1243] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:32.478183  9853 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:32.552305  6019 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:32.466128 10082 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:32.562757  6021 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:32.563094  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:32.565888  6611 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.196:34835 timed out after 1.527s (SENT) [suppressed 20 similar messages]
I20260709 12:26:32.516844  9866 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:32.443439 10085 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:32.554030 10035 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:32.572088  6757 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:32.463056 10084 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:32.580788  6017 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
W20260709 12:26:32.574172  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 206323us
I20260709 12:26:32.570012  9840 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.517069  9883 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:32.597728  6768 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:32.593274  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:32.497395 10061 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:32.604156  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:32.608745  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:26:32.612825  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.567656 10047 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:26:32.560731  9871 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:32.568451  9841 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:32.594988  9842 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:32.619421  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:32.606791  9990 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:32.623592  9841 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:32.607476 10009 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:32.633323  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 67486.8us
I20260709 12:26:32.614259 10035 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:32.641832  9853 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:32.643610  9866 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:32.646502  9990 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:32.647594  5945 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:32.651499  9912 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:32.655840  6015 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:32.657114  6016 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:32.660691  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:32.664126  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:32.652627 10039 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:32.676811  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 2.1005e+06us
I20260709 12:26:32.682828  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:32.684171  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:32.684765  6768 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:32.687321  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:32.693095 10039 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.693985 10039 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:32.674474  9944 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:32.700940 10039 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:32.704844  6785 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:32.671021 10035 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:32.715596  5987 connection.cc:256] RPC call timeout handler was delayed by -1.9953474179997102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26070000000000004s for a response
I20260709 12:26:32.626412  9894 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:32.727137  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 118533us
W20260709 12:26:32.732815  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.728754  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:26:32.734405  5987 leader_election.cc:336] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 2.605s (SENT)
I20260709 12:26:32.730756  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:32.738099  6768 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:32.736753  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:32.721262 10009 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:32.727260 10087 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:26:32.716642  9911 raft_consensus.cc:3055] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:32.751511  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:32.727883 10086 raft_consensus.cc:3055] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:32.727398 10044 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12.
I20260709 12:26:32.767712  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:32.768836  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.728116 10093 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:32.773351  6768 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:32.709775 10096 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.140s
I20260709 12:26:32.746629 10009 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:32.754312  9911 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:32.764647 10086 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:32.796351  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 67705.3us
I20260709 12:26:32.774974  9860 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:32.783298 10094 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 12:26:32.779959 10093 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.815179  6768 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:32.780529 10051 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:32.785899 10009 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.817955 10103 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:32.818121  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 83777.1us
I20260709 12:26:32.820094  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:32.825575  5987 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 2.409s (SENT)
I20260709 12:26:32.827096  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:32.829799  5987 connection.cc:256] RPC call timeout handler was delayed by -1.9205486549999478s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21840000000000004s for a response
W20260709 12:26:32.831590  5987 leader_election.cc:336] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 2.424s (SENT)
W20260709 12:26:32.834266  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:32.892498  5987 connection.cc:256] RPC call timeout handler was delayed by -1.6772001689996614s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20270000000000002s for a response
W20260709 12:26:32.893344  5987 connection.cc:256] RPC call timeout handler was delayed by -1.3275082909999583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21880000000000002s for a response
W20260709 12:26:32.893748  5987 connection.cc:256] RPC call timeout handler was delayed by -1.2158729649995621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2237s for a response
W20260709 12:26:32.894384  5987 connection.cc:256] RPC call timeout handler was delayed by -1.0305149939995317s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19710000000000003s for a response
I20260709 12:26:32.924729  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:32.925606  6768 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:32.939869  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:32.942389  6015 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:32.946211  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:32.947042  6015 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:32.821202 10093 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:32.964555  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 435014us
I20260709 12:26:32.940951  9942 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:32.956900  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:32.955924 10094 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:26:32.954401  6768 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
W20260709 12:26:32.970556  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.891954  9442 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:32.944314 10082 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:32.973491  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:32.941761 10110 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:32.890586 10051 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:32.942576  9911 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:33.004663  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:32.953284 10085 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:32.810815 10035 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:32.989521  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:33.009526  6015 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:32.990237  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:32.987061  9871 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:33.003291  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:33.044972  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:33.046077  6756 raft_consensus.cc:2393] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 14: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:33.047169  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:32.523166  6024 rpcz_store.cc:276] 0709 12:26:31.021708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.022004 (+   296us) service_pool.cc:224] Handling call
0709 12:26:31.031988 (+  9984us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:31.032734 (+   746us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.032781 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:31.045465 (+ 12684us) log.cc:844] Serialized 52 byte log entry
0709 12:26:32.357980 (+1312515us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54003028a0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bed090 00007f39d2bfa6b8 00007f39d2c8392b 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c
0709 12:26:32.358078 (+    98us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:32.403578 (+ 45500us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.502240 (+ 98662us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.505730 (+  3490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.505806 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.522362 (+ 16556us) raft_consensus.cc:1685] finished
0709 12:26:32.522393 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.522554 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36212,"prepare.queue_time_us":28812,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":91,"raft.queue_time_us":46026,"raft.run_cpu_time_us":495,"raft.run_wall_time_us":25685,"spinlock_wait_cycles":265866624,"thread_start_us":243839,"threads_started":3,"wal-append.queue_time_us":148730,"wal-append.run_cpu_time_us":3432,"wal-append.run_wall_time_us":1176981}
I20260709 12:26:33.048671  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:32.939783 10103 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:33.012809  9867 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:32.986886  9942 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:33.013347 10100 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.201s	user 0.000s	sys 0.000s
W20260709 12:26:33.018360 10105 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
W20260709 12:26:33.026135 10102 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
W20260709 12:26:33.077229  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 75269.9us
I20260709 12:26:33.032145 10012 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:33.040235  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:33.042564 10051 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:33.043247  9911 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:33.083431  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:33.049317  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:33.049425  6756 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:26:33.053184  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1120) took 2620 ms. Trace:
I20260709 12:26:33.084528  6793 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:33.084712  5944 rpcz_store.cc:276] 0709 12:26:30.432972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.433160 (+   188us) service_pool.cc:224] Handling call
0709 12:26:30.438927 (+  5767us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.439502 (+   575us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.439538 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.440081 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.450461 (+ 10380us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:30.450514 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.450598 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.846956 (+396358us) spinlock_profiling.cc:243] Waited 6.11 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:33.052784 (+2205828us) raft_consensus.cc:1685] finished
0709 12:26:33.052811 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.053111 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1194,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":4383,"spinlock_wait_cycles":14041984,"thread_start_us":9975,"threads_started":1,"wal-append.queue_time_us":15805,"wal-append.run_cpu_time_us":4987,"wal-append.run_wall_time_us":1401064}
W20260709 12:26:33.056324  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 258891us
W20260709 12:26:33.030378 10109 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
W20260709 12:26:33.079106  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.079702  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:33.087971  5840 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:33.091780  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:33.090323  6015 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:26:33.088820  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:33.064008 10110 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 14 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:33.095409  6768 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:33.094576  9831 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:33.099766  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:33.102684  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:33.102938  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:33.104027  6757 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 16: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:33.104118  6015 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:33.105865  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:33.105086 10103 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:33.112428  5896 debug-util.cc:398] Leaking SignalData structure 0x7b0800280300 after lost signal to thread 5973
I20260709 12:26:33.106873 10048 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:33.142251 10048 raft_consensus.cc:697] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:33.142673  9942 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:33.109949  9837 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:33.143349 10048 consensus_queue.cc:237] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:33.119819  6768 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:33.097803  9867 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:33.144941  9867 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:33.145462  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:33.146430  6768 raft_consensus.cc:2368] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16.
I20260709 12:26:33.146343 10110 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:33.147460  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:33.148458  6768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:33.148958  6768 raft_consensus.cc:606] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:33.149294  6768 raft_consensus.cc:614] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:33.141507 10114 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:26:33.150020  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:33.151175  6757 raft_consensus.cc:2410] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 12:26:33.152244  9990 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.003s
I20260709 12:26:33.152706  6757 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:26:33.153162  6757 pending_rounds.cc:85] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 4
I20260709 12:26:33.171314  6757 pending_rounds.cc:107] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 7.5
W20260709 12:26:33.154162 10111 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:33.157473  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 738) took 1281 ms. Trace:
W20260709 12:26:33.141482  5896 debug-util.cc:398] Leaking SignalData structure 0x7b080037a200 after lost signal to thread 5974
I20260709 12:26:33.181270  6015 raft_consensus.cc:1217] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Deduplicated request from leader. Original: 11.6->[12.7-12.7]   Dedup: 12.7->[]
W20260709 12:26:33.181983  5896 debug-util.cc:398] Leaking SignalData structure 0x7b0800093b80 after lost signal to thread 6048
W20260709 12:26:33.182482  5896 debug-util.cc:398] Leaking SignalData structure 0x7b08003ac140 after lost signal to thread 6049
W20260709 12:26:33.180186  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 68905.1us
W20260709 12:26:33.150063  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 67255.5us
I20260709 12:26:33.184237  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:33.185240  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:33.182278  6015 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:26:33.165445  6775 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:33.169876  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.187328  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:33.107879  9871 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:33.106171 10012 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.093854 10083 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 12:26:33.178960  6757 raft_consensus.cc:2889] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:33.153798  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:33.192974  6756 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:33.193110  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:33.198890  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:33.199928  5871 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 8.
I20260709 12:26:33.179896  5865 rpcz_store.cc:276] 0709 12:26:31.875664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.089747 (+214083us) service_pool.cc:224] Handling call
0709 12:26:32.090160 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.090639 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.090667 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.095134 (+  4467us) log.cc:844] Serialized 26 byte log entry
0709 12:26:32.652720 (+557586us) spinlock_profiling.cc:243] Waited 943 us on lock 0x7b18002fc5c0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:26:32.726384 (+ 73664us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:32.726512 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.726780 (+   268us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:33.157135 (+430355us) raft_consensus.cc:1685] finished
0709 12:26:33.157160 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.157396 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":152,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":11448,"spinlock_wait_cycles":3236480,"thread_start_us":73097,"threads_started":1,"wal-append.queue_time_us":667537}
I20260709 12:26:33.175954  9949 raft_consensus.cc:3055] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:33.213678  6775 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:26:33.213693  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:33.222157  6775 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: [9.6-10.7]
I20260709 12:26:33.226228  9860 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:33.210901 10039 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:33.227418 10039 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:33.145530  9867 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:33.230580  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:33.231518  5868 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:33.211181  9942 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:33.234807  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:33.235569  6015 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:33.235648  6775 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:33.211496  9949 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:33.246243  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:33.212392 10051 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:33.267323  6793 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
W20260709 12:26:33.231110 10100 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 12:26:33.271615  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.239418  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.293962  5865 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:33.329150  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:33.331894  5873 raft_consensus.cc:2393] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:33.215324 10098 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
W20260709 12:26:33.336135  9840 consensus_meta.cc:287] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.733s	user 0.010s	sys 0.005s
I20260709 12:26:33.336661  9840 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:33.217700 10012 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:33.339824 10012 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:33.343247 10009 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:33.345117  5871 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:33.346647 10009 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:33.293006  6015 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:33.217015  9831 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:33.294812  6768 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 12:26:33.295425  6793 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:33.296583  9918 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.351224  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:33.243057 10103 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:33.220659  9442 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:26:33.361665  9918 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:33.366695  9918 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:33.361953 10103 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:33.368113  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:33.369370  5865 raft_consensus.cc:2410] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:33.296334 10039 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:33.369975 10103 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:33.376016  6022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 865) took 2806 ms (client timeout 1527 ms). Trace:
W20260709 12:26:33.376560  6022 rpcz_store.cc:269] 0709 12:26:30.569559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.572044 (+  2485us) service_pool.cc:224] Handling call
0709 12:26:33.375773 (+2803729us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b5400464454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:33.375918 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3938304384}
I20260709 12:26:33.387418  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:33.398419  5896 debug-util.cc:398] Leaking SignalData structure 0x7b08003c03e0 after lost signal to thread 6829
W20260709 12:26:33.400655  5896 debug-util.cc:398] Leaking SignalData structure 0x7b08003be720 after lost signal to thread 9960
I20260709 12:26:33.348886 10051 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:33.382754 10112 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.444s	user 0.003s	sys 0.002s
W20260709 12:26:33.408031  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 307094us
W20260709 12:26:33.412317  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 195191us
I20260709 12:26:33.399040  9800 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:26:33.413578  9840 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:33.450582  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:33.453049  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 939) took 1084 ms. Trace:
I20260709 12:26:33.434412 10103 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.460230  5865 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:26:33.460759  5865 pending_rounds.cc:85] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 6
I20260709 12:26:33.461042  5865 pending_rounds.cc:107] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:26:33.452245  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:33.406958 10093 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:33.461488  5865 raft_consensus.cc:2889] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:33.453703  6778 rpcz_store.cc:276] 0709 12:26:32.368296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.368409 (+   113us) service_pool.cc:224] Handling call
0709 12:26:32.368754 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.369186 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.369207 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.381759 (+ 12552us) log.cc:844] Serialized 26 byte log entry
0709 12:26:32.816118 (+434359us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:32.943042 (+126924us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.943291 (+   249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.943392 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:33.382078 (+438686us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:33.438444 (+ 56366us) raft_consensus.cc:1685] finished
0709 12:26:33.452745 (+ 14301us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.452959 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":126912,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":63,"raft.queue_time_us":9837,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":11713,"spinlock_wait_cycles":8046464,"thread_start_us":349133,"threads_started":1,"wal-append.queue_time_us":514033}
I20260709 12:26:33.463078  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:33.458400  9918 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:33.478142  6778 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:33.490676  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:33.458653  9840 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:33.458521 10043 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:33.477677  9918 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.476255 10098 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:33.482175  9922 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:33.481019 10066 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.502441  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:33.501070  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:33.459232 10051 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:33.508302 10066 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:33.510875 10066 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:33.498088 10043 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:33.514144  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.514559 10043 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:33.506886  9840 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:33.516880  9867 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:33.465286 10103 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:33.497743  9918 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:33.522372  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:33.527400  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:33.529367  5865 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:33.498935 10098 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:33.499804  9922 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:33.536005  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 138889us
I20260709 12:26:33.536964  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:33.544262  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:33.546033  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:33.546818  5865 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:33.549564  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:33.538458  5915 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:33.528213 10043 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:33.572683  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:33.573589 10103 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:33.558037 10132 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:33.574244 10132 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:33.574676 10132 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:33.576659 10134 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:33.577243 10134 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:33.577961  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.578312 10123 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:26:33.578066  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:33.579260  5872 raft_consensus.cc:2410] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
W20260709 12:26:33.579952 10121 log.cc:927] Time spent T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:26:33.580506  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:33.564774  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:33.532599 10098 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:33.587061 10093 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:33.586613  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:33.588804  5872 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 16: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:33.579782  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:33.590832  9867 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:33.532351  9922 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:33.590445 10043 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:33.560534  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:33.567139  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:33.611171  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 50770.8us
I20260709 12:26:33.612646 10118 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:33.615422  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:33.616290  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.617830  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:33.613333 10118 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:33.569797  9841 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:33.568070 10013 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:33.610893 10103 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:33.629842 10089 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 1.072s	user 0.003s	sys 0.005s
I20260709 12:26:33.633148 10084 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:33.625204  5869 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:33.637773 10088 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:26:33.638451 10107 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.700s	user 0.000s	sys 0.003s
I20260709 12:26:33.620719 10124 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:33.622639  5874 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:33.663803  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:33.659584  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:33.660992  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:33.628208  9942 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:33.669553  9831 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:33.650228  9860 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.669926 10118 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:33.670589  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:33.656731 10088 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:33.670683  9860 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:33.674146 10088 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:33.675531  9923 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.642589 10086 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:33.620548  9867 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:26:33.676103  9923 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:33.679325  9923 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:33.680076  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1061) took 1181 ms. Trace:
I20260709 12:26:33.680212 10132 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:33.680724  6016 rpcz_store.cc:276] 0709 12:26:32.498462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.500144 (+  1682us) service_pool.cc:224] Handling call
0709 12:26:32.656787 (+156643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.657778 (+   991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.679826 (+1022048us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54003638d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:33.679998 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":359410048}
I20260709 12:26:33.665580  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:33.682824  5943 raft_consensus.cc:2410] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:26:33.649972 10103 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:33.686753 10103 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:33.676275  9860 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:33.687752 10043 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:33.676594 10086 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:33.676805  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.692993 10134 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:33.698261  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:33.699496  5946 raft_consensus.cc:2410] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:26:33.698385  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:33.699837  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:33.701014  6022 raft_consensus.cc:2410] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 12:26:33.702101  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 45632.4us
I20260709 12:26:33.703080  5838 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:33.704008  6020 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:33.709009  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:33.712466  6015 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:33.716795  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:33.717381  6611 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:33.718014  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:33.719154  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:33.719867  6766 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:33.719065  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:33.729157  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:33.731446  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:33.731709  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:33.733695  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:33.730276  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:33.738794  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.742475  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:33.742514  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:33.744279  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:33.668745  9840 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.744273  6024 raft_consensus.cc:2410] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:26:33.746272  9840 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:33.749042  9840 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:33.735858  9853 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:26:33.749780  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.720958 10066 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:33.707965 10043 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:33.760671  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:33.764031  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:33.766870  5865 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:33.750087  9953 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:33.752979  9918 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:33.771951  9840 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.772706  9840 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:33.773221 10116 raft_consensus.cc:3055] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:33.773856 10116 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:33.774842 10116 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:33.775028  9840 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:33.775484  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:33.776211 10116 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:33.778690  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:33.781544  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:33.778223  9840 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:33.778134  9923 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:33.788610  9840 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:33.788805  9923 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:33.789695  5944 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:26:33.790854  9911 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.700s	user 0.013s	sys 0.000s
I20260709 12:26:33.793881  6757 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:33.794193  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:33.795511  5944 raft_consensus.cc:2410] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:26:33.810369  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:33.810850  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 107799us
I20260709 12:26:33.780757  9953 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.817029  5949 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:26:33.817487  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:33.762841 10043 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:33.827162  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 79746.4us
I20260709 12:26:33.820369 10155 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:33.843025  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:33.844683  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 306289us
I20260709 12:26:33.814852 10075 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.820525  9953 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:33.846192 10147 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:26:33.856014  6611 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:33.874320  9917 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:33.849172  5915 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:33.959475  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 110390us
W20260709 12:26:33.911486  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 96153.5us
I20260709 12:26:33.913735  5870 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:33.928277  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.891510 10138 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
W20260709 12:26:33.947548  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 91612.6us
I20260709 12:26:33.924376 10153 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:26:33.855082 10075 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:33.961248  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:33.925580  9922 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:33.984004  5947 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:26:33.984422  5947 pending_rounds.cc:85] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 6
I20260709 12:26:33.984752  5947 pending_rounds.cc:107] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:26:33.985018  5947 raft_consensus.cc:2889] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:33.923810  9911 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:33.982128 10092 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:33.982679 10148 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:26:33.980827  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:33.927845  9918 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:33.983263 10075 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:33.988987  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.971143 10149 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:26:33.971793 10152 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 12:26:33.999701  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.000274  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:33.989886  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 78197.7us
I20260709 12:26:33.892652 10155 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:33.992242  6775 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d001526e46b24f429f57c91e4a270036"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:33.994475  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:34.001581  6775 raft_consensus.cc:606] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:34.001614  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:34.002050  6775 raft_consensus.cc:614] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:34.003988  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:26:33.971761 10043 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:33.986580 10134 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:34.011013 10075 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:34.021023  9922 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:33.997241  9911 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:34.024303  9911 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:33.969818 10156 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.148s
I20260709 12:26:34.025377 10156 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:34.025786 10156 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:34.026744 10156 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:34.027364 10132 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:34.028234 10132 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:34.028939 10156 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:33.989225 10118 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:34.029793 10146 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:34.030412 10146 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:34.031575  9883 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:34.033640 10159 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:26:34.012377 10155 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:34.036180  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.030133 10118 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:34.031386 10146 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:33.960381 10098 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:34.010778  9840 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:34.055333  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 52385.9us
I20260709 12:26:34.055787 10175 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:34.056667  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.056233  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 67634.8us
I20260709 12:26:34.037109 10035 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:34.066730  5943 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:34.018757 10134 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:34.070369 10155 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:34.070701 10134 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:34.063755 10096 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.252s	user 0.002s	sys 0.005s
I20260709 12:26:34.084295  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:34.096999  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:34.058262 10075 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:34.057031 10175 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:34.162194  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 99429.1us
W20260709 12:26:34.179612  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.110994 10096 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.288s	user 0.002s	sys 0.005s
I20260709 12:26:34.198446 10075 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:34.188614  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 436965us
I20260709 12:26:34.190560 10170 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:34.192135  9953 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:34.194734  5943 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:34.206977  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:34.222738  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.223021  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:34.226486  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:34.217361 10096 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:34.235929  5840 leader_election.cc:400] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16.
I20260709 12:26:34.237898  5840 leader_election.cc:403] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:26:34.235880  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:34.205088 10103 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:26:34.224689 10118 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:34.235939 10162 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:34.248361  5943 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:34.235085 10096 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.232748 10084 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.223840 10032 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:26:34.245488 10173 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16.
I20260709 12:26:34.211254 10170 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:34.247283  5840 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:34.231138 10155 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:26:34.225926 10154 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:26:34.281139 10096 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:34.281785 10059 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.335816  5870 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:34.340000  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 121639us
I20260709 12:26:34.284333 10162 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:34.343667  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:34.331507 10013 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:34.290422 10084 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:34.362883  6773 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:26:34.355160  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 108097us
I20260709 12:26:34.386396  5840 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:34.388232  5840 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:34.363989  6785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 919) took 4863 ms (client timeout 2021 ms). Trace:
I20260709 12:26:34.342830 10098 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:34.364832 10150 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.439s	user 0.009s	sys 0.015s
I20260709 12:26:34.374696  9953 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:34.339002 10032 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:26:34.391575  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 48011.1us
W20260709 12:26:34.389256  6785 rpcz_store.cc:269] 0709 12:26:29.495168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.495339 (+   171us) service_pool.cc:224] Handling call
0709 12:26:34.358516 (+4863177us) spinlock_profiling.cc:243] Waited 2.83 s on lock 0x7b5400007b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:34.358729 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6520345344}
I20260709 12:26:34.394973  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:34.396242 10155 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:34.402362  5840 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:34.400391  5870 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:34.400426 10118 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:34.343971 10059 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:34.387476 10167 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.396322 10181 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:34.396162 10013 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:34.412173  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.414420  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 87986.9us
I20260709 12:26:34.401530 10032 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:34.387746 10134 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:34.408886 10192 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:34.427455  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:34.430104  5869 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 12:26:34.425987  5915 leader_election.cc:336] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 2.021s (SENT)
I20260709 12:26:34.437009  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:34.438931  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:34.442721  9841 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:34.443420  9841 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:34.447314 10084 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:34.448478 10032 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:34.450035 10190 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:34.454195 10032 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:34.454325 10059 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:34.435266 10162 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:34.456249  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:34.418543 10013 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:34.458132 10013 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:34.460379 10134 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:34.461536 10116 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.685s	user 0.013s	sys 0.005s
I20260709 12:26:34.423180 10154 consensus_queue.cc:1243] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:34.488935  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:34.490316  5949 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:34.418550 10167 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:34.432163 10187 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
W20260709 12:26:34.556257  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 154506us
I20260709 12:26:34.558745  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:34.562376  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:34.497493 10134 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.058s
W20260709 12:26:34.587985  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.574743 10154 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:34.592172  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.578095 10116 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.569185 10196 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:34.583364 10191 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:34.603098 10194 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:26:34.640494  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 72861.1us
I20260709 12:26:34.643194  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:34.619437 10154 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:26:34.645853 10196 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.004s
W20260709 12:26:34.651649 10184 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:26:34.652503 10196 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:34.662961 10096 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:34.669083  5915 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:34.672171  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:34.710037  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:34.711694  5940 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:34.709187 10096 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:34.672863  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:34.615813 10191 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:34.715682 10191 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:34.678068 10199 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.083s	user 0.001s	sys 0.001s
I20260709 12:26:34.673292  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:34.718485  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:34.719396  9918 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:34.719673  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:34.720309  5873 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:34.720094  9918 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:34.672948  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:34.721799  9911 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.722352  9911 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:34.722735  5865 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:34.684665 10154 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.688530  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:34.689152  6778 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:34.724634 10196 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:34.673522  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 865) took 2125 ms. Trace:
I20260709 12:26:34.690140  6785 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:34.725226  9918 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:34.725960  9911 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:34.726996  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:34.728549  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:34.729403  6785 raft_consensus.cc:2393] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 14: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:34.691316 10088 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:34.696108  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.748169  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.699649  9923 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:34.702730  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1051) took 2201 ms. Trace:
I20260709 12:26:34.752244  6757 rpcz_store.cc:276] 0709 12:26:32.501546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.215288 (+713742us) service_pool.cc:224] Handling call
0709 12:26:33.226509 (+ 11221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.794317 (+567808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.702432 (+908115us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b54005758d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:34.702658 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1303569152}
I20260709 12:26:34.705781  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 882) took 2135 ms. Trace:
I20260709 12:26:34.709282  5948 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:34.711483  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:34.712777 10187 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:34.713529  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:34.725067  6775 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:26:34.690877  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:34.691752  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:34.729429  6609 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:34.730268  5990 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:34.753744  6018 rpcz_store.cc:276] 0709 12:26:32.570579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.573897 (+  3318us) service_pool.cc:224] Handling call
0709 12:26:32.575498 (+  1601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.628685 (+ 53187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.628735 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.636164 (+  7429us) log.cc:844] Serialized 26 byte log entry
0709 12:26:33.079856 (+443692us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:33.079901 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.705315 (+1625414us) spinlock_profiling.cc:243] Waited 50.1 ms on lock 0x7b54001716d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:34.705371 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.705421 (+    50us) raft_consensus.cc:1685] finished
0709 12:26:34.705434 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.705689 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2484,"raft.queue_time_us":3255,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":5133,"spinlock_wait_cycles":115279104,"thread_start_us":404566,"threads_started":1,"wal-append.queue_time_us":562671,"wal-append.run_cpu_time_us":2712,"wal-append.run_wall_time_us":1189408}
I20260709 12:26:34.732532  9871 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:34.617213 10116 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:34.733664  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:26:34.682452 10198 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:26:34.693491  9883 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:34.733176  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:34.725860  6768 rpcz_store.cc:276] 0709 12:26:32.548149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.230365 (+682216us) service_pool.cc:224] Handling call
0709 12:26:33.294468 (+ 64103us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.349915 (+ 55447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.673311 (+1323396us) spinlock_profiling.cc:243] Waited 58.7 ms on lock 0x7b54005294d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:34.673460 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134965632}
I20260709 12:26:34.738426  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:34.739737  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:34.726996  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:34.759016  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 47613.7us
I20260709 12:26:34.761097  6762 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:34.762754  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:34.763523  6785 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:34.767681  6775 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:34.775693  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:34.779747  6757 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:34.725798 10196 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:34.781294 10174 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:34.769177  9883 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:34.783411 10196 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:34.783324  9883 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:34.784708  9883 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:34.785913  9953 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:34.786656 10207 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:34.787313 10207 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:34.788616  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:34.790771  5868 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:34.791601 10116 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:34.767678  9911 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:34.769687 10013 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:34.724738 10154 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:34.769887  9871 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:34.795006 10013 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:34.795610 10154 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:34.796226  9871 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:34.796859  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:34.796993  9871 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.764325 10075 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:34.799502  9841 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:34.798295  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:34.802474  5873 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:26:34.777958  9918 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:34.820055  6778 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:34.821967  6757 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:34.820400  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:34.826807 10170 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:34.826973  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:34.809437 10115 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:34.827859  6778 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:34.828619  5873 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 12:26:34.830077  5868 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:34.766930 10177 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:26:34.830297  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:34.835280  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:34.809679  9923 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:26:34.847062  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:34.811414 10196 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:34.845928  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:34.821426 10110 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.595s	user 0.008s	sys 0.002s
I20260709 12:26:34.848186  6778 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:34.820379 10207 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:34.852550  9918 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:34.829438  9926 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:34.863091  9841 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:34.863282  9953 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:34.864071  9922 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:34.867224 10115 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:34.870191  6757 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:34.877005 10177 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:34.801659  9871 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:34.817840  9894 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:26:34.852958 10156 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:26:34.887319 10110 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.661s	user 0.008s	sys 0.002s
I20260709 12:26:34.910748  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:34.912714  9918 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:34.965854  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 235678us
I20260709 12:26:34.944805  9841 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.966770  5990 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:34.949523  9842 raft_consensus.cc:3055] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:34.901787 10177 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:34.965179 10207 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:34.969708 10067 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:34.956630 10116 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:26:34.988147 10207 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:34.991600 10207 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:34.992472 10126 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 1.440s	user 0.006s	sys 0.000s
I20260709 12:26:34.965085 10209 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.985386 10067 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:34.971911  9842 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:34.970021  9841 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:35.015560 10170 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:34.999006  9918 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:35.014523  9871 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:35.017355  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1053) took 2502 ms. Trace:
I20260709 12:26:35.011870 10177 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:35.012458 10115 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:35.022264  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 55581.8us
I20260709 12:26:35.022955 10215 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:35.023622  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:35.025429  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:35.026814  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:35.028769  6024 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:35.029145  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:35.029594  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:35.030962  6017 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:35.012475 10167 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:35.031454  9883 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:35.031945  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 750) took 1825 ms. Trace:
I20260709 12:26:35.032737 10207 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.033609  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:35.008316 10195 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.490s	user 0.000s	sys 0.002s
W20260709 12:26:35.036346  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 303550us
I20260709 12:26:35.042037  5874 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:35.074363  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:35.075165  6018 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:35.045324  6756 rpcz_store.cc:276] 0709 12:26:32.515165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.214544 (+699379us) service_pool.cc:224] Handling call
0709 12:26:33.237680 (+ 23136us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.248220 (+ 10540us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:33.248275 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:33.248899 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 12:26:33.432319 (+183420us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:33.432386 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.432497 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.986023 (+1553526us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:35.016922 (+ 30899us) raft_consensus.cc:1685] finished
0709 12:26:35.016946 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:35.017273 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9810,"raft.queue_time_us":10140,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":24708,"spinlock_wait_cycles":55367936,"thread_start_us":182962,"threads_started":1,"wal-append.queue_time_us":269531}
W20260709 12:26:35.048460  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 90772.8us
I20260709 12:26:35.076269  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:35.043522  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:35.031276  6021 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:35.061772  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:35.078441  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:35.065325  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:35.080943  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:35.081270  6015 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:35.066592  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:35.079885  6017 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:35.072024  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:35.064122  5871 rpcz_store.cc:276] 0709 12:26:33.206498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.206713 (+   215us) service_pool.cc:224] Handling call
0709 12:26:33.207347 (+   634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.345616 (+138269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.031700 (+1686084us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54003890d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:35.031885 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":316225920}
I20260709 12:26:35.086606  5990 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:35.088014  5867 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:35.090952  6019 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:35.070178  9883 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.092962  6775 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:35.097200  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:35.098349  6778 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:35.099272  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:35.094062  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:35.112965  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:35.114146  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:35.115260  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:35.115588  6762 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:35.107687  5867 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:35.118527  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:35.108817  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 255949us
I20260709 12:26:35.108426  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:35.113873  5873 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:35.070262 10012 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:35.040894 10209 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:35.049373 10170 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:35.106472  6778 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:35.146842  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:35.148290  6775 raft_consensus.cc:2393] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:35.101873  9883 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:35.102306  9840 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:35.155253 10012 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:35.109412  9842 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:35.161059 10012 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:35.162005 10164 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.242s	user 0.000s	sys 0.005s
I20260709 12:26:35.103739 10216 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:35.162827 10216 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.163264 10216 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:35.164168 10217 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:26:35.102718 10196 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:26:35.116894  9871 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:26:35.165921 10219 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:35.167493 10209 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:35.116894 10067 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:35.168236 10209 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:35.168654 10067 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:35.108063 10115 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:35.171360  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:35.178537  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:35.092299 10207 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:35.107355 10134 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:26:35.189072  6015 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:35.156013  9840 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:35.062527 10215 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:35.189045  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 49311.8us
I20260709 12:26:35.161291  9841 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:35.186422  9911 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:35.194146  6611 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:35.191443 10098 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.214807  6023 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:35.152743 10189 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:26:35.191519  9926 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:35.175419 10209 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:26:35.175083 10132 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:26:35.182893 10156 consensus_queue.cc:1059] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:26:35.188253 10115 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:35.217340  6022 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:35.215472  6611 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:35.226060  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 121169us
I20260709 12:26:35.194183 10221 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:35.242018  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 149341us
I20260709 12:26:35.217950 10084 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:35.231838 10132 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:35.201543  9840 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:35.258355  6611 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:35.211251  9911 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:35.244105 10196 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:26:35.267948  9894 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:35.218305 10098 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:35.264959  9953 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:35.262322 10132 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:35.265548  9911 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:35.348933  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 145249us
I20260709 12:26:35.420214  5913 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:26:35.355792  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 114408us
W20260709 12:26:35.358489  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 236381us
I20260709 12:26:35.424503  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:35.426096  5990 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:35.372372  6020 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:34.768453  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:35.241314  9926 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:35.369822  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:26:35.471441  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 102144us
I20260709 12:26:35.262147 10084 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:35.304083 10132 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:35.512666 10132 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:35.303860 10098 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:35.298834 10115 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:35.514204 10187 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:35.515177 10187 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.309043 10175 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:35.286422 10207 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:35.494657  9871 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 12:26:35.283622 10215 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:35.302922 10216 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:35.281910 10212 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 12:26:35.543927  9926 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:35.566205  9926 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:35.297564 10213 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:26:35.567334 10216 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:35.567198  9926 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:35.570852  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 954) took 2969 ms. Trace:
I20260709 12:26:35.571237 10229 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:35.260424 10221 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:35.572119 10229 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.572754 10229 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:35.479879  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:35.266577 10170 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:35.523553  6022 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:35.281462  9953 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.576112  9953 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:35.544555 10241 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:35.545228 10196 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
W20260709 12:26:35.581780  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.582140  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.582916  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.519372 10187 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:35.533655 10189 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:35.539342 10084 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:35.563297 10228 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:35.506755 10039 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 12:26:35.533285 10158 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:35.587112  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 162369us
W20260709 12:26:35.597954  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 168477us
I20260709 12:26:35.603524  9800 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:35.604233  9800 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.604723  9800 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:35.604377 10134 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:35.571705  6793 rpcz_store.cc:276] 0709 12:26:32.601246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.350746 (+749500us) service_pool.cc:224] Handling call
0709 12:26:33.351243 (+   497us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:33.351767 (+   524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.351807 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:34.697324 (+1345517us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b18004194a0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb642a6 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a
0709 12:26:34.698020 (+   696us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b2c0001ffa8. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:26:34.698749 (+   729us) log.cc:844] Serialized 52 byte log entry
0709 12:26:34.902280 (+203531us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:34.902343 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.902474 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:35.569850 (+667376us) raft_consensus.cc:1685] finished
0709 12:26:35.569914 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:35.570388 (+   474us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2407,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":3498,"spinlock_wait_cycles":15444096,"thread_start_us":203022,"threads_started":1,"wal-append.queue_time_us":335457}
I20260709 12:26:35.607816 10075 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:35.608330 10075 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:35.610113 10075 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:35.613363 10075 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:35.617946 10189 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:35.621490 10221 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:35.628198 10241 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:35.630290 10241 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:35.632390 10175 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.635253 10196 raft_consensus.cc:3055] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:35.635913 10196 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:35.636194  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 934) took 3273 ms. Trace:
I20260709 12:26:35.636827  6773 rpcz_store.cc:276] 0709 12:26:32.363026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.363298 (+   272us) service_pool.cc:224] Handling call
0709 12:26:34.358723 (+1995425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.385462 (+ 26739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.635600 (+1250138us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b5400007b50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:35.635945 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4588024192}
I20260709 12:26:35.588057 10228 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.643354 10228 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:35.646349 10158 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:35.650494 10214 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.508s	user 0.004s	sys 0.000s
I20260709 12:26:35.651907 10085 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:35.652716 10085 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:35.656828 10061 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:35.659906 10215 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.661042 10215 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:35.664350 10215 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:35.666827 10189 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:35.668488  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 498766us
W20260709 12:26:35.671717 10211 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.568s	user 0.001s	sys 0.000s
I20260709 12:26:35.678296  9842 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:35.679311 10176 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.680202 10176 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:35.687127  5945 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:35.692090  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 91816.1us
I20260709 12:26:35.697701  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:35.698033  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:35.699440  5945 raft_consensus.cc:2410] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:26:35.702199  9949 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:35.709108  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:35.633445 10175 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:35.707620  9949 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:35.731629  9894 consensus_queue.cc:1175] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" responder_term: 11 status { last_received { term: 10 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:35.732662  9912 raft_consensus.cc:3055] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:35.735054  5946 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 12:26:35.735565  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:35.744010  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 56002.7us
I20260709 12:26:35.744172  9912 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:35.751924  9912 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:35.763520  9883 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.596s	user 0.003s	sys 0.024s
I20260709 12:26:35.764163  9883 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:35.764578 10085 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:35.766981 10035 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:35.690001 10196 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:35.767570 10035 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:35.773792 10085 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:35.782567  9912 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:35.783543  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:35.790016  9926 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:35.794193  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 535801us
I20260709 12:26:35.798622 10035 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:35.805226 10032 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:26:35.805968  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.807808 10175 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:35.808027  9883 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:35.810871 10084 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:35.812513  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:35.838294  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.773368 10158 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:35.832741  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:35.824242  9991 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:35.847381 10216 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:35.847725  9991 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:35.848631 10216 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:35.850023  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.851058  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:35.852178  6773 raft_consensus.cc:2368] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13.
I20260709 12:26:35.852675 10048 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:35.853713 10048 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:35.854369 10216 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:35.835235 10176 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:35.856513 10048 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:35.859125  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:35.834053 10032 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:35.862802 10032 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:35.864409 10032 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:35.861361  6019 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 10: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:35.868033 10084 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:35.870105  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:35.838635  9953 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:35.871515  5870 raft_consensus.cc:2410] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:26:35.875257  9991 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:35.877703  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1409) took 1092 ms. Trace:
I20260709 12:26:35.878444  5872 rpcz_store.cc:276] 0709 12:26:34.632639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.664883 (+ 32244us) service_pool.cc:224] Handling call
0709 12:26:34.708442 (+ 43559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:34.710170 (+  1728us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:34.710518 (+   348us) log.cc:844] Serialized 14 byte log entry
0709 12:26:35.041822 (+331304us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:35.042434 (+   612us) log.cc:844] Serialized 26 byte log entry
0709 12:26:35.042575 (+   141us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:35.042644 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.042718 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:35.696852 (+654134us) raft_consensus.cc:1685] finished
0709 12:26:35.700912 (+  4060us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:35.724956 (+ 24044us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5181,"prepare.queue_time_us":24327,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":5754,"raft.queue_time_us":10644,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":10861,"thread_start_us":325308,"threads_started":1,"wal-append.queue_time_us":486350}
I20260709 12:26:35.882963  6022 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:35.883338 10115 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.884382 10115 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:35.885654  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:35.885800  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:35.885123  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:35.890142  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.886616  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:35.873339  9953 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:35.907087  6793 raft_consensus.cc:2376] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12. Re-sending same reply.
I20260709 12:26:35.908207  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:35.908295  6773 raft_consensus.cc:2393] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 14: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:35.842803  9800 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:35.913987 10113 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.914696 10113 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:35.916031  9800 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:35.918915 10115 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:35.920296  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:35.922487  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:35.893878  9883 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.904261  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:35.934116  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:35.934850  5872 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:35.946547  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:35.943602  9883 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:35.959040  5948 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:35.966778  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:35.960785  6793 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:35.967545  6793 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:35.961344  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:35.971742  6773 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:26:35.981951  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 155140us
W20260709 12:26:35.983018  9840 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.714s	user 0.015s	sys 0.003s
I20260709 12:26:35.983609  9840 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:35.984567 10061 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:35.986523  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.987547  5943 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:35.965986 10035 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:35.985186 10061 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:36.010039  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:36.011060 10061 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:35.982018  6793 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:36.012315  6793 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:36.012832 10113 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:36.008484  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:36.011024  6016 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:36.017409  6020 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:26:36.027812 10067 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.811s	user 0.021s	sys 0.006s
I20260709 12:26:36.028569  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:36.029536  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:36.031409  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:36.033784  6766 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:36.034204  6015 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:36.035581  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:36.036023  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.029711  6020 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:36.038179  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:36.053325  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:35.973060 10228 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:36.056366 10228 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:36.060731  6793 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:36.061875  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:36.062343  6020 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:36.037619  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:36.064380  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:36.072288  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.039039  9894 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.046118  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.074105  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.090772  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:36.110491  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:36.041133 10158 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:36.113739  6022 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 10: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:36.122566  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.066419  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:36.037595 10229 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:36.128794  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:36.139808  5945 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:36.110271  9840 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:36.141435  6015 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:36.158764  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:36.134559  9911 consensus_meta.cc:287] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.518s	user 0.013s	sys 0.007s
I20260709 12:26:36.164281  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:36.175532  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:36.173949  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:36.115123  9894 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:36.179793  6021 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:26:36.176421  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:36.182207  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.157728  9842 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:36.186699  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.180780  6609 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:36.188303  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:36.184617  6016 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:36.193811  5866 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:36.198956  6023 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:26:36.206328  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 158398us
W20260709 12:26:36.207355  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.203142  6018 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:36.219053  5869 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:26:36.219353 10229 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:36.223912  5840 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:36.225383  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:36.154858 10158 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:36.230521  5870 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:36.235702  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.246145  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.246219  5940 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
W20260709 12:26:36.249965  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 75201us
I20260709 12:26:36.170533  9911 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:36.225392  9883 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:36.265357  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.265926  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:36.272208  9842 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:36.278358  9842 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:36.281983  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.282511  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:36.239917 10243 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:36.234544 10067 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:36.283526  5870 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:26:36.287520  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 124557us
I20260709 12:26:36.292837  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:36.301065  5866 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 1388) took 2759 ms (client timeout 2204 ms). Trace:
W20260709 12:26:36.309787  5866 rpcz_store.cc:269] 0709 12:26:33.541309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.579450 (+ 38141us) service_pool.cc:224] Handling call
0709 12:26:36.300769 (+2721319us) spinlock_profiling.cc:243] Waited 2.33 s on lock 0x7b540083f254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:36.300972 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5352544512}
I20260709 12:26:36.310834  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 902) took 1278 ms. Trace:
I20260709 12:26:36.311424  6024 rpcz_store.cc:276] 0709 12:26:35.031735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.092770 (+ 61035us) service_pool.cc:224] Handling call
0709 12:26:35.102203 (+  9433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:35.238916 (+136713us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:35.375057 (+136141us) log.cc:844] Serialized 14 byte log entry
0709 12:26:36.003395 (+628338us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:36.156111 (+152716us) log.cc:844] Serialized 26 byte log entry
0709 12:26:36.166930 (+ 10819us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:36.166994 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:36.181063 (+ 14069us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:36.310335 (+129272us) raft_consensus.cc:1685] finished
0709 12:26:36.310361 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:36.310577 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50626,"prepare.queue_time_us":230723,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":56,"raft.queue_time_us":138416,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":28825,"thread_start_us":656189,"threads_started":3,"wal-append.queue_time_us":110550}
I20260709 12:26:36.313179  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.314339  5946 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:36.303591 10158 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:36.304545  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:36.305315 10235 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.317042  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.317742 10235 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:36.318607  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:36.319469  5944 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:36.320307 10235 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:36.320611  5946 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 12:26:36.321331 10187 consensus_meta.cc:287] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.721s	user 0.008s	sys 0.004s
I20260709 12:26:36.301159  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:36.308847  5871 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:36.309031 10115 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:36.305959  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:36.306262 10067 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:36.368448 10067 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:36.332145  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.322103  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.370945  5943 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:36.371196 10067 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:36.371340  5946 raft_consensus.cc:2410] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:26:36.373051  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 45152.7us
I20260709 12:26:36.374836 10229 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:36.378413  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:36.382583  9911 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:36.383678  5940 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:36.328411  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:36.330760  6766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 883) took 1253 ms (client timeout 1585 ms). Trace:
W20260709 12:26:36.387203  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 82915us
I20260709 12:26:36.338058 10243 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:36.334123  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 110381us
I20260709 12:26:36.356055 10196 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:36.399349  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:36.399865  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:36.357250  9894 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:36.393924 10254 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:26:36.363018  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:36.402402  5873 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:36.365612 10245 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.366899  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.367596 10154 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:36.387849  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 70447.9us
I20260709 12:26:36.377178 10229 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:26:36.407287  5948 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
W20260709 12:26:36.391449  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 103910us
I20260709 12:26:36.386444  5915 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:36.384537  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.382118  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:36.387645  9918 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:36.388064  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:36.398671 10048 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:26:36.390820  6766 rpcz_store.cc:269] 0709 12:26:35.076921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.077097 (+   176us) service_pool.cc:224] Handling call
0709 12:26:36.330440 (+1253343us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b5400470c54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:36.330622 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2035868544}
I20260709 12:26:36.406592  5871 pending_rounds.cc:85] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 3
I20260709 12:26:36.385299 10032 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:36.389249 10084 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.418787  5872 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:36.418386 10154 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:36.418994 10084 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:36.420209  5870 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:36.423292  9894 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:36.424225  9894 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:36.428251  5944 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:36.415503  5871 pending_rounds.cc:107] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 12:26:36.435150  5871 raft_consensus.cc:2889] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:36.421380 10084 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:36.441282  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:36.406173 10245 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:36.421450 10154 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:36.435778  9800 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:36.466964  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:36.397262 10187 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:36.486229  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:36.487476  6793 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:36.462318  9918 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:26:36.496177  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 93292.1us
W20260709 12:26:36.491236 10256 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.137s	user 0.001s	sys 0.001s
I20260709 12:26:36.490916  5694 catalog_manager.cc:5697] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 6 to 8, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 8 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:36.493341 10227 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:36.503468  5949 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 12:26:36.503438  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 117143us
I20260709 12:26:36.504699  5941 raft_consensus.cc:2410] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:26:36.446425 10235 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:26:36.500869  5694 catalog_manager.cc:5697] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:36.502214  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:36.515596  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:36.524674  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 136721us
I20260709 12:26:36.502275 10084 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.529330  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:36.542039 10251 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:26:36.543342 10245 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:36.544216  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:36.544999  5867 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:36.508796 10085 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:36.557354 10219 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.558039 10219 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:36.565838  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 53768.6us
I20260709 12:26:36.568475  5838 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:36.570807  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:36.571764  6020 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:36.578404  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:36.578719  9894 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:36.582849  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:36.583854  6019 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:36.585832 10243 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:36.585884 10229 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:36.586704  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:36.588599 10187 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:36.588721  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:36.589525 10219 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:36.589687  6022 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:36.520488  9918 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:36.590621 10262 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:36.587137 10229 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:36.553889  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:36.595584  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:36.596789  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 160001us
I20260709 12:26:36.601204  5873 raft_consensus.cc:2410] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:26:36.606004  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:36.608107  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:36.595163  6773 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:36.613029  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:36.617785  6016 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:36.620690  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:36.535739  9949 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:26:36.596590 10113 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:36.594493 10229 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:36.622817 10132 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.109s	user 0.015s	sys 0.008s
I20260709 12:26:36.637138  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:36.555619 10084 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:36.641017  6773 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:36.652112  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:36.654524  5838 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:36.635720 10113 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:36.669708  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:36.670130  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 64262us
I20260709 12:26:36.669814 10113 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:36.671907  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:36.674643  5867 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
W20260709 12:26:36.680742  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:36.682168  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 87610.1us
W20260709 12:26:36.678912  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 57478.5us
I20260709 12:26:36.650926  9918 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:36.690191  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:36.654032 10175 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.642333 10219 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:36.697386  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.687783 10061 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:36.698412  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:36.698494  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.695900  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:36.699254  6778 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:36.699751 10067 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:36.703418  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:36.705050  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:36.695662  6022 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:36.709352  6773 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:36.715989  5987 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:36.712548 10260 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 12:26:36.723299  6762 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:26:36.724193  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 90999.7us
I20260709 12:26:36.696720 10219 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.727522  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:36.730162  6773 raft_consensus.cc:2410] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:26:36.730317  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:36.714704 10243 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:36.719805 10048 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.724309 10035 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:36.696888 10175 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:36.731259 10219 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:36.745867  6778 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:36.748759  6773 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:36.750734  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56892.6us
I20260709 12:26:36.753804  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:36.739640 10245 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:36.755874  6762 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:36.743968 10048 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:36.757910 10084 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:36.776219  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:36.761221  9800 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:36.763897 10187 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:36.764765  5840 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:36.765005  5915 leader_election.cc:400] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13.
I20260709 12:26:36.765228 10075 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:26:36.758783  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:36.750836 10035 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:36.786532  6766 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:36.767501  9883 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:26:36.775197 10158 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:26:36.782644 10245 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.784543 10110 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.789011 10196 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:36.820902  6766 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:26:36.820309 10035 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:36.821904  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 57227.2us
I20260709 12:26:36.841148  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:36.843488  6773 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:36.852553  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:36.824399 10245 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:36.814163 10154 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:36.861878  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 97024.3us
I20260709 12:26:36.840487 10196 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.875504  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:36.841320 10110 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:36.867543 10196 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:36.903009  6773 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
W20260709 12:26:36.914175  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 448284us
I20260709 12:26:36.874490 10215 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:36.877931 10067 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:36.870779 10154 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:36.934376  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46881us
I20260709 12:26:36.912828  9926 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:36.940663 10266 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.063s	user 0.001s	sys 0.001s
I20260709 12:26:36.947649  5990 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:36.947888  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:36.952530  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 432500us
I20260709 12:26:36.963352  6766 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:26:36.965294  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 102179us
I20260709 12:26:36.966351  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:36.941568 10085 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:36.971194  5990 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:36.953926 10048 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:36.984563  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.967403 10158 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:36.962592  9883 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:36.996493  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:36.996335  5990 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:36.998865  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 209870us
I20260709 12:26:36.955714  9926 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:37.010128  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:36.991026  9912 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.000392  9800 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
W20260709 12:26:36.997567  9953 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.021s	sys 0.022s
I20260709 12:26:37.016847  9926 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:37.022348 10247 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:37.022714 10110 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:37.028393 10061 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:37.034812  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:37.000895 10227 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:37.043093  6611 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:37.043617 10227 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:37.044332 10227 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:37.044971  6611 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:36.997733 10187 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:26:37.031368  9953 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.022s	sys 0.023s
I20260709 12:26:37.047426  9953 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:37.047960  9953 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.048331  9953 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:37.050886  5694 catalog_manager.cc:5697] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:37.053608  5694 catalog_manager.cc:5697] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 6 to 7, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 7 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
W20260709 12:26:37.042727 10268 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:26:37.006897 10246 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:36.964989 10067 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:37.079380  6778 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:26:37.006376 10085 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:26:37.031919  9842 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:37.037388 10247 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:37.033141  9883 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:37.085347  6611 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:37.094200 10032 consensus_queue.cc:1059] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
W20260709 12:26:37.114661  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 134814us
I20260709 12:26:37.113344 10175 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:37.118462  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:37.096616 10075 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:26:37.119416 10075 consensus_queue.cc:1243] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:37.084024 10243 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:37.128363  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 217375us
I20260709 12:26:37.099017 10277 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:37.057979  9912 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:37.143394  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 112136us
W20260709 12:26:37.138432  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56886.4us
W20260709 12:26:37.144789  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 149822us
I20260709 12:26:37.142964  5840 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:37.121553 10075 raft_consensus.cc:3055] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:37.164844  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:37.142661 10187 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:37.157680 10269 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.251s	user 0.000s	sys 0.002s
I20260709 12:26:37.155583  9894 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:37.184258  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.181979  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:37.198027  6016 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:37.201053  6611 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:37.206631  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 196796us
I20260709 12:26:37.142932 10277 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.081377 10246 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:37.214684  6017 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:37.122102 10243 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:37.122390 10229 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:37.234066  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 69368.2us
I20260709 12:26:37.169921 10075 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:37.235621  5913 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:37.241542  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:37.245219  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 66791.3us
I20260709 12:26:37.207190 10215 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:37.202250 10289 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:37.241401 10235 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:37.226938 10277 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:37.263198  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 48613.6us
W20260709 12:26:37.263861  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 126321us
I20260709 12:26:37.242928  9918 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:37.243171  9894 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:37.246404 10154 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:37.260145  9912 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:37.288961 10187 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:37.293767 10291 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:37.294598 10291 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:37.297735 10274 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:37.299407  9918 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:37.299839  9918 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:37.301163 10132 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:37.301837 10132 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:37.303037 10132 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:37.304677 10115 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:37.305217 10115 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:37.251107  9800 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:37.254189  9911 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:37.283071 10272 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 12:26:37.261214 10289 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:37.287159 10229 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:37.257586  9883 raft_consensus.cc:697] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:26:37.314317 10279 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:26:37.314798  9911 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:37.316367 10075 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:37.265662 10235 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.320602  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:37.301033  9918 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:37.318650 10265 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.527s	user 0.008s	sys 0.000s
I20260709 12:26:37.340853  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:37.338835  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:37.231437 10247 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:37.343617  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:37.342444  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.342947  6757 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:37.347307  9953 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:37.323366  9800 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:37.348665  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:37.318707  9842 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:26:37.360594  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:37.327903 10289 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:37.360999  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 107726us
I20260709 12:26:37.337185 10235 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:37.305816 10298 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:26:37.343612 10229 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:37.373018  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:37.380175  5867 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:37.374212  5990 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:37.355850  5840 connection.cc:256] RPC call timeout handler was delayed by -1.5140022069999759s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1768s for a response
W20260709 12:26:37.381418  5840 connection.cc:256] RPC call timeout handler was delayed by -1.4969022069999482s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
I20260709 12:26:37.297871  9894 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:37.329838 10291 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:37.375655 10075 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:37.390470 10219 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.621s	user 0.010s	sys 0.005s
I20260709 12:26:37.384727 10247 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:37.406337  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1432) took 1017 ms. Trace:
I20260709 12:26:37.412720  9953 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:37.366364 10115 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:37.413117 10153 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 12:26:37.414211 10153 consensus_queue.cc:1243] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:37.414738  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:37.414856 10153 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:37.415683  6785 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:37.416098  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:37.416472 10293 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:26:37.417479  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:37.418013 10293 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:26:37.418408 10113 consensus_meta.cc:287] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.746s	user 0.011s	sys 0.004s
I20260709 12:26:37.418489 10293 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:37.419221 10113 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:37.419863 10293 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:37.420743  5873 raft_consensus.cc:2368] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13.
I20260709 12:26:37.422751  9911 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:37.423985 10113 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:37.422843 10227 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:37.425789 10300 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:37.426532 10300 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:37.427304 10227 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:37.427462 10300 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:37.418512  6756 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 9: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:37.409757  5871 rpcz_store.cc:276] 0709 12:26:36.388590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.401572 (+ 12982us) service_pool.cc:224] Handling call
0709 12:26:36.406228 (+  4656us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:36.465342 (+ 59114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:36.465394 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:36.484927 (+ 19533us) log.cc:844] Serialized 26 byte log entry
0709 12:26:36.934018 (+449091us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:36.934059 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:36.934129 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:37.403658 (+469529us) raft_consensus.cc:1685] finished
0709 12:26:37.403686 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:37.406241 (+  2555us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":441056,"threads_started":1,"wal-append.queue_time_us":544098}
W20260709 12:26:37.437850  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:37.439636  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.445593  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:37.446326  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:37.446511  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:37.447110  6768 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:37.450532  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:37.451551  6775 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:37.455940  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:37.414438 10282 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:37.462425  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 82054.3us
I20260709 12:26:37.463330  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:37.460516  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 211702us
I20260709 12:26:37.390377 10296 consensus_queue.cc:1059] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.113s
I20260709 12:26:37.438598 10187 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:37.465296  5915 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:37.435676 10243 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:37.470685  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:37.406280 10219 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:37.472307  5990 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:37.481988  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:37.482668  6016 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:37.487097  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:37.466540 10306 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:37.487196  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:37.488610  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:37.490763  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:37.490065  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.491324  6020 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 7.
I20260709 12:26:37.494917  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:37.467667 10187 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:37.467152 10246 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:37.500869  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.504597  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:37.505548  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:37.507037  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:37.530105  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:37.480338 10215 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:37.508560  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:37.442823  9883 consensus_queue.cc:237] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:37.504961 10243 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:37.468321 10282 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:37.505241 10296 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:37.475474  9800 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:37.559813  5949 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:37.500499 10308 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:37.563546  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:37.564436  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:37.563973 10312 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.577826  6768 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:26:37.577486  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:37.579835  5866 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:37.579737  5990 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:37.582048  6756 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:26:37.558337 10243 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:37.589071  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:37.565430  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:37.578016 10245 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.712s	user 0.007s	sys 0.008s
I20260709 12:26:37.500566 10306 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:37.561053 10296 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:37.540546 10215 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:37.601156  5865 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:37.570086 10308 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.585242 10228 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:37.585978 10304 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:37.594800 10243 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:37.619833  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:37.604995 10296 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:37.620934  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:37.623332  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:37.624347  5948 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 13: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:26:37.621433  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 56083.3us
I20260709 12:26:37.622016 10215 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:37.623564  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:37.624505  5940 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:37.625864  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:37.627534  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:37.628989  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:37.609385  9894 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:37.608446 10308 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:37.632467  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.628012  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:37.586522 10312 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:37.615617 10228 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:26:37.636780  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 245088us
I20260709 12:26:37.616735 10032 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:37.600840 10245 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:37.646734  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:37.654738  5948 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:37.655007 10306 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:37.651134  9911 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:37.631899 10304 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:37.656148 10282 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:37.657203 10304 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:37.661896 10245 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:37.650542  5840 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.196:34835 timed out after 2.238s (SENT) [suppressed 17 similar messages]
I20260709 12:26:37.647133  9842 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:37.642086 10219 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:37.690026 10196 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.763s	user 0.010s	sys 0.007s
I20260709 12:26:37.712656  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:37.712226  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 327283us
I20260709 12:26:37.714097  6768 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:37.714778  5940 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:26:37.715242  5940 pending_rounds.cc:85] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 6
I20260709 12:26:37.715508  5940 pending_rounds.cc:107] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:26:37.715734  5940 raft_consensus.cc:2889] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:37.716334  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 65823.2us
W20260709 12:26:37.716745  5840 connection.cc:256] RPC call timeout handler was delayed by -1.3639242600002035s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1723s for a response
W20260709 12:26:37.716933  5840 connection.cc:256] RPC call timeout handler was delayed by -1.1630022069998631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
W20260709 12:26:37.717087  5840 connection.cc:256] RPC call timeout handler was delayed by -1.1110320979996686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
W20260709 12:26:37.708170  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 242943us
I20260709 12:26:37.665406 10235 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:37.720331  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:37.640614 10032 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:37.727090  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:37.731477  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:37.718138  9911 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:26:37.734570  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:37.734899  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:37.735110  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:37.735546  5871 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:37.735843  6024 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:26:37.721931 10228 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:37.736689  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:37.737257 10228 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:37.737839 10067 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:37.740262 10294 raft_consensus.cc:3055] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:37.740687  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:37.741650 10113 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:37.742437 10085 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:37.742905 10085 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:37.743304 10085 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:37.743157 10277 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:37.744313  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:37.745280 10113 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:37.740808 10294 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:37.746978 10235 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:37.748931  6766 raft_consensus.cc:2368] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15.
I20260709 12:26:37.749056 10067 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:37.742477  6017 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:26:37.731410  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:37.750979 10307 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.217s	user 0.001s	sys 0.001s
I20260709 12:26:37.720813 10196 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:37.751582 10187 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:37.752766 10187 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:37.753059 10196 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:37.752682 10113 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:37.758247  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:37.720028 10158 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:37.766494  6768 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:26:37.768926  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:37.771441 10032 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:37.725715 10245 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:37.771761  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:37.771507  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:37.773644  5943 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:37.774540  5990 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:37.778895  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 47942.8us
W20260709 12:26:37.782404  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.783288  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:37.780587  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:37.763585  5840 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:37.784837  6773 pending_rounds.cc:85] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 7
I20260709 12:26:37.785172  6773 pending_rounds.cc:107] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:26:37.785483  6773 raft_consensus.cc:2889] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:37.785216  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:37.783970  6757 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:37.788084  5948 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:26:37.762789  9911 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:37.772279 10245 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:37.793905  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:37.778479 10196 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:37.800283  6023 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:26:37.802642  5915 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:37.782775 10277 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:37.807984  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 53803.8us
I20260709 12:26:37.762439 10113 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:37.803768  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:37.806870  6015 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:37.805094  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:37.776618 10235 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:26:37.810616  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:37.800279 10110 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:37.811558  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:37.812853  6757 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:37.813225  9800 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:37.816010  9800 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.811628  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:37.777462 10158 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:37.798532 10246 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:37.789319 10304 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:37.824666  6023 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 12: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:37.830413  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:37.832249  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:37.833058  6015 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:37.834602 10084 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:37.835237  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:37.836213 10277 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:37.813509 10113 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:37.823796  9800 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:37.827078 10276 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:37.829082 10246 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:37.840896  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:37.808004 10196 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:37.861243 10196 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:37.826186 10298 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:37.861304 10158 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:37.863672 10308 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:37.865408  5944 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 12: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:37.855432  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:37.838239  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:37.872622  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:37.874776  6016 raft_consensus.cc:2410] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:26:37.853440 10227 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:37.878209 10228 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:37.889588  6609 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:37.893081  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:37.867218 10246 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:37.886152 10235 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:37.899032  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 43715.2us
W20260709 12:26:37.906054 10297 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.471s	user 0.006s	sys 0.000s
I20260709 12:26:37.908056 10110 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:37.908767 10110 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:37.913784  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:37.914705  5942 raft_consensus.cc:2410] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:37.915597  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:37.916502  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:37.916965  5942 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:26:37.920032  5838 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:37.893222 10158 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:37.896440  9842 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:37.922258 10061 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:37.927712 10061 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:37.928174 10061 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:37.928328 10308 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:37.748081 10294 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:37.930262 10294 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:37.922472 10277 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:37.933878 10277 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:37.926142  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:37.934386 10277 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:37.926571  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:37.939990 10247 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.001s
I20260709 12:26:37.941224 10245 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:37.941918 10245 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:37.942559 10293 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:37.943602  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:37.924612 10158 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:37.945580 10154 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:37.946283 10158 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:37.946425 10154 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:37.947009 10154 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:37.949731  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:37.925429 10296 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:37.959607 10276 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:37.960547  9918 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:37.960353 10276 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:37.961109 10296 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:37.961889  5945 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:37.963474  6024 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
W20260709 12:26:37.975776  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.967792  5990 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:37.964715  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:37.899529 10304 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:26:38.001974  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.004477  5943 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:37.998706 10061 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:38.016183 10061 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:38.017009 10061 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:38.019064 10085 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:37.963414  9842 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.036659  5948 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:38.037000 10085 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:38.038329 10243 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:38.041148  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 75183.3us
I20260709 12:26:38.042294  5990 leader_election.cc:400] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15.
I20260709 12:26:38.042690  5990 leader_election.cc:403] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:38.043701  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 79060.7us
W20260709 12:26:38.005674 10323 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:38.049491 10243 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:38.059513 10306 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:38.060168 10306 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.060520 10306 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:38.065828 10298 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:26:38.072535  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 179489us
I20260709 12:26:38.045864  9842 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:38.067735 10245 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:38.081578  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:38.067682 10333 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 15
W20260709 12:26:38.076450  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.108039  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:38.113262  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:38.115402  5943 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:38.117383  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:38.090432 10132 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:38.121577  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 199920us
I20260709 12:26:38.110513  9953 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:38.132155  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:38.132836  6762 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:38.126628  5694 catalog_manager.cc:5697] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 7 to 9, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 9 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:38.080365 10243 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:38.111753 10235 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:38.134526  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:38.136853  6757 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:38.079474 10154 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:38.083420 10299 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:26:38.135308  9953 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.116698 10277 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:38.157126  9953 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:38.159188 10277 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:38.159560  9911 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:38.131490 10298 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:38.161489  6773 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:38.161304 10298 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:38.164000 10298 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:38.165906 10276 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:38.171849 10320 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.448s	user 0.003s	sys 0.000s
I20260709 12:26:38.150089  6609 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:38.137900 10243 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:38.126410 10304 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:38.188733 10243 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:38.193145  5685 maintenance_manager.cc:643] P 03f70467f9c34de887520aec76068d6f: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 6.290s	user 0.172s	sys 0.102s Metrics: {"bytes_written":29047,"cfile_init":1,"dirs.queue_time_us":409908,"dirs.run_cpu_time_us":610,"dirs.run_wall_time_us":16383,"drs_written":1,"lbm_read_time_us":778,"lbm_reads_lt_1ms":4,"lbm_write_time_us":85722,"lbm_writes_1-10_ms":3,"lbm_writes_10-100_ms":2,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"rows_written":63,"thread_start_us":351352,"threads_started":2,"wal-append.queue_time_us":141264}
I20260709 12:26:38.197736  5686 maintenance_manager.cc:419] P 03f70467f9c34de887520aec76068d6f: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7013 bytes on disk
I20260709 12:26:38.198962  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:38.199363 10312 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:38.204733  5866 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:38.214658  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 74996.6us
I20260709 12:26:38.210855  6793 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:38.215483  6609 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:38.201292  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.216504  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.216822  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.205871  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:38.219800  5870 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:38.222489  5945 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:26:38.223560  5948 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:26:38.225339  5948 pending_rounds.cc:85] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 7
I20260709 12:26:38.225574  5948 pending_rounds.cc:107] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:26:38.225824  5948 raft_consensus.cc:2889] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:38.223568  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:38.234931  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:38.175244 10300 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:38.237996  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:38.239189  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:38.241700  5949 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:26:38.242214  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:38.243979  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:38.245330  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:38.247093  6766 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:38.201363 10327 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:38.252283  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:38.253360  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:38.253953  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:38.235747 10235 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:38.256466  5685 maintenance_manager.cc:643] P 03f70467f9c34de887520aec76068d6f: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.039s	user 0.001s	sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":595,"lbm_reads_lt_1ms":4}
I20260709 12:26:38.264367  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:38.265116  5867 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:38.266741 10115 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:38.269538  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:38.280390  5913 leader_election.cc:400] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13.
I20260709 12:26:38.280824  6019 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:38.281093  5694 catalog_manager.cc:5697] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 8 to 9, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 9 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:38.274434  6762 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:38.283231  5694 catalog_manager.cc:5697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 3 to 5, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 5 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:38.284724  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:38.285951  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:38.287181  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:38.291147  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:38.293211  5870 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:38.292181  5945 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:38.294644  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:38.296921  6017 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:38.233799 10243 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:26:38.290326  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:38.255473 10310 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:38.313750  6020 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:38.316363  6762 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 12:26:38.295293 10289 consensus_meta.cc:287] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.904s	user 0.013s	sys 0.002s
I20260709 12:26:38.271723 10311 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.249233 10158 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:38.325103  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 106440us
W20260709 12:26:38.278976  5692 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50588 (request call id 56) took 22305 ms (client timeout 15000 ms). Trace:
I20260709 12:26:38.330317  5694 catalog_manager.cc:5697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 1 to 5, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 5 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.301415 10061 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:38.311645  5867 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:38.250790 10300 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:38.342643  6757 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:38.294790 10293 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:26:38.295161  9953 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:38.322784 10327 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:38.321259 10085 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:38.344123  5694 catalog_manager.cc:5697] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 5 to 6, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 6 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:38.344889  5865 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:38.340685  5913 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:38.295977 10306 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:38.276430 10246 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:38.339320 10310 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:38.354628  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:38.324867 10289 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:38.329092 10311 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:38.359721 10085 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:38.370913 10300 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:38.371248 10311 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:38.371412 10085 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:38.338271 10308 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
W20260709 12:26:38.372429 10339 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:26:38.372627 10229 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:38.373143 10229 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:38.374410  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:38.374150 10229 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:38.375262 10310 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:38.375612 10229 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:38.376259 10310 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:38.366941  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:38.378386  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:38.378996  9953 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:38.379944  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:38.381376 10337 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:38.359704 10175 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:38.382243 10337 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:38.382314 10175 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:38.383497 10175 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:38.388646  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:38.388513  6021 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:38.388087  5694 catalog_manager.cc:5697] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 6 to 7, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 7 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:38.389945  5866 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:38.357990 10293 consensus_queue.cc:1243] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:38.388043  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:38.393458  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:38.396078  6016 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:38.392911  5694 catalog_manager.cc:5697] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:38.397118  5874 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:38.407917 10110 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:38.408622  5866 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:38.409547  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:38.410321  5940 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:38.413530  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:38.414382  5943 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:38.422158  5694 catalog_manager.cc:5697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 4 to 5, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 5 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.425307  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:38.426146  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:38.427793 10110 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:26:38.431874  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:38.432624  5940 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:38.432675 10110 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:38.433259 10110 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:38.433352  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.434543 10110 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:38.435223 10132 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:38.459025 10132 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:38.460232  9800 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.002s
I20260709 12:26:38.460779  9800 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:38.461385 10110 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:38.461885  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 58179.7us
I20260709 12:26:38.461988  9949 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:38.463851  5694 catalog_manager.cc:5697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.440850  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:38.466414  5944 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:26:38.333189  5692 rpcz_store.cc:269] 0709 12:26:15.973132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.020655 (+ 47523us) service_pool.cc:224] Handling call
0709 12:26:38.278664 (+22258009us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:30.745737 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:30.748437 (+  2700us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:30.748528 (+    91us) write_op.cc:435] Acquired schema lock
0709 12:26:30.748563 (+    35us) tablet.cc:662] Decoding operations
0709 12:26:30.751502 (+  2939us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:30.751664 (+   162us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:30.751717 (+    53us) tablet.cc:685] Acquiring locks for 21 operations
0709 12:26:30.761872 (+ 10155us) tablet.cc:701] Row locks acquired
0709 12:26:30.761933 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:26:30.762185 (+   252us) write_op.cc:270] Start()
0709 12:26:30.762375 (+   190us) write_op.cc:276] Timestamp: P: 1783599990762092 usec, L: 0
0709 12:26:30.762396 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:26:30.763370 (+   974us) log.cc:844] Serialized 8171 byte log entry
0709 12:26:30.922425 (+159055us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b1800405400. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:26:31.086304 (+163879us) op_driver.cc:464] REPLICATION: finished
0709 12:26:31.386366 (+300062us) write_op.cc:301] APPLY: starting
0709 12:26:31.400003 (+ 13637us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:31.472845 (+ 72842us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:31.479344 (+  6499us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:31.673791 (+194447us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:31.674381 (+   590us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=21
0709 12:26:31.674456 (+    75us) write_op.cc:312] APPLY: finished
0709 12:26:31.688053 (+ 13597us) log.cc:844] Serialized 144 byte log entry
0709 12:26:31.688327 (+   274us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:31.688565 (+   238us) write_op.cc:454] Released schema lock
0709 12:26:31.688653 (+    88us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:31.689137 (+   484us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:31.689163 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":454233,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":193932,"apply.run_cpu_time_us":10593,"apply.run_wall_time_us":323054,"num_ops":21,"prepare.queue_time_us":17443,"prepare.run_cpu_time_us":29091,"prepare.run_wall_time_us":72937,"raft.queue_time_us":143445,"raft.run_cpu_time_us":24775,"raft.run_wall_time_us":156157,"replication_time_us":317173,"spinlock_wait_cycles":6980352,"thread_start_us":329129,"threads_started":4,"wal-append.queue_time_us":54990,"wal-append.run_cpu_time_us":32725,"wal-append.run_wall_time_us":1772600}]]}
I20260709 12:26:38.469481  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:38.470629  6773 raft_consensus.cc:606] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:38.471055  6773 raft_consensus.cc:614] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:38.473537 10246 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:38.443593  5940 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:26:38.475004  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:38.476011  6793 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 16: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:38.476847 10289 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:38.422019 10337 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:38.479176 10337 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:38.478468  5694 catalog_manager.cc:5697] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 5 to 8, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 8 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:38.479600 10337 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:38.480399  5694 catalog_manager.cc:5697] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 8 to 9, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 9 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:26:38.482064  6017 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:38.454624  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:38.456696 10274 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:38.457254 10235 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:38.464902  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:38.460062 10132 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:38.475190 10158 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:38.452546  5840 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:38.436374  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:38.501483  6018 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:38.484440  5946 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:38.459913  6768 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 14. Current term is 15. Ops: [12.8-14.9]
I20260709 12:26:38.488013  5866 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:38.488396  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:38.496316  5874 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:38.414845 10309 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:38.400207 10154 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:38.451949 10306 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:38.530001 10032 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:38.532450  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:38.508219  5871 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:38.510018  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:38.511329 10132 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:38.512346  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 85108.5us
W20260709 12:26:38.514060  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 49257.4us
I20260709 12:26:38.512408  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:38.509434  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:38.513023 10333 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15.
I20260709 12:26:38.523859  9949 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:38.524874 10245 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:38.525473  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:38.563691  5949 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: [14.6-16.7]
I20260709 12:26:38.505944  5694 catalog_manager.cc:5697] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:38.568032 10245 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:38.568857  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:38.569530  5867 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:38.570590 10245 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:38.571483  6024 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:38.574476  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:38.575347  5944 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:38.574581  9800 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:38.578999  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.579551 10235 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:38.574476  6017 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:38.544000  6020 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:38.572461  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:38.584777  9949 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:38.586354  9949 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:38.604038  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:38.539477  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:38.608718  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:38.596500  5867 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:38.609865  6609 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:38.615809  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:38.616652  6757 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:38.610816  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:38.619843  6768 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:38.624567 10229 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:38.625218 10229 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:38.626192 10229 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:38.500527 10274 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:38.531581 10085 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:38.503415  9842 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:38.559866 10304 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:38.564108 10299 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:38.653967  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:38.658146  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.663515  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 53134.8us
I20260709 12:26:38.685354  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:38.538295 10309 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:38.610894 10276 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:38.687254  5840 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:38.669031  6023 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:38.695959  6785 raft_consensus.cc:2368] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13.
W20260709 12:26:38.677986 10277 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.009s	sys 0.006s
I20260709 12:26:38.692298  5694 catalog_manager.cc:5697] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 1 to 3, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 3 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:26:38.696610  5872 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:38.673259 10303 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:38.651819 10085 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:38.615078  9800 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:26:38.716027  5915 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:38.717947  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:38.646729 10084 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:26:38.690917 10281 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:38.724967  5694 catalog_manager.cc:5697] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 4 to 7, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 7 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:38.727689  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:38.735193  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:38.728410  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:38.692559 10158 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:38.730110  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:38.736866  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:38.737776  5866 raft_consensus.cc:2393] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:38.693337 10304 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:38.739840  5990 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:38.740104  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 148056us
I20260709 12:26:38.697074 10061 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:26:38.698736 10277 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.009s	sys 0.006s
W20260709 12:26:38.748912  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.727475  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:38.743917  5869 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 16: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:38.702384 10276 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:38.754062  5871 raft_consensus.cc:2368] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12.
I20260709 12:26:38.714574 10303 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:38.755927  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 72551.9us
I20260709 12:26:38.732482  6757 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:38.725594 10085 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:38.759093  5942 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 12:26:38.759783  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.759892  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:38.760775  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:38.761022  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:38.761667  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:38.761969  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:38.747934 10061 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.781116 10061 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:38.764168 10310 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:38.783720 10310 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:38.765559  9800 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.784951  9800 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:38.786592  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:38.788894  9800 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:38.790118 10310 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:38.768033  6018 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:38.777849  5866 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:38.780460  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:38.789439  5868 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:38.798535  5946 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:26:38.799170  5946 pending_rounds.cc:85] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 8
I20260709 12:26:38.799479  5946 pending_rounds.cc:107] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:26:38.799782  5946 raft_consensus.cc:2889] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:38.738767 10158 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:38.691304 10299 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:38.755585 10245 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:38.813954  5948 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:38.820325  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:38.821175  5869 raft_consensus.cc:2393] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:38.778435 10303 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:38.816056  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:38.825209 10311 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:38.825589  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:38.826504  5869 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:26:38.832983 10351 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:26:38.810956 10158 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:38.841782 10274 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:38.846710  5872 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:26:38.847211  5872 pending_rounds.cc:85] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 7
I20260709 12:26:38.847415  5872 pending_rounds.cc:107] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:26:38.847699  5872 raft_consensus.cc:2889] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:38.850322  9842 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:38.813525 10309 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:38.777829 10306 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:26:38.856595 10276 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:38.861335 10299 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:38.816668  9800 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:26:38.822669 10154 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:26:38.798764 10084 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:26:38.864253 10084 consensus_queue.cc:1243] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 16.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:38.865298 10304 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:38.814424 10335 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.868979 10335 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:38.871521 10335 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:38.880115  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:38.880656  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:38.884428  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:38.885241  5942 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 13: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:38.885542  5940 pending_rounds.cc:85] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 7
I20260709 12:26:38.885963  5940 pending_rounds.cc:107] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 12:26:38.886284  5940 raft_consensus.cc:2889] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:38.889125  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:38.909767  5868 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 16: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:26:38.912864  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 187798us
W20260709 12:26:38.916185  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 205440us
I20260709 12:26:38.920940  5915 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:38.926954  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.928844  5868 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:38.864338 10061 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:38.929534  5868 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:38.937238 10303 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:38.943734  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:38.942929  6611 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:38.987854  5867 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 8: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:38.944636  5868 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 6, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:38.948432  9800 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:38.994786  5868 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:39.000000  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:39.008023  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:39.009824  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 67114.5us
I20260709 12:26:39.016714  6757 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:39.024108  6793 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:38.997339 10293 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.016484 10276 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:39.030324  9800 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:39.051997  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:39.031011  9883 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:26:39.031529 10110 consensus_meta.cc:287] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.569s	user 0.012s	sys 0.000s
I20260709 12:26:39.033464  6773 raft_consensus.cc:606] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:39.035857 10067 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:39.036084  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.042368 10153 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:39.045980 10061 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:39.049182  9911 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:39.050915 10303 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:38.950480 10337 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:39.055473  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:39.056110  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.073096  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:39.054708 10339 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:26:39.065732 10360 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:26:39.066479 10243 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:39.062314 10153 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:39.074527  5990 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.067649 10289 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:39.054764  9883 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:39.103114  5840 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.075161  5868 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:26:39.109778  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.110183  5868 pending_rounds.cc:85] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 8
I20260709 12:26:39.110630  5868 pending_rounds.cc:107] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:26:39.111119  5868 raft_consensus.cc:2889] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:39.113312  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:39.114431  6775 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:39.120054  5870 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:39.102237 10360 consensus_queue.cc:1243] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:39.062618 10293 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:39.104115 10061 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:39.125077  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:39.131108  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:39.131956  6024 raft_consensus.cc:2393] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 12: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:39.076581 10276 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:39.129673  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.107812 10067 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:39.107340 10357 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.111218 10299 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:39.131211  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.120494 10113 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:39.117681 10110 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:39.137365  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:39.149230  6768 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:39.125195 10229 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:39.145195  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:39.145356  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:39.139966  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:39.134969  9912 raft_consensus.cc:3055] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:39.118113 10158 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:39.150583 10247 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:39.155972  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:39.165304  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:39.171265  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:39.159293  6762 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:39.138307 10312 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:39.175432  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.175675  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:39.165948  5942 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:39.177354  6762 raft_consensus.cc:2368] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12.
I20260709 12:26:39.159911 10229 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:39.173058 10246 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:39.159808 10154 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:39.159967  9912 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:39.184623 10229 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:39.157857 10153 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:39.160410 10291 raft_consensus.cc:3055] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:39.186720 10291 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:39.147692 10357 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:39.187620 10291 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:39.188827  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:39.189209 10357 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:39.182157  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:39.159631 10085 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:39.194489 10359 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:26:39.194190 10312 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:39.196259 10366 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:39.196552  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:39.196972  5871 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:39.194181 10158 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:39.198629  6019 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:26:39.187013 10153 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:39.185297  9912 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:39.190006  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:39.210106  9912 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:39.210141  6024 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:39.211663  6757 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:39.212352  6757 raft_consensus.cc:606] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:26:39.138839 10175 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:26:39.199417  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:39.215337  6766 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:39.216524 10110 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:39.138550 10327 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:26:39.202198  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.189682  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:39.204275  6773 pending_rounds.cc:85] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 7
I20260709 12:26:39.234210  6773 pending_rounds.cc:107] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:26:39.207587 10293 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:39.144086  9800 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:39.234413  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:39.234314  6024 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:39.234577  6773 raft_consensus.cc:2889] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:39.244024  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.245559 10299 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:39.251993  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.196590 10085 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:39.232174 10289 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:39.233556 10276 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:39.237361  6757 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:39.237932 10337 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:39.239259 10175 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:39.252327  5840 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.254112  6019 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:26:39.256742  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:39.274490  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:39.235761  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 46227.7us
I20260709 12:26:39.277985  6015 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 11: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:39.262744  6757 raft_consensus.cc:606] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:39.275024  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.278851  6757 raft_consensus.cc:614] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:39.253093 10299 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:39.277083 10300 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.286161 10300 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:39.288812 10300 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:39.276691 10306 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.290009 10306 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:39.292811 10306 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:39.263954 10298 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:39.274356 10289 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:39.294806 10245 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:39.295764 10289 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:39.275664 10246 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:39.305764  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.306181  6775 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:39.310127  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:39.282677  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:39.310895  5940 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:26:39.311825  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:39.312355  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:39.282881  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:39.316318  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.319998  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:39.311000  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:39.320814  6775 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
W20260709 12:26:39.322301  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 70064.2us
I20260709 12:26:39.324993  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:39.326143  6775 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:39.341526  5838 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:39.348176  6793 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:26:39.350952  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:39.335968  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:39.354017  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:39.360018  5940 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 8.
I20260709 12:26:39.363234  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:39.364144  5872 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:39.363441  5948 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:39.284062 10311 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:39.365895  5840 leader_election.cc:400] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12.
I20260709 12:26:39.366922  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:39.370163  5869 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:39.324748 10352 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:39.320832 10299 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:39.320751 10245 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:39.367337  5873 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:39.375051  5990 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:39.330886 10289 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:39.338117 10294 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:39.338876 10300 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:39.377959  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 1452) took 1167 ms. Trace:
I20260709 12:26:39.310591  9953 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:26:39.380101  5840 leader_election.cc:403] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:26:39.382596  6609 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:39.384845  5840 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.366585 10061 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:39.381858  5874 rpcz_store.cc:276] 0709 12:26:38.204361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.208406 (+  4045us) service_pool.cc:224] Handling call
0709 12:26:39.371703 (+1163297us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54004f60d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:39.371823 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370199936}
I20260709 12:26:39.374392 10299 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:39.390908  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:39.391620 10306 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:39.392136 10306 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.392479 10306 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:39.386104 10110 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12.
I20260709 12:26:39.353518 10327 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:26:39.391644  6757 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:39.372875 10377 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:39.373090 10352 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:39.389351  5840 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.375509 10357 consensus_queue.cc:1059] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:26:39.401480  6775 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:26:39.369124 10085 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:26:39.402413 10061 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:39.413390  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:39.389338 10282 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:39.388329 10311 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:39.376050 10229 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:26:39.413403  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:39.395572 10299 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.409359 10352 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:39.387235 10245 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:39.401117 10158 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:39.513984  6766 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:39.420077 10110 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:39.515022  5915 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:39.403337 10309 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 12:26:39.416038 10357 raft_consensus.cc:3055] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:39.421932 10282 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.515632 10299 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:39.548041  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 212169us
I20260709 12:26:39.569164 10158 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:39.535175  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 88470.1us
W20260709 12:26:39.569010 10382 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:39.570361  5987 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:39.588536  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:39.542629  9894 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:39.607860  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1098) took 1496 ms. Trace:
I20260709 12:26:39.548597 10282 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:39.549239 10110 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:39.610227  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:39.567340 10245 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:39.608405  6785 rpcz_store.cc:276] 0709 12:26:38.111743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.112982 (+  1239us) service_pool.cc:224] Handling call
0709 12:26:39.607571 (+1494589us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b5400006254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:39.607774 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1333327616}
I20260709 12:26:39.572969 10377 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:39.530124 10357 raft_consensus.cc:740] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:39.662619  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.599344 10311 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:39.704037  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 93956.5us
W20260709 12:26:39.725162  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.608315  9894 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:39.662693 10387 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:26:39.730628  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 354594us
I20260709 12:26:39.662782 10276 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:39.677297 10377 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.603688 10229 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.125s
I20260709 12:26:39.723569 10327 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:39.662303 10061 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:39.787699  5869 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:39.790922  6609 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:39.724113 10336 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:39.825914  5915 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:39.774477 10377 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:39.832697  6021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 938) took 1465 ms (client timeout 1662 ms). Trace:
I20260709 12:26:39.842816 10390 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:39.843792 10390 raft_consensus.cc:697] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:39.773977 10311 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:39.839895  6021 rpcz_store.cc:269] 0709 12:26:38.335389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.345973 (+ 10584us) service_pool.cc:224] Handling call
0709 12:26:39.800490 (+1454517us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b5400364cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:39.800757 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53708160}
I20260709 12:26:39.794862 10376 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:39.745815 10388 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.122s
I20260709 12:26:39.835726 10153 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:39.824249 10158 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.773135 10276 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:39.876292  5990 leader_election.cc:400] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13.
I20260709 12:26:39.837322 10360 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.881177  5990 leader_election.cc:403] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:26:39.852699 10245 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.782285  9894 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:39.892298 10352 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:39.897842 10360 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:39.901891 10085 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.902842 10356 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:39.903421 10391 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:26:39.824196 10357 consensus_queue.cc:260] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:39.904553 10389 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:26:39.905160 10388 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:39.864130 10311 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:39.881554 10158 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:39.907457 10245 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:39.838027 10110 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:39.876437 10153 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:39.918722  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 127917us
I20260709 12:26:39.833377 10336 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:39.845185 10390 consensus_queue.cc:237] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:39.922719 10085 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:39.940001 10085 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:39.942091  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:39.928207 10311 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:39.928315 10352 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:39.931558  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.937247 10153 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:39.953207 10153 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:39.955571 10153 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:39.958025 10276 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:39.935326 10394 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:39.964761 10158 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:39.965757  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:39.966684  5948 raft_consensus.cc:2410] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:26:39.967057 10245 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:39.967895 10388 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:39.971091 10388 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:39.979197  6609 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:39.979490 10115 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:39.981009  5838 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:39.983611 10085 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:26:39.990751  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 119129us
I20260709 12:26:40.008111  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:40.021091  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:39.940047 10336 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:40.025122  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.026542  6017 raft_consensus.cc:2368] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8.
I20260709 12:26:40.032574  5949 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:26:40.027979  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:39.925895 10356 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:40.034612  6021 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:40.042119  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:40.018620 10306 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.626s	user 0.007s	sys 0.007s
W20260709 12:26:40.047268  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 64776.7us
I20260709 12:26:40.007589 10276 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:39.991334 10282 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:40.052944  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:40.006654 10388 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:40.060225  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:40.056656  6019 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:40.061735  6017 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:40.020570 10352 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.995886 10337 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:26:40.018139 10327 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:40.066756  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:39.997483 10354 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:26:40.024650 10397 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:40.064428  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:40.067847  6021 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:40.045168 10158 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:40.084749  5915 leader_election.cc:400] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8.
I20260709 12:26:40.085424  5915 leader_election.cc:403] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:40.052119 10085 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:40.089257  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:40.092602  6017 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:40.093293 10032 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 7. Current term is 8.
I20260709 12:26:40.098502  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1467) took 1287 ms. Trace:
I20260709 12:26:40.107750  5871 rpcz_store.cc:276] 0709 12:26:38.810816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.812519 (+  1703us) service_pool.cc:224] Handling call
0709 12:26:38.812848 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.197526 (+384678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.098221 (+900695us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b5400210a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:40.098438 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882861952}
I20260709 12:26:40.113052  5874 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:40.113140  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:40.058048 10306 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:40.114084  6017 raft_consensus.cc:2376] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14. Re-sending same reply.
I20260709 12:26:40.070773 10352 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:40.133728  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.134608  5871 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:40.070259 10327 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:40.097111 10296 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:40.137471  5865 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:26:40.139277  6021 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:26:40.139478  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 115655us
I20260709 12:26:40.097661 10281 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:40.076813 10388 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:40.145673  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 49284.7us
I20260709 12:26:40.147819  6015 raft_consensus.cc:2368] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14.
I20260709 12:26:40.090197 10158 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:40.082329 10397 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:40.143018 10247 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.142304 10296 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.165683  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1103) took 1792 ms. Trace:
W20260709 12:26:40.167208  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 140418us
I20260709 12:26:40.166404  6778 rpcz_store.cc:276] 0709 12:26:38.373605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.374151 (+   546us) service_pool.cc:224] Handling call
0709 12:26:40.165345 (+1791194us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b5400005854. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:40.165557 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1782067456}
I20260709 12:26:40.154865 10281 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:40.144048 10352 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:40.159986 10175 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:40.159276 10276 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:40.168022 10296 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:40.192780 10175 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:40.193251 10401 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:26:40.196405 10158 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:40.170902 10247 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:40.258458  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 157006us
I20260709 12:26:40.231981 10414 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.291296 10414 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:40.278301 10276 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:40.294731  5990 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:40.295998 10298 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:40.296533 10394 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13.
I20260709 12:26:40.297160 10298 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.298108 10298 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:40.302260 10294 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:26:40.310148  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.255844 10282 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:40.315582  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 167039us
I20260709 12:26:40.303325 10294 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:40.166199 10085 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:40.262771 10360 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:40.265308 10356 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:40.338078  6024 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
W20260709 12:26:40.335427 10402 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 12:26:40.335991 10404 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 12:26:40.321852 10406 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:40.362316 10327 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:40.366645 10417 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.367839 10414 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:40.369438 10281 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:40.371737 10299 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.761s	user 0.009s	sys 0.005s
I20260709 12:26:40.372329 10299 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:40.372752 10247 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:40.374305 10388 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:40.375126 10406 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:40.376951 10175 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:40.377555 10406 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:40.371963 10298 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:40.378044 10397 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:40.367548 10417 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:40.387516 10294 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:40.387749 10084 raft_consensus.cc:993] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: : Instructing follower 6add331326ca47038308a447bf798fa7 to start an election
W20260709 12:26:40.393483 10377 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.004s
I20260709 12:26:40.380950 10229 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:40.385943 10414 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:40.412806 10397 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:40.418248  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 1.00485e+06us
I20260709 12:26:40.418455 10229 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:40.419286  5840 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:40.420884  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:40.421200 10229 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:40.418097 10397 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:40.423815 10397 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:40.425192 10235 raft_consensus.cc:1081] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Signalling peer 6add331326ca47038308a447bf798fa7 to start an election
I20260709 12:26:40.428210 10408 raft_consensus.cc:993] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: : Instructing follower 7a0204b931394fc48ffc503716e0d111 to start an election
I20260709 12:26:40.428753 10408 raft_consensus.cc:1081] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Signalling peer 7a0204b931394fc48ffc503716e0d111 to start an election
I20260709 12:26:40.425921 10291 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:40.426064 10426 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.431908  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:40.433672  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:40.437521  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:40.442610  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:40.443538  6778 raft_consensus.cc:2393] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:40.444975  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:40.445919  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:40.430375 10312 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:40.451082  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 123921us
I20260709 12:26:40.452260  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1102) took 1754 ms. Trace:
I20260709 12:26:40.438446 10336 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:40.438457 10376 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:40.453598  5867 rpcz_store.cc:276] 0709 12:26:38.697787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.697923 (+   136us) service_pool.cc:224] Handling call
0709 12:26:40.452043 (+1754120us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b540069ee54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:40.452195 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":578594176}
I20260709 12:26:40.442171 10158 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:40.413429 10377 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:40.447868 10110 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
W20260709 12:26:40.546739  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.440032 10296 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:40.438499 10414 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:40.552973  5943 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:40.555091  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:40.526697 10276 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.555817  5948 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:40.560022  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:40.567504  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:40.568771  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 125960us
I20260709 12:26:40.569108  5946 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:26:40.534878 10303 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:26:40.568940  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:40.569718  5840 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:40.577714  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:40.439914 10426 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:40.584249  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:40.586400  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:40.586831  5866 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:40.588277 10399 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:26:40.561957 10158 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:40.589311 10110 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:40.589504 10084 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:40.589527 10158 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:40.590014  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:40.590387 10084 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:40.590818  6020 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:40.590920 10158 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:40.591387 10084 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:40.592005 10418 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:26:40.589354 10303 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.592502 10158 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:40.593043 10303 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:40.595233 10303 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:40.598747  5870 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:40.600597  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:40.602003  5874 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:26:40.603878  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:40.604830  5868 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:40.608657  5865 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:26:40.611433  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.615980  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1107) took 1977 ms. Trace:
I20260709 12:26:40.568398 10299 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:40.613891  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.618203  6019 raft_consensus.cc:2393] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:40.616475  6768 rpcz_store.cc:276] 0709 12:26:38.628065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.638861 (+ 10796us) service_pool.cc:224] Handling call
0709 12:26:38.686374 (+ 47513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.149806 (+463432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.605124 (+1455318us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b54002c1ad4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:40.605353 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1063762688}
I20260709 12:26:40.567121 10424 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:40.621414  5868 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:40.548610 10397 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:26:40.631851 10397 consensus_queue.cc:1243] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 17.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:40.562920 10276 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:40.634384  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:40.634435  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:40.639128  5866 raft_consensus.cc:2376] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10. Re-sending same reply.
I20260709 12:26:40.639226  5865 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:40.644219  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.644294  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.646258 10405 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:40.646807  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.647218  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:40.646436  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:40.657986 10433 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:26:40.650584 10294 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.659689 10294 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:40.666065 10384 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:40.667034 10384 consensus_queue.cc:1243] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:40.629799 10303 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:26:40.668084 10429 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.668213 10303 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:40.668733 10303 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:40.668880 10247 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.669611 10426 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:40.669772 10303 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:40.670580  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:40.671146 10303 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:40.671286  5944 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:40.671797  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:40.672467  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:40.672873  5945 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:40.673240  5944 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:40.674110  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:40.674677  5947 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:40.675635  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:40.671052  5946 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:26:40.676031  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:40.676268  5943 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:40.672801  5941 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:40.669517 10247 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:40.683954  5866 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:26:40.685503  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 425225us
I20260709 12:26:40.691746  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:40.645076 10417 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:40.692951  5872 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:40.700769  5942 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:40.706481  6611 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:40.711948  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:40.712010  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 111490us
I20260709 12:26:40.712764  6022 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:40.716470  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.718910  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.722729  6019 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:40.723577  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:40.724449  5866 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:40.733875  6017 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:26:40.730155  6020 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:40.736482  6020 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:40.739123 10110 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.739825 10110 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:40.741150 10411 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.450s	user 0.002s	sys 0.002s
I20260709 12:26:40.688932 10308 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:26:40.668694 10429 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:40.696005 10032 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:26:40.739825 10276 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:40.758297 10338 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:40.749385  6019 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c"
dest_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:34888
I20260709 12:26:40.767207  6019 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:40.767700  6019 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:40.772509  6022 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
W20260709 12:26:40.817859  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.831722  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.714432 10399 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:26:40.783172 10235 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:40.714222 10335 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:26:40.848619  6020 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:40.849845  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.853808  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:40.856354  5944 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:26:40.788633  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:40.789270 10408 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:40.796486  9894 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:40.798236  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:40.798658  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.800014 10312 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:40.803861  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:40.740777 10229 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:26:40.805141 10110 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:40.807708  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 900) took 1694 ms. Trace:
I20260709 12:26:40.870414  6793 rpcz_store.cc:276] 0709 12:26:39.112936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.113264 (+   328us) service_pool.cc:224] Handling call
0709 12:26:39.113621 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.348749 (+235128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.807455 (+1458706us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54002a0dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:40.807649 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538587904}
I20260709 12:26:40.808229  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:40.873056  6775 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:40.809617 10247 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:40.811805 10429 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:40.785041 10296 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:40.801187  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:40.770897 10395 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.819478  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:40.869499  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:40.883023  5868 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:26:40.820814  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:40.824204  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.751062 10377 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:40.823565  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:40.884521  6021 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:40.751653 10438 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:40.846737  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 155641us
I20260709 12:26:40.884056  6015 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:40.890820  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:40.872083  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 50502.9us
I20260709 12:26:40.872457  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:40.866092  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:40.878561  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:40.869120  5949 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:26:40.869378  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:40.893218  5872 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:26:40.882710  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:40.834246  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.867125  6756 raft_consensus.cc:2368] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14.
I20260709 12:26:40.867825  6022 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:26:40.860538 10424 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.931908 10424 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:40.935223 10424 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:40.867015  6611 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:40.895159  5942 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:40.895202  6017 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:26:40.896199  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.987792  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.988834  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:40.898067  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:40.901497  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:40.990302  5868 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:40.902729  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:40.865130 10061 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:40.903414  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:40.903995 10439 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:40.889554 10438 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.906571  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:40.907884  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:40.908601 10394 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.910238  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.912302 10442 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:41.006413  6609 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:40.914234 10296 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:40.923092  5872 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:41.010722  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:40.925805  6021 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:40.926285 10395 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:40.868920  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:40.860108 10308 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.020512  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:41.021692  5872 raft_consensus.cc:2410] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:26:40.985575 10300 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:40.951725 10306 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:40.951920 10153 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.864665 10408 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:40.954589 10440 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 12:26:40.956156 10427 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.548s	user 0.001s	sys 0.000s
I20260709 12:26:40.860776 10235 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:40.963442 10357 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:40.766006  9883 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:40.966466  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:40.862836 10393 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 12:26:40.884987 10390 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 12:26:41.058578  5869 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:41.060804  5869 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:26:41.061342  5869 pending_rounds.cc:85] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 7
I20260709 12:26:40.978340  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.880697  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:41.023007 10110 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:40.865962 10312 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:40.993067  6756 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:41.068485  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:40.994889  5948 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: [14.6-16.7]
I20260709 12:26:40.995595  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:40.998592  5868 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:54876
I20260709 12:26:41.069936  6022 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:26:41.070163  5868 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:41.070930  5868 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:41.002200  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.009102  6015 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 13: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:26:41.012126  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 104329us
I20260709 12:26:41.013854  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:41.017480  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:41.075834  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.941610  9949 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:40.993659  9911 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:41.023847  5872 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:41.025137  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.005880 10438 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:41.083757  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.086396  6793 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:26:41.086669  9883 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:41.008991 10424 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:41.089339 10424 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:41.044080  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.020958 10409 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:41.093952 10409 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.094525 10409 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:41.095784 10384 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:41.020727 10245 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:41.078248 10110 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:41.100517 10110 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:41.025679 10308 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:41.103428 10294 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:41.105280  6778 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:26:41.105713 10308 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:41.106938 10395 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:41.078249  9949 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:41.029392 10300 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:41.108530  9949 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:41.109686  9949 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:41.109922 10300 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:41.043001 10153 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:41.112630 10153 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:41.114435  6778 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:40.974980  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.115911  5942 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:41.055465  6019 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:41.062702  6775 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
W20260709 12:26:40.990810  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 123904us
I20260709 12:26:41.062764  6023 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:41.118644  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.062160  5869 pending_rounds.cc:107] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:26:41.118719  6611 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:41.026880  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:41.015631 10394 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:41.076100  5866 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:41.121323  5871 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:41.122210  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 95443.3us
I20260709 12:26:41.123186  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:41.074326  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:26:41.114543  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 100905us
I20260709 12:26:41.119427  5869 raft_consensus.cc:2889] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:41.003139 10061 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:41.129148  6019 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:41.136032  5869 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:41.133410 10061 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:41.137768 10110 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:41.138144 10235 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:41.127375 10296 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:41.130867  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:41.073238 10419 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:26:41.165606  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 47031.2us
I20260709 12:26:41.162639  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:41.173218  6611 leader_election.cc:400] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14.
I20260709 12:26:41.173704  6611 leader_election.cc:403] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:41.173059  6785 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:41.172322  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:41.174099  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 262985us
I20260709 12:26:41.175026  6766 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 14
W20260709 12:26:41.174356  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 100139us
I20260709 12:26:41.162920  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:41.177856  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 47064.8us
I20260709 12:26:41.179183  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:41.179414  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:41.179971  5871 raft_consensus.cc:2393] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:41.180524  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:41.181973  5868 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:41.183138  5866 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:41.173106 10303 consensus_meta.cc:287] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.501s	user 0.006s	sys 0.010s
I20260709 12:26:41.184531  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:41.183640  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:41.186538  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:41.168429 10158 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:41.193029  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:41.165567 10296 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:41.188597  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:41.188669  6768 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
W20260709 12:26:41.176298 10440 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:26:41.198580  5871 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:26:41.172729 10419 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:41.195178  5840 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:41.173753 10439 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:41.196879  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:41.188184  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:41.216657  5868 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:41.219086  5869 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:41.226109 10439 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:41.227306 10439 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:41.218405 10399 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.229836 10399 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:41.231448  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.234618 10408 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:41.238807 10394 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:41.242075  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:41.243367 10447 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:41.245395  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:41.246948  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:41.249838 10308 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:41.237279 10419 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:41.185931 10110 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:41.252440  6773 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
W20260709 12:26:41.225700  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.175580 10300 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:41.186074 10298 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:41.268316  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 80203.5us
I20260709 12:26:41.271284  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:41.253286  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 66837us
I20260709 12:26:41.196007 10358 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:41.190783  9894 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:41.258205 10308 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.285087  9894 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:41.285701 10308 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:41.287734  9894 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:41.263752  6785 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:41.294723  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.295549  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.296033  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.297140 10414 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:41.299835  5945 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:26:41.197261 10335 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:26:41.301455 10296 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:41.302071  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:41.303320  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:41.304455 10399 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:41.264871  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:41.306175  6775 raft_consensus.cc:3055] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:41.306564  6775 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:41.192444 10175 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:41.254740  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:41.308368  6024 raft_consensus.cc:2410] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:26:41.192054 10032 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:26:41.308728 10294 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:41.293104 10409 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:41.279754  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.259371 10447 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:41.258452 10153 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:41.279480 10300 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:41.253448 10158 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:41.334264  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 79619.6us
I20260709 12:26:41.292297 10247 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:41.317860  6775 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:41.338517  6775 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:41.311942  6778 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 13: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:41.267527 10298 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:41.313431 10299 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.316426  6793 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:26:41.309285  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:41.273794 10333 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:41.296062  9883 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:41.263293 10110 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.263221 10434 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:41.342998  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:41.344064  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 72934.1us
W20260709 12:26:41.399572  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 90396.8us
I20260709 12:26:41.309324 10294 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.400775 10298 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:41.290462  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.430079  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:41.431504  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:41.432451  6019 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:26:41.337024 10153 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:41.433581  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 45657.6us
I20260709 12:26:41.423585  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:41.402206  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:41.342382 10175 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:41.466316  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.408589 10447 raft_consensus.cc:697] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:41.416046 10306 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:41.417789 10110 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:41.322933  6766 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
W20260709 12:26:41.491744  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 56114.6us
I20260709 12:26:41.341938  9949 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.121s
I20260709 12:26:41.493034  5990 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:41.498293  6775 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:41.415239 10229 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:26:41.463765 10032 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:41.502151 10229 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:41.494228 10276 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14.
I20260709 12:26:41.505851 10158 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:41.507014 10358 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:41.488564 10394 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.507819 10358 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:41.323370  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 128387us
I20260709 12:26:41.510914  5840 leader_election.cc:400] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14.
I20260709 12:26:41.511384  5840 leader_election.cc:403] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:41.415845 10299 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:41.461370 10153 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:41.404207 10247 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:41.489841 10243 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:41.419415 10294 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:41.518294  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.517246  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:41.520306  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.520725  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.516155  5990 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:41.521065  6766 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:41.523976  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:41.525728  6015 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:41.526760  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:41.522662 10406 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:26:41.534040 10405 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:41.535516  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:41.507504 10399 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:26:41.537756 10447 consensus_queue.cc:237] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:41.538548  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:41.516677 10449 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:26:41.507242 10409 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:41.546998  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:41.559043  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.547643  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:41.544706 10454 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:41.552090 10405 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.552520 10308 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:41.508104 10394 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:41.553835 10300 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:41.562048  6775 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:41.565114  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:41.579716  6775 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:41.556931  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.517902 10387 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:41.580638  6775 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:41.469767 10377 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 12:26:41.581568  6775 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:41.518187 10434 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14.
I20260709 12:26:41.541725 10333 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:41.582183  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:41.515177 10306 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:41.590226  5943 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:41.599278  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 137232us
I20260709 12:26:41.597550  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:41.583137 10452 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:41.584476 10335 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:41.601240  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:41.552207 10438 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:41.595698 10300 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:41.624189  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:41.628021  6775 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:26:41.585994 10153 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:41.598815 10333 raft_consensus.cc:697] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:41.608817 10452 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.609519 10376 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:41.620451 10387 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:41.644486  6757 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 11: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:41.640702  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:41.582441 10405 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:41.653726  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.655424 10405 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:41.619897 10296 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:41.656888  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.593390 10451 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:26:41.648895  5943 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:41.666826  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:41.668922 10447 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:41.672246 10175 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:41.651825 10247 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:41.677798 10387 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:41.680263 10153 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:41.620522 10335 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:41.702327 10335 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:41.675028  6609 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:41.703882 10419 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:41.644094 10158 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:41.704406 10356 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:41.705025 10356 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:41.705951 10356 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:41.706722 10438 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:41.708488 10394 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:41.708856 10358 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:41.644052 10294 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:41.711400 10358 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:41.712935 10294 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:41.692951  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:41.714133  6017 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:41.714459  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:41.714026 10333 consensus_queue.cc:237] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:41.715372  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:41.715919  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:41.704732 10158 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:41.717540 10451 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:41.718124 10451 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:41.719475 10158 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:41.720142  5870 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:41.719048 10451 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:41.715071  6022 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:41.696254  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:41.724139  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:41.724550  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:41.725835  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.647360 10452 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:41.731981  5948 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:41.733505  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:41.734400  5949 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:41.734481 10308 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:41.754957  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:41.756084  5943 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:41.760201  6022 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:41.762904  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:41.766284  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:41.777876  6022 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:41.779172  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:41.783731 10408 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:26:41.795288  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:41.796020  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:41.801079  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:41.809782  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 467980us
I20260709 12:26:41.793095 10296 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:41.691798 10306 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:41.793161 10299 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:41.744954 10110 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:41.828348 10110 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:41.831549  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:41.835618  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:41.807123 10298 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:41.862048  5945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda"
dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:48330
I20260709 12:26:41.862802  5945 raft_consensus.cc:606] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:41.863166  5945 raft_consensus.cc:614] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:41.876940  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 1020) took 1073 ms. Trace:
I20260709 12:26:41.878400  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:41.877527  6757 rpcz_store.cc:276] 0709 12:26:40.803277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.865771 (+ 62494us) service_pool.cc:224] Handling call
0709 12:26:41.876704 (+1010933us) spinlock_profiling.cc:243] Waited 742 ms on lock 0x7b5400004954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:41.876860 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1706449664}
I20260709 12:26:41.881778  6019 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:41.883671  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:41.885293  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:41.880009  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:41.881615  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:41.889566  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:41.886173  5946 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:26:41.892675  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:41.893285  5948 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:41.902240  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:41.902983  5949 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:41.904834  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:41.905543  5943 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:41.919565  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.899379 10475 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.934106 10475 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:41.939625  5948 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:41.944725  6021 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:41.960983 10475 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:41.962604  5949 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:41.963152 10298 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:41.966495  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:41.968442  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:41.969964  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:41.970674  5873 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 11
W20260709 12:26:41.972539  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.973089  5865 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:41.975960  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:41.975438 10282 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:41.978284  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:41.978471 10282 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:41.980765 10282 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:41.976754  6020 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
W20260709 12:26:41.985567  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 343093us
I20260709 12:26:41.986853  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:41.992197  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:41.997299  6020 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:41.997437  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:42.003691  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:42.004675  5865 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:42.006196  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:42.015573  5942 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:42.027920  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:42.029853  6020 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:42.031867  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:42.032052  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:42.032657  5949 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:42.033080  5943 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:42.033234  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:42.038573  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:42.040860  5873 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:42.042158  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:42.042861  5865 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:42.044195  5913 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:42.046082  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:42.046849  5874 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:42.048532  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:42.050434  6017 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:42.054821  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:42.060117  5913 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:42.033545 10337 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:42.068630  5913 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:42.049350 10333 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:42.082288  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:26:42.065954 10474 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:26:42.056975 10243 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:42.063336 10475 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:42.002902 10452 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:42.031355 10291 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:26:42.065555 10406 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:42.068678 10153 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:42.088274  5913 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:42.074769 10337 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:42.071175 10293 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:42.121317  5871 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:26:42.064512 10306 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:26:42.086050 10459 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:42.094244 10377 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:42.090803 10110 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:42.071538 10434 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
W20260709 12:26:42.193104  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 62034us
W20260709 12:26:42.194283  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.194993  6611 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:42.086859 10358 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 12:26:42.120361 10337 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:42.206833  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 154246us
W20260709 12:26:42.196795 10486 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:26:42.192912 10061 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:42.208912  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:42.194859 10474 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:26:42.212450  6611 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:42.176388 10442 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:42.196791 10406 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.216725  5987 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:42.218478 10377 raft_consensus.cc:697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:42.221405 10480 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
W20260709 12:26:42.221571 10481 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:26:42.197044 10376 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:42.232806 10377 consensus_queue.cc:237] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:42.204429 10115 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:26:42.238359  9883 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:42.223551  9953 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:26:42.229800 10406 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:42.241110 10483 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:26:42.243137  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 1018) took 1440 ms. Trace:
I20260709 12:26:42.243578  6762 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:42.219847 10061 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:42.250126  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:42.218526 10438 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:42.197842 10110 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:42.220762 10442 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.296170  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:42.297874  5946 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:42.325377  6773 rpcz_store.cc:276] 0709 12:26:40.802703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.803562 (+   859us) service_pool.cc:224] Handling call
0709 12:26:42.242898 (+1439336us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54002a17d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:42.243065 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":763857152}
I20260709 12:26:42.371743  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:42.381824  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.362754  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:26:42.327545  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 77504.6us
I20260709 12:26:42.415879  5942 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:42.333235  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:42.416954  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:42.419575  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:42.422897  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:26:42.426550  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 214222us
W20260709 12:26:42.427925  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 180174us
I20260709 12:26:42.282769 10387 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:42.429479  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:42.430449  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:42.430619  5946 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:42.431136  5943 raft_consensus.cc:2376] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14. Re-sending same reply.
I20260709 12:26:42.435590  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:42.372012 10491 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:26:42.441414  5874 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:42.357741 10306 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:42.418531 10438 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.323293 10294 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.451270  5942 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:26:42.439538  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:42.385274 10452 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:42.352367 10475 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:42.438683 10387 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.490703  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:42.372921 10494 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.444259 10453 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:42.328773  9883 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:42.421213 10442 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:42.327913 10333 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:42.373395 10406 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:42.420434 10243 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:42.352053 10393 raft_consensus.cc:3055] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:26:42.484652  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:42.486418  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.504380  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:42.480252 10438 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:42.372423 10158 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.104s
W20260709 12:26:42.476877 10486 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 12:26:42.497176  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 207781us
I20260709 12:26:42.551473  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:42.291746 10354 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:42.439162 10298 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:42.283730 10376 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:42.377372 10312 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:42.446121 10356 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
W20260709 12:26:42.554847  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 57922.7us
I20260709 12:26:42.493839 10387 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:42.481439 10306 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:26:42.567319  6024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1113) took 1956 ms (client timeout 1861 ms). Trace:
I20260709 12:26:42.505162 10393 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:42.502385 10453 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.489543 10294 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:42.576212  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:42.576654  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 137203us
I20260709 12:26:42.553519 10243 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:26:42.586860  6024 rpcz_store.cc:269] 0709 12:26:40.610896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.611088 (+   192us) service_pool.cc:224] Handling call
0709 12:26:42.566819 (+1955731us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b540041da54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:42.567226 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1599872256}
I20260709 12:26:42.557659 10354 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:42.561367 10298 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.512887  9883 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:42.498937 10475 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:42.502018 10494 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:42.502367 10501 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 12:26:42.558944 10502 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:42.617283  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:42.571394 10406 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:42.512779 10333 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:42.622433  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 117969us
I20260709 12:26:42.589996 10306 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:42.629151  5945 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:26:42.629354  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 53275.9us
I20260709 12:26:42.631311  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:42.653177  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:42.598305 10453 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:42.622406 10354 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:42.622673 10312 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:42.622121 10298 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:42.627369 10450 raft_consensus.cc:3055] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:42.594020 10503 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:26:42.650689 10061 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:42.666704 10497 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.161s	user 0.000s	sys 0.008s
I20260709 12:26:42.658116 10417 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:42.662830 10399 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:42.672317 10508 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:42.684273 10356 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:42.684203 10508 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:42.685292 10424 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:42.668896 10450 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:42.692075 10499 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 12:26:42.690560 10500 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:26:42.696440  6021 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:42.704250 10110 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:42.704648  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 849) took 1602 ms. Trace:
I20260709 12:26:42.705166  5866 rpcz_store.cc:276] 0709 12:26:41.102508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.121938 (+ 19430us) service_pool.cc:224] Handling call
0709 12:26:41.160210 (+ 38272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.187718 (+ 27508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.704369 (+1516651us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b54004a0454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:42.704581 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52267392}
I20260709 12:26:42.705745 10475 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:42.680485 10417 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.707389 10417 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:42.734706  9883 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:42.736617 10494 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:42.665459 10393 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:42.742137 10333 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:42.718858 10450 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:42.748199 10294 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:42.745908 10517 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:42.750474 10517 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:42.750957 10517 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:42.751356 10376 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:42.798132 10517 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:42.802752 10517 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:42.805929 10296 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:42.806811 10296 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:42.807976 10084 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:42.711262 10459 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
W20260709 12:26:42.774071 10506 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:26:42.809490 10296 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:42.822075 10337 consensus_meta.cc:287] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.585s	user 0.009s	sys 0.004s
I20260709 12:26:42.831471 10337 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:42.832865 10473 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.911s	user 0.000s	sys 0.011s
W20260709 12:26:42.834553 10490 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.601s	user 0.002s	sys 0.000s
I20260709 12:26:42.814381 10508 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:42.838816 10376 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.841699 10354 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:42.826232 10406 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:26:42.853266  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 405987us
I20260709 12:26:42.826893 10175 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:42.881387  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 182933us
I20260709 12:26:42.848503 10387 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:42.887321  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 224957us
I20260709 12:26:42.905525  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:42.909044  5949 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:42.845230 10417 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:42.880741 10300 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:42.914479 10419 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:26:42.922999 10494 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:42.917136 10309 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:42.923889 10494 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:42.924598 10309 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:42.925158 10309 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:42.889868 10376 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:42.939002 10300 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:42.900741 10175 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:42.926162 10512 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:26:42.972880  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:42.976612  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:42.977799  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:42.896672 10406 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:42.994752  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:42.996724  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:42.997994  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 1040) took 1327 ms. Trace:
I20260709 12:26:42.998430  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:43.000452  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:43.003643  5869 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:42.998849  6766 rpcz_store.cc:276] 0709 12:26:41.670459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.670590 (+   131us) service_pool.cc:224] Handling call
0709 12:26:41.670894 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.671202 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.671227 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:41.671602 (+   375us) log.cc:844] Serialized 26 byte log entry
0709 12:26:41.673355 (+  1753us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:41.673404 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.673490 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.832141 (+1158651us) spinlock_profiling.cc:243] Waited 4.64 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:42.932199 (+100058us) raft_consensus.cc:1685] finished
0709 12:26:42.932227 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.997916 (+ 65689us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":10665344,"thread_start_us":1408,"threads_started":1,"wal-append.queue_time_us":190021}
W20260709 12:26:43.005095  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 88922.8us
I20260709 12:26:43.010332  5873 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:43.014333  5949 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:43.014923  5943 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:43.014433  5941 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:26:43.020047  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.020951  6021 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:43.022740  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.022984  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:43.023452  6021 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 8.
I20260709 12:26:43.023806  5940 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:42.908025 10354 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:42.991931 10110 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:42.992908 10424 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:43.029736  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 476469us
W20260709 12:26:43.034124  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 371559us
W20260709 12:26:43.037222  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.036960  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:43.041025  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:43.043015  6015 raft_consensus.cc:2368] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17.
I20260709 12:26:42.981381 10430 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:26:43.042821  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.048249  5869 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:26:43.048269  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:43.050469  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:43.051342  5867 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:43.052178  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:43.053818  5838 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:43.055176  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:43.066424  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.007356 10438 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:43.102072  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:43.101796  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 48082.2us
I20260709 12:26:43.105041  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:43.106107  5866 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:43.123375  5870 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:43.106894  5915 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:43.135284  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:43.051764 10061 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:43.139995  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:43.140341  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:43.140033  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:43.144204 10523 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.139921 10061 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:43.152479  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 43648.6us
I20260709 12:26:43.149248  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:43.024964 10442 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:43.151480  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:43.067979 10452 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:43.156795  5867 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:43.181869  6766 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:43.165010  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:43.158424  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:43.184227  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:43.184456  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44734.7us
I20260709 12:26:43.184602  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1498) took 1263 ms. Trace:
I20260709 12:26:43.187229  5871 rpcz_store.cc:276] 0709 12:26:41.920945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.921227 (+   282us) service_pool.cc:224] Handling call
0709 12:26:41.921736 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.124805 (+203069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.184259 (+1059454us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b5400144bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:43.184515 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":458245504}
I20260709 12:26:43.035017 10110 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:43.043289 10424 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:43.053279 10298 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:43.175311 10358 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:43.205581  6611 leader_election.cc:400] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17.
I20260709 12:26:43.207319  6611 leader_election.cc:403] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:26:43.180707 10475 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:43.195684 10399 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.181197 10296 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:43.211025  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 107123us
I20260709 12:26:43.183890 10333 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:43.180811 10387 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:43.214711  5838 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:43.221365  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 211896us
I20260709 12:26:43.055126 10453 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:43.226823  6016 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:43.227526  5868 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 9: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:43.195925 10337 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:43.205547 10312 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:43.067528 10406 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:43.200677 10494 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:43.255050  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 47358us
I20260709 12:26:43.201112 10354 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
W20260709 12:26:43.262377  6609 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.795s (SENT) [suppressed 10 similar messages]
I20260709 12:26:43.225461 10413 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:43.155973 10523 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:43.214386 10158 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:26:43.286093  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.286983  5949 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:43.231052 10333 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:43.289983 10333 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:43.290341 10509 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:43.290831 10509 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:43.291267 10509 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:43.291894 10453 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:43.292892 10417 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:43.226099 10399 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:43.293800 10529 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:26:43.295576 10438 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:43.296823 10522 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:26:43.296968 10503 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.297641 10503 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:43.298245 10520 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:43.298892 10520 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:43.300422 10503 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:43.301105 10399 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:43.297984 10110 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:43.301942 10528 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:43.304399  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1148) took 1877 ms. Trace:
W20260709 12:26:43.304898 10525 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:26:43.260739 10494 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:43.267860 10512 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:43.267207 10439 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:43.306316  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 925) took 1640 ms. Trace:
I20260709 12:26:43.306989 10439 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:43.306881  6775 rpcz_store.cc:276] 0709 12:26:41.665535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.665910 (+   375us) service_pool.cc:224] Handling call
0709 12:26:41.666351 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.666715 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.666759 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:41.669187 (+  2428us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.733490 (+1064303us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b1800080a40. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:26:42.981114 (+247624us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:42.981194 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.981307 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:43.305953 (+324646us) raft_consensus.cc:1685] finished
0709 12:26:43.305978 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.306241 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1243,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":23510,"spinlock_wait_cycles":7465856,"thread_start_us":247374,"threads_started":1,"wal-append.queue_time_us":1509892}
W20260709 12:26:43.308625  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 55634.8us
I20260709 12:26:43.310253  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 983) took 1773 ms. Trace:
I20260709 12:26:43.307870  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:43.310868  6022 rpcz_store.cc:276] 0709 12:26:41.537160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.811864 (+274704us) service_pool.cc:224] Handling call
0709 12:26:41.818692 (+  6828us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.820052 (+  1360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.820135 (+    83us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:41.822236 (+  2101us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.841736 (+1019500us) spinlock_profiling.cc:243] Waited 6.84 ms on lock 0x7b1800137f20. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:26:43.160279 (+318543us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:43.160338 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.160412 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:43.309894 (+149482us) raft_consensus.cc:1685] finished
0709 12:26:43.309927 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.310175 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":215,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":3087,"spinlock_wait_cycles":15733120,"thread_start_us":318279,"threads_started":1,"wal-append.queue_time_us":1422080}
W20260709 12:26:43.318218  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 62046.8us
W20260709 12:26:43.280227 10306 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.614s	user 0.008s	sys 0.008s
I20260709 12:26:43.304908  6023 rpcz_store.cc:276] 0709 12:26:41.427318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.429712 (+  2394us) service_pool.cc:224] Handling call
0709 12:26:41.488521 (+ 58809us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.494395 (+  5874us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.494448 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:41.495234 (+   786us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.391349 (+896115us) spinlock_profiling.cc:243] Waited 9.05 ms on lock 0x7b18001f8c40. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:26:42.852671 (+461322us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:42.852741 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.852856 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:43.303929 (+451073us) raft_consensus.cc:1685] finished
0709 12:26:43.303956 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.304323 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1631,"raft.queue_time_us":9835,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":12084,"spinlock_wait_cycles":20813568,"thread_start_us":461032,"threads_started":1,"wal-append.queue_time_us":1523730}
I20260709 12:26:43.322124 10306 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:43.311021  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:43.324165 10376 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:43.228626 10296 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:43.244763 10426 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:26:43.312461 10406 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:43.347318  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 141911us
I20260709 12:26:43.314280 10439 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:43.326032 10158 consensus_queue.cc:1243] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:43.278560 10337 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:26:43.355675  5915 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:43.330765 10306 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:43.289829 10507 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 12:26:43.334595 10520 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:43.357841 10084 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.549s	user 0.011s	sys 0.004s
I20260709 12:26:43.387530  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.369607 10541 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:43.378750 10424 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:43.379660 10061 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:43.370290 10532 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:26:43.383034  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:43.384356  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.385818  5915 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:43.381260  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 118909us
W20260709 12:26:43.365700 10527 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.161s	user 0.001s	sys 0.001s
I20260709 12:26:43.320665 10309 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:43.393127  6022 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:43.393361  6018 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:43.399667  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.349373 10459 consensus_queue.cc:1059] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:26:43.390836  6762 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:26:43.313349 10413 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:43.404836  6020 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:43.370898 10531 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.418107 10531 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:43.420692 10531 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:43.423214 10413 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:43.373946 10505 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:43.426236 10298 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:43.427035 10312 raft_consensus.cc:3055] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:43.427534 10312 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:43.428491 10312 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:43.373828 10537 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:43.430117 10537 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:43.431887 10309 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:43.407130 10276 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.434115 10276 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:43.435355 10444 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.364s	user 0.000s	sys 0.004s
I20260709 12:26:43.401938 10541 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.437615  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.438120 10541 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:43.438567  6017 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
W20260709 12:26:43.440903  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 44428.1us
I20260709 12:26:43.441056  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:43.412623  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.441784 10298 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:43.416990 10442 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:43.442664  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.443197 10454 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:43.442417 10298 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:43.410622 10523 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:43.445211  5915 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:43.446051  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:43.444792  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.466002  6015 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:43.391528 10377 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:43.471060  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:43.474382  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.403539 10475 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:43.432840 10537 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:43.479781  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:43.481884 10536 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:43.482489 10536 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:43.483529 10536 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:43.484980 10536 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:43.470667 10534 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:26:43.487747 10298 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:43.466315 10338 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:43.489619 10338 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:43.492340 10338 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:43.471416 10377 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:43.493288 10377 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:43.478650 10358 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:43.466372 10395 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:43.443749 10454 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:43.495502 10276 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:43.496923 10454 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:43.497562 10300 consensus_meta.cc:287] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.004s
I20260709 12:26:43.497743 10358 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:43.498100 10300 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:43.498371 10358 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:43.501240 10300 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:43.501428 10358 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:43.508232  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:43.521587  6022 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:43.478009 10475 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.506563 10084 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:43.504881  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 58946.9us
I20260709 12:26:43.524065  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:43.504633  6015 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 12: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:43.501847  6017 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:43.526038  6611 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:43.527861  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.529430  6015 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:43.522998  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.534926  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.532627  6611 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:43.543982  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.467267 10507 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:26:43.493296 10459 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.553462  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:43.524668 10475 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:43.527647 10413 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:43.524183 10517 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:43.554951 10338 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:43.523782 10282 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:26:43.546293 10528 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:43.546682 10276 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:43.584168 10528 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:43.584614 10276 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:43.585109 10276 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:43.585426 10528 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:43.494769 10395 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:43.565272 10338 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.579737 10541 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:43.558904 10300 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:43.598572  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.598939  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:43.599471  5940 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:43.600584  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:43.601122  5940 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:43.599162  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.603192  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:43.604452  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.605015  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:43.607015  5942 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:43.608193  5949 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:43.609688  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.610752  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:43.611451  5867 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:43.617865  5940 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:26:43.610842  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.563580 10413 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.627039  5941 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:43.621192  5867 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:43.624655  5874 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:26:43.630098  6018 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:43.634872  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:43.636391  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:43.565198 10545 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:26:43.627108 10175 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.618s	user 0.011s	sys 0.004s
I20260709 12:26:43.597223 10338 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:43.655810  5913 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:43.633700 10413 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:43.627465 10512 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17.
I20260709 12:26:43.563067 10546 consensus_queue.cc:1059] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 12:26:43.567937 10517 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:43.561448 10459 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:43.649194 10513 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.643421 10454 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:43.657058 10523 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:26:43.660579 10532 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:26:43.663193 10531 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:26:43.673738  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.674600 10512 raft_consensus.cc:3055] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:43.676612 10517 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:43.649436 10384 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:43.678836  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.685593 10459 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:43.686947 10450 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:43.694386  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.699579  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.746837  6024 raft_consensus.cc:2410] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:26:43.741004  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1508) took 1359 ms. Trace:
I20260709 12:26:43.626163 10300 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:43.731875  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:43.732142  5987 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:43.659819 10175 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:43.706070  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 964) took 2972 ms. Trace:
I20260709 12:26:43.749783  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.745291 10084 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:26:43.724737 10466 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.497s	user 0.018s	sys 0.027s
I20260709 12:26:43.751662  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.760982  5949 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:43.756043  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.747637  5944 rpcz_store.cc:276] 0709 12:26:42.370159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.370356 (+   197us) service_pool.cc:224] Handling call
0709 12:26:42.370761 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.627067 (+256306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.627099 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.627141 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:42.627174 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.728704 (+1101530us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b5400833954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:43.728780 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.728945 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588878720}
W20260709 12:26:43.758005 10550 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.097s	user 0.001s	sys 0.001s
I20260709 12:26:43.759342  6016 rpcz_store.cc:276] 0709 12:26:40.721089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.721257 (+   168us) service_pool.cc:224] Handling call
0709 12:26:42.567198 (+1845941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:43.234122 (+666924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.691728 (+457606us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b540041da54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:43.693186 (+  1458us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b540041da50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:43.693537 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5721470592}
I20260709 12:26:43.764693  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.725679 10512 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:43.744122 10384 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:43.743398 10531 raft_consensus.cc:697] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:43.764344 10466 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.539s	user 0.018s	sys 0.027s
I20260709 12:26:43.778029  5948 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:43.762928 10276 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:43.772686  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:43.781813 10556 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:26:43.781687  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.759361 10395 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:43.789047  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.793177 10300 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:43.794763 10276 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:43.730665 10509 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:43.803392 10175 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:43.805158  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:43.706938 10513 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:43.811256 10513 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:43.767985 10084 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:43.817440 10084 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:43.817905 10541 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:43.780997 10384 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:43.823972  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 51366.7us
I20260709 12:26:43.816841 10377 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:43.782241 10496 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:43.835846  6023 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:43.837028 10509 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:43.839183 10523 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:43.847499  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.848913 10175 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.853854 10512 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:43.878804 10512 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:43.880759 10557 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:26:43.864729 10496 raft_consensus.cc:697] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:43.884321 10496 consensus_queue.cc:237] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:43.886579 10562 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
W20260709 12:26:43.897009  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.897433  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1210) took 1198 ms. Trace:
I20260709 12:26:43.861250 10523 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:43.897854  5945 rpcz_store.cc:276] 0709 12:26:42.698822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.699013 (+   191us) service_pool.cc:224] Handling call
0709 12:26:42.722022 (+ 23009us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.722540 (+   518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.722582 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.738774 (+ 16192us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.578148 (+839374us) spinlock_profiling.cc:243] Waited 25.6 ms on lock 0x7b1800440a40. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:26:43.710005 (+131857us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:43.710052 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.710133 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:43.887795 (+177662us) raft_consensus.cc:1685] finished
0709 12:26:43.887822 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.897330 (+  9508us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8391,"raft.queue_time_us":3478,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":12808,"spinlock_wait_cycles":58838272,"thread_start_us":131601,"threads_started":1,"wal-append.queue_time_us":1057099}
I20260709 12:26:43.843813 10531 consensus_queue.cc:237] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:43.868999 10175 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:43.936830  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:43.953507 10509 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.959201 10509 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:43.953506 10450 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.962118 10450 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:43.963024 10312 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:43.964269 10450 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:43.965009 10523 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:43.961367 10509 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:43.970929 10175 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:43.972572  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:43.974234  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.974277  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.975163  6017 raft_consensus.cc:2393] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:43.979445 10494 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:43.979530  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:43.980114 10494 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:43.980412  5941 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:43.982318  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:43.983418 10393 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:43.932933 10540 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:43.991868  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:43.984297  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:43.985443  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:43.985576  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:43.985996  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:43.958518 10377 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:43.989055  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.989868  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.996068  6022 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:43.990967  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:43.991564  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:43.994416  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:43.995810  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:43.997753  5838 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:43.997991  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:43.991020  6017 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:44.000553  5870 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:43.998553  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:44.001979  6775 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:44.002728  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:44.005309  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:44.005987 10561 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:44.006656 10561 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:44.008245 10567 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:26:44.008929 10561 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:44.009604 10541 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:44.010046 10541 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.010423 10541 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:44.011181 10405 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:44.013017 10158 raft_consensus.cc:3055] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:44.013466 10158 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:44.014461 10158 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:44.016534  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:44.018675  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:44.019064  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.025923  5941 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:43.981086 10494 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:43.980448 10504 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:44.043341  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:44.049754 10566 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:26:44.063226 10309 raft_consensus.cc:3055] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:44.064169 10309 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:44.065313 10309 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:44.066761 10309 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:44.069180 10512 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:44.002221 10540 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:44.070035 10512 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:44.070703 10356 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:44.071497 10356 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:44.072052 10540 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:44.072721 10540 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:44.073096 10540 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:44.074239 10512 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:44.076433 10501 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:44.077126 10450 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.121977  6785 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:44.078430  6775 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:44.082685  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.085354  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.073475 10540 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:44.110975  6766 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:44.127841  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.110976  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:44.116803  6762 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:44.131845  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:44.133687  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.077544 10518 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
W20260709 12:26:44.138909  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 122432us
I20260709 12:26:44.139593  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.145756  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:44.146371  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:44.147873  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.141359  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:44.151021 10568 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.193s	user 0.000s	sys 0.000s
W20260709 12:26:44.171600  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 173928us
I20260709 12:26:44.106865 10356 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:44.055084 10504 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:44.181598  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.183405  6773 raft_consensus.cc:606] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:26:44.183897  6773 raft_consensus.cc:614] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:44.090480 10413 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:44.189694  6766 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:44.195638  6768 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
W20260709 12:26:44.196246  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 50636us
I20260709 12:26:44.199358 10451 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:44.199513 10413 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:44.200065 10451 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.197539  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:44.138388 10450 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:44.130452 10501 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:44.203802  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.190601 10523 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.217010  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.200636 10451 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:44.225360  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:44.227629  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.228533  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.229650  6775 raft_consensus.cc:2410] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:26:44.232494  6766 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:44.233366  5990 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:44.238231 10579 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:44.240360 10581 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:44.241010 10581 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:44.221560 10338 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.003s
I20260709 12:26:44.241433 10581 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:44.251277  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:44.265048  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:44.260759  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 58534.9us
I20260709 12:26:44.267592  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.252620 10515 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:44.270835  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:44.274248  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.223105 10523 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:44.276135  5943 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:44.271397  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:44.244516 10511 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:26:44.248664 10300 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:26:44.249214 10524 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:26:44.252278 10384 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:44.290800 10384 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:44.292204 10579 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:44.294412 10573 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:26:44.296037  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 998) took 1258 ms. Trace:
I20260709 12:26:44.275262 10580 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:44.296433  6021 rpcz_store.cc:276] 0709 12:26:43.037793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.037946 (+   153us) service_pool.cc:224] Handling call
0709 12:26:43.038418 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:43.134736 (+ 96318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:43.134771 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:43.153435 (+ 18664us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.959864 (+806429us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b1800236740. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:26:44.089839 (+129975us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:44.089886 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.089973 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.295654 (+205681us) raft_consensus.cc:1685] finished
0709 12:26:44.295686 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.295961 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95700,"raft.queue_time_us":113827,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":34379,"spinlock_wait_cycles":6867328,"thread_start_us":129737,"threads_started":1,"wal-append.queue_time_us":1019546}
W20260709 12:26:44.298431 10475 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.733s	user 0.014s	sys 0.000s
I20260709 12:26:44.299010 10475 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:44.299988 10450 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:44.253106 10338 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:44.302191  5940 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 12:26:44.318710  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.302671  6768 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:44.346873  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.347798  6768 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:44.254417 10434 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:26:44.300860 10541 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:44.358206 10576 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:26:44.371445  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 788) took 1349 ms. Trace:
I20260709 12:26:44.441722  5948 rpcz_store.cc:276] 0709 12:26:43.022238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.022369 (+   131us) service_pool.cc:224] Handling call
0709 12:26:43.032688 (+ 10319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:43.783869 (+751181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.371153 (+587284us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54004678d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:44.371365 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1713137920}
I20260709 12:26:44.383334 10243 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:44.397500 10526 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:44.443277 10526 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:44.443827 10526 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:44.443320 10243 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:44.398554 10501 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:44.399500 10504 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:44.352250 10580 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:44.406278  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1164) took 1007 ms. Trace:
I20260709 12:26:44.446666  6020 rpcz_store.cc:276] 0709 12:26:43.398579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.400424 (+  1845us) service_pool.cc:224] Handling call
0709 12:26:43.404380 (+  3956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:43.405684 (+  1304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.406064 (+1000380us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b5400172ad0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:44.406212 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8176512}
I20260709 12:26:44.447142 10580 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 3, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:44.405202  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:44.448401  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.450460  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:44.451212  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.451543  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:44.451784  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:44.452236  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:44.416312  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:44.416453  5944 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:44.423615  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55542 (request call id 21) took 1919 ms. Trace:
I20260709 12:26:44.454627  6773 rpcz_store.cc:276] 0709 12:26:42.503933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.504139 (+   206us) service_pool.cc:224] Handling call
0709 12:26:44.423402 (+1919263us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b5400005854. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c7d853 00007f39d7c4b60a 00007f39d11d17d4 00007f39d11d1762 00007f39d11d1691 00007f39d11d160c 00007f39d11d08c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:44.423522 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3851250432}
I20260709 12:26:44.430238  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:44.456136  5947 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:44.430877  6793 raft_consensus.cc:2368] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15.
I20260709 12:26:44.457151 10515 raft_consensus.cc:3055] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:44.457607 10515 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:44.458782  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:44.458568 10515 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:44.459925 10515 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:44.462045  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:44.462975  5947 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:44.420804 10494 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:44.433117 10296 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:44.513607  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 238856us
W20260709 12:26:44.514851  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.515779  5990 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:44.312788 10475 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:44.446038 10243 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:44.521952  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:44.524943  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:44.525616  5871 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:44.526324  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:44.518510  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:26:44.449282  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:44.456430 10545 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:44.475131  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:44.478907  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:44.486774  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:44.300071 10523 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:44.511298  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:44.496117 10553 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.585749  6023 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:44.494589 10501 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.520648  5990 leader_election.cc:400] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15.
I20260709 12:26:44.519104  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:44.527454  6020 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:44.524295  5873 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:26:44.529841  5867 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:44.432363 10338 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:44.494513 10504 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:44.595996  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:44.519050 10414 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.514564 10294 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:44.598670  5866 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:44.642691  5990 leader_election.cc:403] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:44.612067  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.530120 10384 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:44.658578  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:44.520229  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.660564  5940 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:44.581459 10434 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:26:44.560705 10312 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 12:26:44.596869 10553 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:44.547260 10413 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.527922 10426 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:44.668532  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 147993us
I20260709 12:26:44.668828  5871 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:44.538527  5943 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 9.
I20260709 12:26:44.642504 10541 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:44.635928 10561 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:44.523268 10578 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.708084  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:44.582563 10405 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.118s
I20260709 12:26:44.637981 10450 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:44.713132  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:44.713207  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:44.715205  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:44.721696  6021 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:44.698434 10519 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:26:44.724227  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:44.668852 10384 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.738191  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.711542 10243 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.705914 10406 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.657444 10504 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:44.755995  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:44.758545  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:44.651631 10501 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:44.774485 10158 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:44.776247  6778 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:44.777738  6766 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:26:44.670084 10414 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:44.781916 10579 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:44.733564 10384 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:44.701481 10451 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:44.721125 10578 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:44.782449 10296 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:44.696152 10294 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:44.710134 10582 consensus_queue.cc:1059] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:26:44.791203 10406 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:44.794639 10553 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:44.710527 10426 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:44.712822 10561 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:44.710671 10413 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:44.795735 10243 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:44.874802  5945 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
W20260709 12:26:44.747072 10309 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.680s	user 0.012s	sys 0.003s
I20260709 12:26:44.801021  6023 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
W20260709 12:26:44.811425  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.879053  6020 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: [14.9-15.10]
I20260709 12:26:44.803150  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:44.880506  5940 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:44.881839  5949 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:44.882932  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:44.820742 10338 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:44.816875  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:44.557473  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 286121us
I20260709 12:26:44.835148  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:44.885421  5867 raft_consensus.cc:2410] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 12:26:44.827154  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 306654us
W20260709 12:26:44.860108  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.836963  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:44.877326  5866 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:44.888778  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:44.820262 10579 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:44.891163  6778 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:44.828850 10296 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:44.892992 10243 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:44.827365 10523 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:44.894049 10523 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:44.896263 10523 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:44.899385 10406 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:44.899755 10578 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:44.827625 10393 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:44.900327 10583 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:44.900808 10583 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:44.901257 10583 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:44.903322 10294 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:44.905989  6773 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:26:44.906474 10561 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:44.907233 10584 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:44.907867 10584 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:44.908939 10426 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:44.910907 10306 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:44.912606  6756 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
W20260709 12:26:44.826961 10577 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.011s	sys 0.019s
I20260709 12:26:44.917609  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.920351  5840 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:44.927196  6762 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:26:44.931595  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:44.929373  5990 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:44.904995  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:44.951839  5990 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:44.952409  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.950417  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:44.954377  6020 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:26:44.955317  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.950239  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:44.876634 10504 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:44.926692 10577 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.013s	sys 0.022s
I20260709 12:26:44.958443  5945 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:26:44.900343 10393 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:44.935382 10467 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:26:44.943077 10413 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:44.962610  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:44.973708 10505 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:44.974383  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:44.974474 10505 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:44.965075  5990 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:44.967293  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:44.969372  5696 catalog_manager.cc:5697] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 9 to 10, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 10 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:44.970650 10515 consensus_meta.cc:287] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.510s	user 0.006s	sys 0.006s
I20260709 12:26:44.963259 10434 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:44.964000 10561 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:44.970989  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:44.949479 10582 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:44.996886  6020 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:44.973757  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:44.977411  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:44.931015 10406 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:45.002230  6019 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
W20260709 12:26:44.977070  5694 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40818 (request call id 148) took 46701 ms (client timeout 15000 ms). Trace:
I20260709 12:26:45.039740  5949 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:44.986529 10526 consensus_meta.cc:287] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.004s
I20260709 12:26:44.960027 10243 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 12:26:44.985108 10523 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:44.949822 10541 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:45.083835  6022 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:45.001449 10467 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:45.039988  5696 catalog_manager.cc:5697] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:26:44.932861 10338 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:44.978178 10509 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:45.084098  5838 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:45.085830  6020 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 9.
I20260709 12:26:45.085873  6024 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:44.891268 10579 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:45.009953 10561 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:45.091203 10509 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:45.099952 10509 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:45.100045 10467 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:45.100529 10467 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:45.104817 10494 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:45.105305 10494 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.106344 10338 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:45.107625 10494 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:45.037909  5694 rpcz_store.cc:269] 0709 12:25:58.275676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.276001 (+   325us) service_pool.cc:224] Handling call
0709 12:25:59.230798 (+954797us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b540077f954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bed090 00007f39d2c8bf3a 00007f39d8a41e9c 00007f39d8b4004e 00007f39d70012a4 00007f39d7001232 00007f39d7001161 00007f39d70010dc 00007f39d7000392 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:44.976938 (+45746140us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:39.544217 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:39.545184 (+   967us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:39.545271 (+    87us) write_op.cc:435] Acquired schema lock
0709 12:26:39.545304 (+    33us) tablet.cc:662] Decoding operations
0709 12:26:39.899414 (+354110us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:39.899572 (+   158us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:39.899616 (+    44us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:26:39.901580 (+  1964us) tablet.cc:701] Row locks acquired
0709 12:26:39.922488 (+ 20908us) write_op.cc:260] PREPARE: finished
0709 12:26:39.993250 (+ 70762us) write_op.cc:270] Start()
0709 12:26:39.993840 (+   590us) write_op.cc:276] Timestamp: P: 1783599999936477 usec, L: 0
0709 12:26:39.993889 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 12:26:39.994790 (+   901us) log.cc:844] Serialized 7025 byte log entry
0709 12:26:40.374255 (+379465us) op_driver.cc:464] REPLICATION: finished
0709 12:26:40.562338 (+188083us) write_op.cc:301] APPLY: starting
0709 12:26:40.576059 (+ 13721us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:41.165936 (+589877us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:41.182571 (+ 16635us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:41.931653 (+749082us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:41.932063 (+   410us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 12:26:41.932113 (+    50us) write_op.cc:312] APPLY: finished
0709 12:26:41.932615 (+   502us) log.cc:844] Serialized 163 byte log entry
0709 12:26:42.090350 (+157735us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:42.157422 (+ 67072us) write_op.cc:454] Released schema lock
0709 12:26:42.201734 (+ 44312us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:42.241831 (+ 40097us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:42.241882 (+    51us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2067864,"reported_tablets":44,"spinlock_wait_cycles":1375695232,"child_traces":[["op",{"apply.queue_time_us":167206,"apply.run_cpu_time_us":57731,"apply.run_wall_time_us":1672982,"cfile_cache_hit":70,"cfile_cache_hit_bytes":73220,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6219,"cfile_init":1,"lbm_read_time_us":855,"lbm_reads_lt_1ms":8,"num_ops":18,"prepare.queue_time_us":210952,"prepare.run_cpu_time_us":20428,"prepare.run_wall_time_us":650498,"raft.queue_time_us":177446,"raft.run_cpu_time_us":7727,"raft.run_wall_time_us":41969,"replication_time_us":380130,"thread_start_us":593532,"threads_started":5,"wal-append.queue_time_us":467010,"wal-append.run_cpu_time_us":25573,"wal-append.run_wall_time_us":2185765}]]}
I20260709 12:26:45.090205 10309 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:45.046554  6021 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:45.070940 10061 raft_consensus.cc:3055] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:45.121384  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:45.039795 10426 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:26:45.169241  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:45.170526 10426 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:45.170989 10426 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:45.125805 10413 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:45.172145 10426 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:45.173058 10413 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:45.174515 10413 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:45.131162 10501 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:45.132474 10393 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:45.133095 10579 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:45.063086 10434 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 12:26:45.088162 10377 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.180071 10377 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:44.963536 10296 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:45.182680 10296 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:45.087215 10450 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:26:45.184607 10450 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:45.185173 10450 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:45.186350 10450 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:45.188077 10450 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:45.133939 10414 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:45.140602 10494 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:45.191536 10494 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:45.151005  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:45.156127 10583 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:45.194047  6611 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:45.194661 10519 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15.
I20260709 12:26:45.195364 10583 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:45.089290 10523 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:45.159430 10584 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:45.197566 10523 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:45.067657 10526 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:45.117393 10561 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:45.202919 10512 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:45.203742 10512 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.204478 10512 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:45.118705 10581 consensus_meta.cc:287] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.854s	user 0.008s	sys 0.004s
I20260709 12:26:45.206064 10581 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:45.206983  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:45.207876  5940 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:45.208571 10581 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:45.121536 10061 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:45.099061 10540 consensus_meta.cc:287] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.933s	user 0.011s	sys 0.002s
I20260709 12:26:45.214643  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:45.215598  5696 catalog_manager.cc:5697] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 4 to 7, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 7 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:26:45.169476 10451 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:45.230595  6611 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:45.238353 10061 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:45.239554 10061 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:45.236934  5696 catalog_manager.cc:5697] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:26:45.241466  5696 catalog_manager.cc:5697] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:26:45.248210  5947 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:45.259366  6018 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:45.221046 10505 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:45.289527  6611 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:45.241868 10426 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:45.317212  5949 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
W20260709 12:26:45.262192 10520 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.005s	sys 0.024s
I20260709 12:26:45.245167 10438 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:26:45.393977 10580 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.942s	user 0.020s	sys 0.053s
W20260709 12:26:45.421916 10580 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.970s	user 0.020s	sys 0.053s
I20260709 12:26:45.310205  5940 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
W20260709 12:26:45.365021  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.288089 10545 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.011s	sys 0.018s
I20260709 12:26:45.316820 10393 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:26:45.318636 10531 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.006s	sys 0.024s
I20260709 12:26:45.391590 10509 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:45.234755 10540 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:45.253961 10299 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.872s	user 0.012s	sys 0.041s
I20260709 12:26:45.368180  5945 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
W20260709 12:26:45.401546  6609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.074s	sys 0.127s
W20260709 12:26:45.461283  6609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.879s	user 0.074s	sys 0.127s
W20260709 12:26:45.461740  6793 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.011s	sys 0.040s
W20260709 12:26:45.462275  6793 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.011s	sys 0.040s
W20260709 12:26:45.462180  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 940180us
I20260709 12:26:45.429306  5696 catalog_manager.cc:5697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:26:45.367152  5838 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:45.386686 10377 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:45.453902  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.401783 10520 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.006s	sys 0.024s
W20260709 12:26:45.430830 10545 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.011s	sys 0.019s
I20260709 12:26:45.475049  5696 catalog_manager.cc:5697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 5 to 6, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 6 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:45.463567  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:45.466305  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 147209us
W20260709 12:26:45.477114  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.477942  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.478955  5838 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:45.480340  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:45.472510  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.473903  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.506019  5914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.002s	sys 0.016s
W20260709 12:26:45.512669  5987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.027s	sys 0.039s
W20260709 12:26:45.455979 10531 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.007s	sys 0.024s
I20260709 12:26:45.422043 10438 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:45.459389 10430 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.089s	sys 0.124s
W20260709 12:26:45.466142 10299 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.078s	user 0.012s	sys 0.051s
W20260709 12:26:45.561183  5987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.031s	sys 0.045s
I20260709 12:26:45.556850  5871 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:45.559561  5914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.003s	sys 0.016s
W20260709 12:26:45.561172  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 97708.7us
W20260709 12:26:45.587908  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 740195us
W20260709 12:26:45.523229  5868 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.977s	user 0.012s	sys 0.022s
I20260709 12:26:45.560848  5696 catalog_manager.cc:5697] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 7 to 8, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 8 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
W20260709 12:26:45.598577  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.597733  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.000s	sys 0.013s
I20260709 12:26:45.460695 10393 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:45.561916 10430 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.089s	sys 0.125s
W20260709 12:26:45.622150  6757 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.006s	sys 0.009s
W20260709 12:26:45.622792  6757 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.006s	sys 0.009s
I20260709 12:26:45.601483  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:45.604511  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.000s	sys 0.013s
W20260709 12:26:45.626361  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 739109us
I20260709 12:26:45.632324  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:45.605523  5913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.008s	sys 0.016s
W20260709 12:26:45.636934  5913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.008s	sys 0.017s
I20260709 12:26:45.507759 10505 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:45.609958  5696 catalog_manager.cc:5697] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 2 to 6, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 6 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
W20260709 12:26:45.521827 10439 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.018s	sys 0.016s
I20260709 12:26:45.560918 10377 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:45.600673  5868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.055s	user 0.012s	sys 0.027s
W20260709 12:26:45.638083  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 808942us
I20260709 12:26:45.554688 10309 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:45.565697 10580 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:45.644841  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:45.554781 10061 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:45.558615 10582 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:45.677979 10582 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:45.680760 10582 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:45.592327 10523 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:45.681710 10523 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:45.684429 10523 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:45.565042 10414 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:45.685519 10414 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:45.687927 10414 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:45.688463 10414 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:45.689738 10393 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:45.654239 10439 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.023s	sys 0.017s
I20260709 12:26:45.611994 10588 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:45.566689 10561 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:45.664110 10115 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:45.673358  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:45.728526  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 55307.9us
I20260709 12:26:45.730329  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:45.648473 10299 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:45.658043 10590 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:26:45.693998  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 61798.4us
I20260709 12:26:45.706578  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:45.741762  5840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.013s	sys 0.019s
W20260709 12:26:45.742381  5840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.013s	sys 0.019s
W20260709 12:26:45.667088 10158 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.845s	user 0.010s	sys 0.003s
I20260709 12:26:45.731597  6023 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:45.672868  5696 catalog_manager.cc:5697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:45.742766  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 775412us
I20260709 12:26:45.705267 10501 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:45.667423 10505 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:45.745085  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:45.746238  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:45.760623  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:45.694568 10592 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:45.698534 10580 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:45.749305  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:45.705940 10061 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:45.747762  5839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.010s	sys 0.018s
I20260709 12:26:45.670239 10377 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:45.736250 10531 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:45.784124  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:45.742514 10596 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:45.727818 10115 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:45.772298  5839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.010s	sys 0.018s
I20260709 12:26:45.562950 10438 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:45.791399 10384 consensus_meta.cc:287] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.923s	user 0.014s	sys 0.000s
I20260709 12:26:45.772199 10595 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:45.806272  5871 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:26:45.807322  5943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.887s	user 0.025s	sys 0.015s
W20260709 12:26:45.807762  5943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.025s	sys 0.015s
I20260709 12:26:45.808470  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:45.773999 10358 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
W20260709 12:26:45.736047 10577 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.034s	sys 0.048s
I20260709 12:26:45.780125  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:45.719966 10588 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:45.768280  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:45.840845  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 72680.9us
I20260709 12:26:45.841877  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:45.744973 10414 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:45.788280 10158 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:45.750517 10526 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:45.770824 10467 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.670s	user 0.012s	sys 0.000s
I20260709 12:26:45.757510 10581 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.757550 10299 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:26:45.810355  6021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.011s	sys 0.014s
W20260709 12:26:45.856076  6021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.011s	sys 0.014s
I20260709 12:26:45.751574 10590 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:45.805645 10384 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:45.809491  5871 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:45.765697  5696 catalog_manager.cc:5697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 5 to 6, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 6 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:45.813846  6773 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.030s	sys 0.056s
W20260709 12:26:45.878724  6773 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.030s	sys 0.059s
I20260709 12:26:45.797070 10596 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:45.823424  6778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.006s	sys 0.025s
I20260709 12:26:45.814617  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:45.779367 10501 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:45.815616  6015 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.936s	user 0.008s	sys 0.026s
W20260709 12:26:45.884419  6015 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.005s	user 0.008s	sys 0.027s
W20260709 12:26:45.815423 10508 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.926s	user 0.008s	sys 0.026s
I20260709 12:26:45.806926 10531 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:45.769793 10579 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.593s	user 0.009s	sys 0.003s
W20260709 12:26:45.832273 10413 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.000s	sys 0.031s
I20260709 12:26:45.847695  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:45.828851 10540 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:45.837896 10577 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.035s	sys 0.048s
W20260709 12:26:45.843506 10529 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.874s	user 0.007s	sys 0.029s
I20260709 12:26:45.796049 10520 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:45.883420  6778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.006s	sys 0.025s
I20260709 12:26:45.848909 10582 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:45.885061  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.849324 10426 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:45.885475  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1027) took 1067 ms. Trace:
I20260709 12:26:45.849958 10509 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:45.851915 10595 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:45.895859 10508 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.024s	user 0.009s	sys 0.026s
I20260709 12:26:45.899359 10590 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:45.900276 10599 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:26:45.919349  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 283452us
W20260709 12:26:45.924645  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 77053.3us
I20260709 12:26:45.930827  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:45.879595 10561 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:45.971755  6756 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:26:45.971750  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:45.832247 10115 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:45.974606 10593 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:45.990933  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1031) took 1022 ms. Trace:
I20260709 12:26:45.991695  6023 rpcz_store.cc:276] 0709 12:26:44.968583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.974064 (+  5481us) service_pool.cc:224] Handling call
0709 12:26:45.990642 (+1016578us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b54003642d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:45.990836 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1739226880}
I20260709 12:26:45.995340  6611 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:45.920964 10413 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.000s	sys 0.032s
I20260709 12:26:45.998705 10413 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:45.999254 10413 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.999697 10413 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:45.935027  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:45.937222  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:45.831933 10061 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:45.884935 10606 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:45.854702 10467 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:45.840157 10505 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:45.874815 10581 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:45.938309 10529 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.007s	sys 0.030s
I20260709 12:26:45.954200  6015 rpcz_store.cc:276] 0709 12:26:44.818062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.818214 (+   152us) service_pool.cc:224] Handling call
0709 12:26:44.852973 (+ 34759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:44.853237 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:44.879616 (+ 26379us) log.cc:844] Serialized 14 byte log entry
0709 12:26:45.885079 (+1005463us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:45.885137 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:45.885183 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.885251 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:45.885407 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":147,"prepare.queue_time_us":26716,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":2264,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":2379,"thread_start_us":1004914,"threads_started":1}
I20260709 12:26:45.836596 10580 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:46.033893 10600 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
W20260709 12:26:46.038141 10589 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:26:46.040113  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1185) took 1040 ms. Trace:
I20260709 12:26:46.040683  6773 rpcz_store.cc:276] 0709 12:26:44.999385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.999560 (+   175us) service_pool.cc:224] Handling call
0709 12:26:44.999928 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:45.000287 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:45.000319 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:45.010584 (+ 10265us) log.cc:844] Serialized 26 byte log entry
0709 12:26:45.187339 (+176755us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b18001e1420. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:26:45.879555 (+692216us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:45.879647 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.879768 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:46.039790 (+160022us) raft_consensus.cc:1685] finished
0709 12:26:46.039816 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:46.040043 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1468,"raft.queue_time_us":487,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":32771,"spinlock_wait_cycles":4840320,"thread_start_us":691933,"threads_started":1,"wal-append.queue_time_us":1001849}
I20260709 12:26:45.938077 10309 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:45.957830 10602 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.020797  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:45.951400 10520 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:46.051975  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1174) took 1321 ms. Trace:
I20260709 12:26:46.057394  6793 rpcz_store.cc:276] 0709 12:26:44.730783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.730979 (+   196us) service_pool.cc:224] Handling call
0709 12:26:44.757620 (+ 26641us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:44.758093 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:44.758126 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:44.758718 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 12:26:45.463160 (+704442us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:45.463231 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.463350 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:46.051705 (+588355us) raft_consensus.cc:1685] finished
0709 12:26:46.051729 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:46.051897 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43540,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":4879,"thread_start_us":700494,"threads_started":1,"wal-append.queue_time_us":1041045}
I20260709 12:26:46.063864 10529 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:46.065689 10377 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:46.092598  6775 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:26:46.027786 10299 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:46.077468 10512 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.868s	user 0.010s	sys 0.005s
I20260709 12:26:46.056926 10602 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:46.073134 10590 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:46.086392 10529 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:46.113058  5990 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:46.118203 10529 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:46.116204 10384 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:46.123102 10588 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:46.108258 10377 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:46.106804 10512 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:46.175251  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:46.176411  6778 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:46.116986 10590 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:46.137253 10501 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:46.184881  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 210803us
I20260709 12:26:46.166061 10509 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:46.187286  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:46.197817 10520 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:46.198833 10596 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:46.176262 10377 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:46.202371 10608 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:26:46.203802 10602 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:46.204376 10590 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:46.207155 10512 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:46.196796 10509 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:46.210572  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1175) took 1479 ms. Trace:
I20260709 12:26:46.211282 10509 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:46.210991  6757 rpcz_store.cc:276] 0709 12:26:44.731149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.733705 (+  2556us) service_pool.cc:224] Handling call
0709 12:26:44.757809 (+ 24104us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:44.849108 (+ 91299us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:45.623415 (+774307us) log.cc:844] Serialized 14 byte log entry
0709 12:26:45.920983 (+297568us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:45.933126 (+ 12143us) log.cc:844] Serialized 26 byte log entry
0709 12:26:45.938376 (+  5250us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:45.938755 (+   379us) log.cc:844] Serialized 14 byte log entry
0709 12:26:45.939034 (+   279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.994700 (+ 55666us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b5400008054. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:45.994778 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:46.210301 (+215523us) raft_consensus.cc:1685] finished
0709 12:26:46.210329 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:46.210493 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":163718,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":2854,"raft.queue_time_us":12809,"raft.run_cpu_time_us":547,"raft.run_wall_time_us":39299,"spinlock_wait_cycles":208696576,"thread_start_us":1048124,"threads_started":2,"wal-append.queue_time_us":515554}
W20260709 12:26:46.224704 10450 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.036s	user 0.006s	sys 0.011s
I20260709 12:26:46.225184 10581 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:46.195782 10501 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:46.201499 10377 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:46.241737 10413 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:46.244881 10413 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:46.251940 10467 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:46.256071 10585 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.817s	user 0.006s	sys 0.000s
I20260709 12:26:46.257422 10529 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:46.260255 10598 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.366s	user 0.001s	sys 0.001s
W20260709 12:26:45.840996  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 61015us
I20260709 12:26:46.309183  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 880) took 1791 ms. Trace:
I20260709 12:26:46.310669  5868 rpcz_store.cc:276] 0709 12:26:44.517255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.517371 (+   116us) service_pool.cc:224] Handling call
0709 12:26:44.517708 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:44.519658 (+  1950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:44.519692 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:44.527603 (+  7911us) log.cc:844] Serialized 26 byte log entry
0709 12:26:45.653363 (+1125760us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:45.653430 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.653541 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:46.307571 (+654030us) raft_consensus.cc:1685] finished
0709 12:26:46.307711 (+   140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:46.308625 (+   914us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1497,"raft.queue_time_us":5146,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":18507,"thread_start_us":1107145,"threads_started":1,"wal-append.queue_time_us":1239146}
I20260709 12:26:46.304136  5696 catalog_manager.cc:5697] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 4 to 7, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 7 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:46.317871  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:46.323590  5696 catalog_manager.cc:5697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 4 to 7, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 7 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:46.324663  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.360898  5866 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:46.324745  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:46.362763  5871 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:46.363040  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:46.364157  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:46.324756  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 536849us
I20260709 12:26:46.362864 10467 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:46.366235 10467 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:46.332945  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:46.368463  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.368646  5865 raft_consensus.cc:2410] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 12:26:46.369244  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.335115  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.330757  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.370157  5872 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:46.370640  5873 raft_consensus.cc:2410] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:26:46.370484  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.371399  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.336043  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:46.336522  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:46.339514  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.007s	sys 0.022s
I20260709 12:26:46.371946  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:46.372507  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.008s	sys 0.023s
I20260709 12:26:46.373127  5873 raft_consensus.cc:2393] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:46.372911  5870 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:46.217584  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:46.374240 10434 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:46.336706  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.230504  6778 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:26:46.353931 10061 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:46.377058  5947 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:46.377118  5869 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 13: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:46.378116 10061 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:46.376499  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 159207us
I20260709 12:26:46.352514  6766 raft_consensus.cc:2368] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16.
I20260709 12:26:46.381068 10158 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:46.381809 10158 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:46.382681  5872 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:46.383903  5874 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:46.384032 10158 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:46.386619 10430 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:46.386907  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:46.388664  5944 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:46.328491  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:46.327047  5868 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:46.371721  5871 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
W20260709 12:26:46.373049  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 609758us
I20260709 12:26:46.364552  5866 raft_consensus.cc:2393] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:46.393618 10377 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:46.394315 10061 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:46.395361 10061 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:46.396291 10377 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:46.360677 10505 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:46.398470 10377 raft_consensus.cc:3055] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:46.399066 10377 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:46.399304 10505 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:46.400704 10377 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:46.344462  5942 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:26:46.340073  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.384107  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:46.403321 10358 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:46.403680  5940 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:46.404388 10358 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:46.405309 10505 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:46.406250 10505 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:46.406718 10358 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:46.408807  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:46.409611 10296 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.226s	user 0.013s	sys 0.008s
I20260709 12:26:46.410018 10434 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:46.410290  6778 raft_consensus.cc:2410] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:26:46.411041 10434 raft_consensus.cc:697] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 12 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:46.413336  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:46.409379 10358 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:46.414780  5838 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:46.415043  6757 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
W20260709 12:26:46.416424  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.418727  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:46.396324 10061 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:46.422171 10451 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:46.423731 10451 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:46.426935 10451 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:46.407194 10505 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:46.430917  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1527) took 1633 ms. Trace:
I20260709 12:26:46.424057  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:46.436882  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:46.438619  6757 raft_consensus.cc:2368] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
I20260709 12:26:46.453117 10061 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:46.453828  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:46.454192 10467 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:46.456496  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:46.457224  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:46.457965  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:46.459789  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:46.460574  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:46.462095  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:46.481873  5940 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:46.431615  5943 rpcz_store.cc:276] 0709 12:26:44.797073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.803268 (+  6195us) service_pool.cc:224] Handling call
0709 12:26:44.918914 (+115646us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:44.919462 (+   548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:44.919502 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:44.920150 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 12:26:45.808265 (+888115us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:45.808332 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.808440 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:46.340863 (+532423us) raft_consensus.cc:1685] finished
0709 12:26:46.340920 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:46.430184 (+ 89264us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b540043f8d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:46.430620 (+   436us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1292,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":21307,"spinlock_wait_cycles":254792064,"thread_start_us":887643,"threads_started":1,"wal-append.queue_time_us":1146066}
I20260709 12:26:46.483722  5947 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:46.481564 10158 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:46.487391  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:46.488520  6757 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:46.490453  5944 raft_consensus.cc:2393] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:26:46.491223  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.494717  5945 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:46.496416 10519 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:46.496790  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:46.497938  6015 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:46.499413 10529 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:46.500109 10523 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:46.501698  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:46.504128  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 82783.4us
I20260709 12:26:46.504359  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:46.505054 10426 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.505497  5873 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:46.506098 10426 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:46.507853  6023 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:46.510300  6018 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:26:46.512418  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:46.513159  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 66236us
I20260709 12:26:46.513617  5870 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:46.413232 10434 consensus_queue.cc:237] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:46.514048 10523 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:46.514835  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.515334  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.516376  5872 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:46.516723 10523 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:46.517944  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 103677us
I20260709 12:26:46.504410  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.497503 10519 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:46.523111  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 133498us
I20260709 12:26:46.524396  6609 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:46.525761 10592 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:46.526085 10593 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:46.513411 10595 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.513384  6022 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:46.527695 10595 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:46.528261 10426 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:46.539153 10505 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:46.531726  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:46.535220  5990 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:46.534195 10377 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:46.543619 10586 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:46.544486 10586 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:46.534756  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:46.533823 10621 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:26:46.549146 10621 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:46.531823  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:46.534415  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:46.554675  5949 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:46.535233  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:46.568711  5913 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:46.556995 10531 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:26:46.555277 10624 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.571444  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.571084 10624 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:46.572017 10531 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.574092 10624 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:46.577688 10611 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.572314  5865 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 13: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:46.578704 10611 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:46.579350  6018 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:46.579680  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:46.550516 10586 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:46.580984  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:46.581612  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:46.582681  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:46.583658  5942 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:26:46.583765  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:46.557201  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:46.559201  5874 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:26:46.529711  5867 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:46.561450  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:46.592680  6015 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:46.601250  6022 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:26:46.572949 10531 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:46.603559  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:46.607628  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:46.610772  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:46.533898 10519 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:46.571044  5915 connection.cc:256] RPC call timeout handler was delayed by -1.0065909210002246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17390000000000003s for a response
I20260709 12:26:46.600637 10061 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:46.554575 10595 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:46.549605 10621 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:46.521347 10115 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:46.620656  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:46.622318  5867 raft_consensus.cc:2376] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12. Re-sending same reply.
I20260709 12:26:46.611622  6018 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:46.604161 10611 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:46.628122  6023 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:46.633734 10624 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:46.634409 10624 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.634807 10624 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:46.637760  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:46.643850  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:46.646010  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 121877us
I20260709 12:26:46.647359  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:46.647974 10611 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:46.648589 10611 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:46.649096 10611 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:46.654793 10061 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.655530 10061 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:46.645174  5941 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:46.649128  5915 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:46.663125  6778 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:46.668491  5838 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:46.618433 10592 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:26:46.674454 10356 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:46.675192 10356 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:46.675637 10356 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:46.676259  5871 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:46.676764  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:46.677184  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.677654  5866 raft_consensus.cc:2376] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15. Re-sending same reply.
I20260709 12:26:46.678174  5874 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:46.679486  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.680482  5865 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:46.681582  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.689004  5868 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:46.690320  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.691068  5868 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:46.692090  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:46.692919  5868 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:46.682142  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.694515  5872 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:46.682796  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.696128  5870 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:46.698011  6611 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:46.699162  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 69271.4us
I20260709 12:26:46.700121  6611 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:46.704551 10426 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:46.705286 10426 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:46.706498 10592 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:46.685863  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:46.686918  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.707945 10592 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.707979  5869 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:46.708459 10592 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:46.706558 10426 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:46.686400  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.714166  5873 raft_consensus.cc:2393] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:46.714192  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:46.715870  6766 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 17: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:46.719343  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:46.683491  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:46.721470  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 53757.3us
I20260709 12:26:46.723134 10434 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:46.723925 10434 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.724596 10434 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:46.734211  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:46.734388 10032 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:46.724305  5940 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:26:46.736292  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:46.736415  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 87485.8us
I20260709 12:26:46.724242  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:46.743394  5865 raft_consensus.cc:2410] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 12:26:46.745898  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 62601.9us
I20260709 12:26:46.747494  6609 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:46.749017 10438 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:46.749480 10438 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.749897 10438 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:46.752079 10450 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:46.752627 10450 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:46.753047 10450 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:46.757279 10306 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.758062  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:46.758298 10306 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:46.759351 10032 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:46.760155 10032 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:46.743445  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.760366  5915 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:46.761736  5874 raft_consensus.cc:2410] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:26:46.775956  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:46.781365  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:46.795516  5940 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:46.798913  5944 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:26:46.781383 10595 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:46.802791  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.725116 10621 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:46.819396  5990 leader_election.cc:400] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
I20260709 12:26:46.819989  5990 leader_election.cc:403] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:26:46.802781 10377 raft_consensus.cc:2764] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 12:26:46.835764  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 88640.7us
I20260709 12:26:46.814014 10158 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:26:46.837862 10531 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:46.837992 10032 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:46.842676 10356 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:46.826385 10595 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:46.845877 10356 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:46.847200 10519 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
W20260709 12:26:46.848019  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 77109.9us
I20260709 12:26:46.850572  5990 leader_election.cc:400] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16.
I20260709 12:26:46.851837  5990 leader_election.cc:403] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:26:46.853029 10519 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16.
I20260709 12:26:46.850680 10595 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:46.856084  6022 raft_consensus.cc:1217] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 12.7->[13.8-13.8]   Dedup: 13.8->[]
I20260709 12:26:46.858645  6022 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:26:46.842512 10621 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:46.862133  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 101977us
I20260709 12:26:46.884232  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:46.891048 10611 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:46.892045  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:46.902064  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.908842  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:46.887023 10306 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:46.918637  5915 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:46.922442  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:46.924676  6611 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:46.925657 10426 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:46.925897 10611 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:46.926139 10426 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.926649 10426 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:46.928695  6611 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:46.929808 10115 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:46.930477 10115 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:46.931175  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:46.934306  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.934466  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:46.941294  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:46.941172 10377 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:46.948586 10377 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:46.949890 10377 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:46.955363  6766 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:26:46.955561 10505 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:46.960304  6015 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:46.962478  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:46.964717  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:46.968470 10592 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:46.970568 10592 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:46.960424 10115 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:46.974586 10610 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.975214 10610 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:46.978036  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:46.982295  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 79237.2us
I20260709 12:26:46.983913 10606 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:46.985796 10606 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:46.988189 10606 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:46.988698 10606 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:46.990329 10606 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:46.992321  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:46.992398 10606 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:46.994203  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:46.995102  6017 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:46.995718  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:46.996794  6019 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 11: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:46.998298  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:47.004036 10450 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:47.006016  6756 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:47.010506  6022 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:47.018693  6020 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:47.023042 10508 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:47.024969 10508 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:47.028090  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:47.028734  5867 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:47.026839 10450 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:47.029903  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:47.030128  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:47.030711  5865 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:26:47.030910  5874 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:47.032379  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:47.034626  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:47.035352  6020 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:47.042829  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:47.047200  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:47.047577  5870 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:47.048895  5869 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:47.048730  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:47.049697  5871 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:47.046375  5865 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:26:47.050006  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:47.050530  5865 pending_rounds.cc:85] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 7
I20260709 12:26:47.050891  5865 pending_rounds.cc:107] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:26:47.051039  5868 raft_consensus.cc:2393] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 12: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:47.051192  5865 raft_consensus.cc:2889] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:47.052217  6021 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 11: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:47.053614  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:47.054438  6016 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:47.055836  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:47.056566  5947 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:47.057021  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:47.057618  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:47.058291  5942 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:47.059252  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:47.060096  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 74484.8us
W20260709 12:26:47.080775  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 47866.4us
I20260709 12:26:47.088392  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.017030 10624 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:47.093807  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:47.090600  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:46.977502 10610 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:47.089505  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:47.099777  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:47.102653  6021 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 16: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:47.103237  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:47.107951  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:47.110594  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 45933.8us
I20260709 12:26:47.104743  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:47.096128  6022 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:47.042680 10061 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:47.119622  5948 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:47.121454  5949 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:47.115940  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.129767  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:47.139815  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.148924  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:47.137791 10505 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:47.155004  6756 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:26:47.157892  5948 raft_consensus.cc:2368] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13.
W20260709 12:26:47.160617  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.175273  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.175125  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.179709  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.180514  5941 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:26:47.180668  6757 raft_consensus.cc:2410] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 12:26:47.178236  6762 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:47.181241  5915 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:47.196885  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.204564  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:47.209533 10658 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:26:47.211984  5943 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:47.214836 10061 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:47.221997  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 88592.1us
I20260709 12:26:47.223054  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:47.225854 10508 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:47.226449 10508 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.226861 10508 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:47.249596  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:47.245575 10434 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.006s
W20260709 12:26:47.268970 10660 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.196s	user 0.002s	sys 0.000s
I20260709 12:26:47.176337 10505 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:47.251895 10624 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:47.277448  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:47.288960  6016 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:47.297070  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:47.305186  6756 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:26:47.308429  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.191039 10529 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:26:47.318715  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:47.273520 10654 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.201s	user 0.002s	sys 0.000s
I20260709 12:26:47.278684  5840 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:47.323799  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:47.329072  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:47.332814  5948 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:47.314287 10438 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.564s	user 0.012s	sys 0.003s
I20260709 12:26:47.338383  6762 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:47.340580 10438 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:47.349426  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:47.350481  5946 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:47.350329  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:47.352375  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.350796 10611 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:47.360343  6778 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
W20260709 12:26:47.361820  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.360378  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.366384  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:47.367740 10657 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:26:47.366593  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:47.367614  6773 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
W20260709 12:26:47.371446  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.391417 10438 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:47.400648  5949 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:47.404718  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:47.420149  5947 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:26:47.421496  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:47.422552  5941 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:26:47.427881  6022 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
W20260709 12:26:47.438997  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 295249us
I20260709 12:26:47.307312 10434 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:47.447603  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:47.449796  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 172882us
I20260709 12:26:47.450532 10545 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:26:47.436479 10592 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:26:47.470649  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.461705 10430 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:47.474566 10430 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:47.475020 10430 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:47.477794 10413 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:47.478466 10413 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:47.480692 10413 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:47.482211  5946 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:47.486632  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.487897  6757 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:47.489849  6756 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:26:47.490147  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:47.491987  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.461980 10438 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.508466 10438 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:47.512229 10438 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:47.521754 10434 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:47.528270 10426 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.601s	user 0.014s	sys 0.000s
W20260709 12:26:47.532209  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 83016.9us
I20260709 12:26:47.537721  6773 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:26:47.564205  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 92177.2us
I20260709 12:26:47.529050 10426 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:47.588147  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 146026us
I20260709 12:26:47.558844  5941 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:47.596796  5948 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:26:47.599067 10665 consensus_queue.cc:1059] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:26:47.599767 10450 raft_consensus.cc:3055] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:47.665997 10450 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:47.667310 10450 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:47.626289  6773 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:47.628129  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 62769.5us
I20260709 12:26:47.634960 10426 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:47.586202 10438 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
W20260709 12:26:47.674031  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 140646us
I20260709 12:26:47.674261  6023 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:47.669394 10450 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:47.690851  6023 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:26:47.704985  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 511513us
W20260709 12:26:47.717727  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 536594us
I20260709 12:26:47.718760  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:47.719655  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 47323.5us
I20260709 12:26:47.707401  5913 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:47.729161 10032 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.729878 10032 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:47.732419 10032 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:47.733193 10032 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.742549  6611 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:47.726176 10624 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:47.747463  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:47.747875  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:47.748253  6022 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:26:47.748580  6016 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 9.
I20260709 12:26:47.749075  6023 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:47.744596 10669 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:26:47.755275  6023 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:47.760392  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:47.764421 10621 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:47.752388 10426 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:47.750465 10606 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:47.741323 10032 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:47.768786  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 50132.3us
I20260709 12:26:47.761158  6021 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:47.758615 10545 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:26:47.785073  5946 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:47.786685  6016 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:47.787294  6016 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:47.775946  6785 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:26:47.778316 10508 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.551s	user 0.014s	sys 0.001s
I20260709 12:26:47.790467 10508 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:47.779124  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:47.791664  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:47.792557  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:47.793409  5873 raft_consensus.cc:2410] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:26:47.779286  5948 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:26:47.754331  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:47.782195  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:47.783283  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:47.767632 10426 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.802978  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:47.803187  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:47.793984  5948 pending_rounds.cc:85] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 7
I20260709 12:26:47.804018  5948 pending_rounds.cc:107] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:26:47.804287  5948 raft_consensus.cc:2889] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:47.804450  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:47.804495  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:47.794447  6773 raft_consensus.cc:606] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:26:47.800652 10606 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:47.803479  5865 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:26:47.806974  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:47.808248 10606 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:47.813874 10621 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:47.814172  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:47.816720 10674 raft_consensus.cc:3055] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:47.817274  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:47.818753 10624 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:47.819212 10624 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:47.822592  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:47.804039  5872 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:47.825888  5867 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:47.820080 10426 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:47.820144 10434 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:47.830631  6022 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:47.779187 10664 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:26:47.834559  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:47.819629 10624 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:47.883724  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 134903us
I20260709 12:26:47.830967  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:47.833534 10434 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:47.820586 10602 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:47.817276 10674 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:47.881762 10606 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:47.882586 10673 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:47.826195 10377 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:47.903540  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 84089.8us
I20260709 12:26:47.906260  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:47.828565  6016 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:47.903239  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:47.908134  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:47.910945  6016 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:47.881417 10671 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:47.913306  5872 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:47.881593 10032 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:47.894904 10602 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:26:47.921097  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 162878us
W20260709 12:26:47.919335  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 156788us
I20260709 12:26:47.922914  6017 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:47.924742  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:47.925863  6018 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:47.926461  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:47.910626 10590 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:47.914039 10671 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:26:47.937471  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:47.938671  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 231389us
I20260709 12:26:47.924440 10602 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:47.922549 10061 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:47.943171  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56681.9us
I20260709 12:26:47.944247  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:47.940435  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:47.950686 10434 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:47.952003  6024 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:47.954408 10508 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:47.943230 10586 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:26:47.947113  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:47.962862  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:47.967983 10505 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:47.968997 10505 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:47.938449 10430 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:47.930876 10523 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:26:47.973673  6016 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:26:47.976325  6016 pending_rounds.cc:85] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 4
I20260709 12:26:47.976641  6016 pending_rounds.cc:107] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:26:47.976928  6016 raft_consensus.cc:2889] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:47.971939 10430 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:47.982677  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:47.983481 10032 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:47.984077 10032 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.984413 10032 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:47.993923  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.994282  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.994851  6775 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:47.995535  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:47.996156  6785 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:26:47.994045  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:47.998672  6762 raft_consensus.cc:2410] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:26:47.970095  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:47.969609 10505 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 14 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:48.010305 10590 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:48.012487 10590 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:48.012962 10590 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:48.019779  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.023392  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:48.024119 10586 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:48.024657 10586 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:48.024835  6020 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:48.025043 10586 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:48.026167  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 56212.7us
I20260709 12:26:48.027915  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.029955  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:48.026196 10685 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:48.033980 10685 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:48.032577  6785 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
W20260709 12:26:48.037747  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 93571.5us
W20260709 12:26:48.038082  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 55492.6us
I20260709 12:26:48.042591  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:48.044018  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:48.046747 10681 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:48.044660  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:48.047346 10681 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:48.047700  6017 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:48.047762 10681 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:48.047353  6019 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:26:48.050789 10652 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:48.051342 10652 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:48.052609  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.052412 10652 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:48.054556  6020 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:48.056301 10450 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:48.056684 10624 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:48.058324  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.059381  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.041708 10606 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:48.070577 10652 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.070338  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:48.071300  5868 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:48.072789 10606 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:48.074157 10665 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:48.075225 10671 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:48.075968 10606 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:48.078023  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 54713.3us
I20260709 12:26:48.078601  6018 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:48.078653  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:48.084591 10545 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:48.086091  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:48.087039  5872 raft_consensus.cc:2410] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:26:48.082571  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.105046  6017 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:26:48.105360  5873 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:48.112918  5871 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:48.081817 10606 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:48.065590 10611 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:26:48.133239  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 73957.6us
I20260709 12:26:48.139073  6609 leader_election.cc:400] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13.
I20260709 12:26:48.139439  6609 leader_election.cc:403] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:48.139706  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 97202.1us
I20260709 12:26:48.145704  5944 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:48.131639 10061 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:48.149771  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:48.117621 10663 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:48.161283  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 83971.6us
I20260709 12:26:48.132480 10606 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:26:48.165853  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 51274.6us
W20260709 12:26:48.169376  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 86832.6us
I20260709 12:26:48.168622  6762 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:48.170989  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.173458  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.147230 10688 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:48.168557  5838 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:48.138778 10611 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:48.166185 10545 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13.
I20260709 12:26:48.190635 10689 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:26:48.176769 10688 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:48.197600 10688 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:48.198279 10624 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:48.199030 10676 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:48.202672  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.177024 10606 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:48.152859 10505 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:48.215200  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 1080) took 1131 ms. Trace:
W20260709 12:26:48.223397  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.177400 10426 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:48.192638 10679 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:48.205557 10676 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:48.273375 10697 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:48.278227 10697 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.278594 10697 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:48.282126  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 115981us
I20260709 12:26:48.283089  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:48.190379 10681 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:48.293612  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 63239.4us
I20260709 12:26:48.292155  5868 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:48.297288 10692 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:48.298026 10692 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:48.299060 10692 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:48.301895 10438 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:48.302753 10690 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:26:48.302879 10438 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:48.268235  6766 rpcz_store.cc:276] 0709 12:26:47.077997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.078152 (+   155us) service_pool.cc:224] Handling call
0709 12:26:47.118705 (+ 40553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:47.140851 (+ 22146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:47.140882 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:47.163632 (+ 22750us) log.cc:844] Serialized 26 byte log entry
0709 12:26:47.422104 (+258472us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:47.445441 (+ 23337us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.467880 (+ 22439us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.467966 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.014371 (+546405us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b1800345820. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb642a6 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1
0709 12:26:48.015301 (+   930us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b1800345820. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:26:48.209485 (+194184us) raft_consensus.cc:1685] finished
0709 12:26:48.209511 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.209745 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41667,"prepare.queue_time_us":26206,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":26494,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":25027,"spinlock_wait_cycles":81875584,"thread_start_us":200847,"threads_started":1,"wal-append.queue_time_us":309329}
I20260709 12:26:48.310858 10592 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:48.311261 10685 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:48.312505 10676 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:48.215608 10061 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:48.314108 10606 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:48.319767 10032 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:48.268963 10624 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
W20260709 12:26:48.332677  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:48.341784  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.352931  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.328298 10679 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:26:48.375305  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 240572us
I20260709 12:26:48.350287 10664 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:26:48.378316  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 95343.8us
I20260709 12:26:48.354923 10438 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:48.385891  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:48.391037  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.354578 10652 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:26:48.364782 10592 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:48.399581  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.401250  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.400712  5913 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:48.404387  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:48.390378 10664 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:26:48.411883  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 237902us
I20260709 12:26:48.405541 10450 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:48.375771 10602 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:48.414449 10611 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:48.402794 10032 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:48.415170 10679 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:48.418417  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.399303 10685 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:48.427336  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:48.407579 10438 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:48.417400 10624 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:48.430922  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:48.431645  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:48.432976  6021 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:48.433506  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:48.433851 10602 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:48.439456  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 55461.3us
I20260709 12:26:48.434055  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:48.434383 10703 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:26:48.434706 10681 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:48.436610 10678 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:26:48.434582 10698 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:26:48.444823 10682 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:48.456867 10682 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:48.429193 10685 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.457353 10682 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:48.457489 10685 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.448464 10695 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:48.460633 10695 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:48.463131 10695 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:48.451648 10697 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:48.469655 10682 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:48.472173 10682 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:48.447310 10590 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:48.476197 10698 consensus_queue.cc:1243] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:48.477162 10602 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:48.478060 10602 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:48.479133 10602 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:48.480386 10602 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:48.480504  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:48.480887  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:48.481398  6785 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 14: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:48.481743  6775 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:48.484762  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.485113 10698 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.485718 10698 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:48.450875 10671 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.447067 10519 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:48.430342 10592 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:48.453929 10706 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.454387 10624 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:48.454505  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:48.455047 10621 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.447610 10586 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:48.445546 10664 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:48.500125  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:48.511454 10676 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:48.517095 10592 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.513104  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:48.475137 10697 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:48.519759  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 115474us
W20260709 12:26:48.522687  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68264.6us
I20260709 12:26:48.505400  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.525425  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:48.511781 10426 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:48.485376 10681 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:48.538415  6775 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:48.541800  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:48.526861 10621 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:48.546334 10606 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:48.550856  5944 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:26:48.533223  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 364793us
I20260709 12:26:48.555569 10426 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:48.556684  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 116336us
I20260709 12:26:48.556703  5949 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:48.516531 10519 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:48.533955 10712 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:48.532836 10678 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:48.538766 10592 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:48.558993 10426 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:48.560242  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:48.570043  5945 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:26:48.572202  5941 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:48.538053 10590 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:48.568570  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:48.578992  5873 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:48.523705 10706 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:48.572242  5871 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260709 12:26:48.554991  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 49704.6us
I20260709 12:26:48.546339 10621 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:48.585124  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 435456us
I20260709 12:26:48.585947  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.555514 10713 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:48.562738 10698 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:48.594873 10688 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:48.595549 10519 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:48.593978 10061 consensus_queue.cc:1059] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:48.533500 10676 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:48.597043  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:48.547149 10606 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:48.549866 10681 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:48.602185  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 54342.5us
I20260709 12:26:48.603246 10710 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:48.610628  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:48.612557  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:48.578490 10652 consensus_queue.cc:1059] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:26:48.613941  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:48.606951  5871 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:26:48.614828  5874 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 13: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:48.615069  5872 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:48.574755 10678 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:48.617693  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:48.588227 10545 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:26:48.620581 10426 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:48.605103 10682 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:48.621184 10700 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.228s	user 0.000s	sys 0.002s
I20260709 12:26:48.626914  6762 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:48.634258  6024 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:48.635952 10715 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:26:48.635696 10061 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:48.636940 10717 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:26:48.637593 10714 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:26:48.637502 10717 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:48.638845 10677 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:26:48.639498 10717 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:48.641374 10717 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:48.613327 10438 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:26:48.634059  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:48.644894  5874 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:48.651559 10694 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.465s	user 0.004s	sys 0.000s
I20260709 12:26:48.652204 10032 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:48.653196 10711 raft_consensus.cc:3055] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:48.653409 10032 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:48.653672 10711 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:48.653865 10032 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:48.654721 10711 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:48.655989 10711 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:48.655882 10685 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:48.654721 10032 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:48.659603 10032 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:48.661926  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.616535 10695 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:48.671960  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:48.672901  6766 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:26:48.674834 10703 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:48.627063 10682 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:48.600903 10624 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:48.686034  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:48.685312  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:48.685014  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:48.692878  5874 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:48.691566 10592 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:48.706442  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 139157us
I20260709 12:26:48.707701  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:48.665467 10710 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:48.717552  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.725339  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:48.726173  6768 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.730719 10676 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:48.730909 10674 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:48.731604 10586 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:48.731998  5873 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:48.732347 10674 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.733098 10681 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:48.712471 10505 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:48.735119 10685 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:48.716385 10679 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:48.736341 10679 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:48.736723 10679 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:48.738494 10606 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:48.705904 10682 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:48.685235 10611 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:48.684296 10061 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:48.768522  6775 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:26:48.692781 10688 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:48.759950  5867 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:48.691313 10706 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:48.705648 10592 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:48.723708 10692 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:48.793797  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.748862 10681 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:26:48.801960  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 92638.7us
I20260709 12:26:48.782358 10611 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:48.802986  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.812034  6768 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
W20260709 12:26:48.825381  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 239967us
W20260709 12:26:48.824924  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 228169us
W20260709 12:26:48.829511  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.831045  6611 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:48.833463  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:48.834146  5873 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:48.834025  6611 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:48.839867  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 126928us
I20260709 12:26:48.832324  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:48.840822  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:48.840544  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.843900  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:48.844841  6015 raft_consensus.cc:2410] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:26:48.796954 10678 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:48.741981 10710 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:48.782773 10061 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:48.840137 10430 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:48.846127  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:48.848693  6023 pending_rounds.cc:85] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 4
I20260709 12:26:48.865962  6023 pending_rounds.cc:107] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 12:26:48.866340  6023 raft_consensus.cc:2889] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:48.849197  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:48.868466  6019 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 12: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:48.849597  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:48.869694  5867 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:48.826390 10665 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:48.870394 10665 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:48.870826 10665 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:48.874666  6017 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:48.790280 10692 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:48.875382  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:48.876109  6015 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:48.847469  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:48.877492 10692 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:48.847355  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:48.852067 10717 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:48.852494  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:48.878628  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:48.857848  6018 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:48.858059  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:48.879900  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 77022.8us
I20260709 12:26:48.880167  5874 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:48.882052  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:48.878650  6016 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:48.852655  5840 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.885212  5873 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:48.891222  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:48.892831  5868 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:26:48.894567  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:48.888491  5915 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:48.895797  5873 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:48.896554  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:48.897842  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 57448.9us
I20260709 12:26:48.894702  6015 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:48.899789 10721 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:48.900435 10721 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:48.900554  6016 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:26:48.900969 10706 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:48.901594 10706 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:48.902127  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:48.902568 10706 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:48.903591  6018 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:48.904479 10682 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:48.905447 10678 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:48.906301 10678 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:48.906865 10678 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:48.907655 10682 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:48.908263 10721 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:48.908615  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 74665.6us
I20260709 12:26:48.909341 10592 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:48.910004  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:48.912211 10676 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:48.912204 10730 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:48.913012 10730 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:48.910050 10592 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:48.907097 10430 raft_consensus.cc:697] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:48.915256 10430 consensus_queue.cc:237] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:48.918414 10676 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:48.920770 10676 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:48.921705 10717 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:48.916229 10438 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:26:48.922264  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:48.921877  6775 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:48.923573 10438 raft_consensus.cc:3055] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:48.923969 10438 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:48.922899  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:48.924921 10438 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:48.926020  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:48.927286 10438 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:48.927572  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:48.929540  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 76958.3us
I20260709 12:26:48.922130  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:48.919144  6609 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:48.930856  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:48.913987  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:48.932768  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:48.934176  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:48.898108  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:48.934877  6773 raft_consensus.cc:606] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:48.935185  6773 raft_consensus.cc:614] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:48.846175 10624 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:48.942580 10698 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:48.943125 10698 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:48.943544 10450 raft_consensus.cc:3055] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:48.943688 10698 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:48.943997 10450 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:48.945070  6018 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:48.950702 10611 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:48.951428 10730 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:48.951560  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 68283.2us
I20260709 12:26:48.958709  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:48.960511 10061 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:48.962597  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:48.961721  5949 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:26:48.936794 10717 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:48.967154 10665 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:48.887763 10032 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:48.978999  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:48.985054  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:48.985708  5868 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.985972  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:48.986745  6017 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 17: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:48.979260  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:48.988013  5873 raft_consensus.cc:2410] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:26:48.973501  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:48.975754  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:48.978029  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:48.984340  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:48.915947 10592 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:48.990780  5945 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:48.982105  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:26:48.983270  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 50668us
I20260709 12:26:48.996315  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:48.974198  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:48.997244  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:48.995342  5913 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:48.980916  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 48239us
I20260709 12:26:49.000828  5838 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:49.005645  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:49.005813  5940 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:48.941789 10664 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:48.972633  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:49.011233  6024 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:48.945014 10450 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:48.940955 10652 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:48.974045  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:48.991328  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:48.918577 10679 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:49.006398 10695 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:48.969534 10665 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:49.016032  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:48.990543 10730 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:49.012549  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:49.008973 10685 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:49.012998 10681 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:49.029315  5868 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:49.042128  6015 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:49.031131 10624 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:49.008473 10061 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:49.031594  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.975162 10688 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:26:49.031785  6021 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:49.045869  5840 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:49.020488 10695 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:26:49.022625  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.023965  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:49.057556  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:49.020718 10711 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:49.048871  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:49.031455  6019 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:49.041603 10683 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:26:49.066447  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.017999 10592 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:26:49.064746  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:49.043912 10061 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.079428  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:49.043907 10681 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:49.029759 10611 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:26:49.063485 10695 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:49.068819 10450 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:49.065872 10697 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:49.070837 10712 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:49.054925 10685 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:49.064209 10710 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:49.098521  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.085733 10545 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:49.105727 10545 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:49.106169 10545 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:49.106897  5940 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126"
dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:48330
I20260709 12:26:49.107632  5940 raft_consensus.cc:606] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:49.107998  5940 raft_consensus.cc:614] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:49.087136 10061 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:49.078316 10675 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:49.099685  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 65104.4us
I20260709 12:26:49.130132  5840 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:49.064847 10652 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:49.109654 10746 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:49.090984 10681 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:49.120771  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 60838.6us
I20260709 12:26:49.134956  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:49.136548  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:49.145690  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 148525us
I20260709 12:26:49.141425  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:49.148849  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:49.151903  5943 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 17: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:49.155169  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:49.103852 10730 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:49.157333  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:49.098982 10621 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:26:49.167330 10747 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:49.167930 10747 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:49.168931 10724 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:49.169454 10724 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.169869 10724 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:49.171277 10739 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:49.171764 10739 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:49.172819 10739 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:49.176143 10679 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:49.158424 10697 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:49.178961 10741 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:49.168006 10685 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:49.180375  5840 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:49.138484 10426 consensus_queue.cc:1059] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:26:49.191170  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:49.163096 10523 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:49.162410 10746 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:49.205860 10738 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:49.182680 10710 raft_consensus.cc:2764] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:26:49.207753 10739 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:49.207867  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 89105.8us
I20260709 12:26:49.209228 10674 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:26:49.214207  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 67661.2us
I20260709 12:26:49.227654  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:49.180603 10747 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:49.229590  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:49.219378 10717 consensus_queue.cc:1059] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:26:49.234202 10717 raft_consensus.cc:3055] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:49.234614 10717 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:49.235627 10717 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:49.216472  5696 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40812 (request call id 142) took 42757 ms (client timeout 15000 ms). Trace:
W20260709 12:26:49.221552  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56036us
I20260709 12:26:49.236827 10717 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:49.238125 10746 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:49.193557 10689 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:49.242692  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:49.216482 10652 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:49.246045 10674 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:49.246553 10590 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:49.246908  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:49.254674  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 175377us
I20260709 12:26:49.256069  5915 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:49.258744  6024 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:49.260412  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:26:49.267537  5987 connection.cc:256] RPC call timeout handler was delayed by -1.4467307410000103s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
I20260709 12:26:49.276355  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:49.277077  6773 raft_consensus.cc:606] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:49.217057 10611 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:49.281687  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:49.285261  5940 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:49.221319 10523 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:49.290194  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:49.291463 10529 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:49.291790  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:49.274672 10679 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:49.277758 10683 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:49.301949  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:26:49.307341  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 91182.8us
I20260709 12:26:49.315744  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:49.316769  5948 raft_consensus.cc:2393] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:49.320011  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:49.254036 10590 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:49.330369  5840 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:49.333007 10697 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:49.374141  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.374523  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:49.375765  6022 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 11: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:49.381430  5987 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:49.383667  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:49.303037 10683 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:49.384596  6773 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:49.388219  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:49.292516 10606 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:26:49.391048  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:49.322357 10529 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:49.395124  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:49.236752  5696 rpcz_store.cc:269] 0709 12:26:06.552278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.567123 (+ 14845us) service_pool.cc:224] Handling call
0709 12:26:07.854697 (+1287574us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b540077f954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bed090 00007f39d2c8bf3a 00007f39d8a41e9c 00007f39d8b4004e 00007f39d70012a4 00007f39d7001232 00007f39d7001161 00007f39d70010dc 00007f39d7000392 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:49.195519 (+41340822us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:46.926042 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:46.926789 (+   747us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:46.926845 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:26:46.926887 (+    42us) tablet.cc:662] Decoding operations
0709 12:26:46.927913 (+  1026us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:46.928013 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:46.928033 (+    20us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:26:46.929322 (+  1289us) tablet.cc:701] Row locks acquired
0709 12:26:46.929384 (+    62us) write_op.cc:260] PREPARE: finished
0709 12:26:46.929585 (+   201us) write_op.cc:270] Start()
0709 12:26:46.929763 (+   178us) write_op.cc:276] Timestamp: P: 1783600006929511 usec, L: 0
0709 12:26:46.929809 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 12:26:46.930612 (+   803us) log.cc:844] Serialized 5119 byte log entry
0709 12:26:47.618481 (+687869us) op_driver.cc:464] REPLICATION: finished
0709 12:26:47.664339 (+ 45858us) write_op.cc:301] APPLY: starting
0709 12:26:47.664473 (+   134us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:47.737458 (+ 72985us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:47.751039 (+ 13581us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:48.084787 (+333748us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:48.085135 (+   348us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 12:26:48.085208 (+    73us) write_op.cc:312] APPLY: finished
0709 12:26:48.085715 (+   507us) log.cc:844] Serialized 120 byte log entry
0709 12:26:48.086755 (+  1040us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:48.086928 (+   173us) write_op.cc:454] Released schema lock
0709 12:26:48.087023 (+    95us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:48.087523 (+   500us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:48.087555 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":37603,"reported_tablets":44,"spinlock_wait_cycles":40568192,"child_traces":[["op",{"apply.queue_time_us":43855,"apply.run_cpu_time_us":23736,"apply.run_wall_time_us":423187,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58519,"num_ops":13,"prepare.queue_time_us":85190,"prepare.run_cpu_time_us":74050,"prepare.run_wall_time_us":154613,"raft.queue_time_us":373967,"raft.run_cpu_time_us":3946,"raft.run_wall_time_us":73491,"replication_time_us":681659,"thread_start_us":480051,"threads_started":4,"wal-append.queue_time_us":224754,"wal-append.run_cpu_time_us":54403,"wal-append.run_wall_time_us":1927145}]]}
I20260709 12:26:49.407262  6022 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:26:49.409724  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.405700  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:49.411772  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:49.439208 10529 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:49.390854 10682 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:26:49.453938 10676 raft_consensus.cc:3055] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:49.454172 10061 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:49.458678 10377 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:49.324476 10586 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 12:26:49.289875 10730 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:49.320436 10706 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 12:26:49.459007  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:49.411058  6015 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:49.392174 10683 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:49.438678 10750 consensus_queue.cc:1059] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.157s
I20260709 12:26:49.435731 10757 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:49.432118 10751 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.505792  6773 raft_consensus.cc:606] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
W20260709 12:26:49.506062  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 186139us
W20260709 12:26:49.514549  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 121592us
I20260709 12:26:49.515915 10676 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:49.534924  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.535279  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:49.396588 10689 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 12:26:49.547982  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:49.516024 10438 consensus_meta.cc:287] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.588s	user 0.015s	sys 0.000s
I20260709 12:26:49.563023  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:49.563340  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:49.578837  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:49.580868  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:49.646782  5870 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:49.581648  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:49.514132 10590 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:49.586583  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:49.588696  5867 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:49.589843  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:49.592281  6775 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:49.591554 10529 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:49.498255 10545 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:49.540135 10377 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:49.591755  5840 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:49.515537 10664 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:49.397750 10519 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:49.642935  5868 raft_consensus.cc:2410] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 12:26:49.650290  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 43862.9us
I20260709 12:26:49.652681  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:49.653921  6756 raft_consensus.cc:2410] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:26:49.657004  5872 raft_consensus.cc:2368] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12.
W20260709 12:26:49.661489  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 69793.8us
I20260709 12:26:49.667057  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:49.661794  5946 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:49.660353 10756 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:26:49.663453 10711 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:49.676254  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:49.658114 10602 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:49.684846  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:49.715812  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:49.716616  5946 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:49.717651  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:49.716166 10697 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:49.616938 10751 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:49.710528 10713 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.724772 10606 raft_consensus.cc:3055] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:49.753103  5946 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:49.756397  5838 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:49.756965  5946 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:49.758510 10710 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:49.758730 10692 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:49.759457 10710 raft_consensus.cc:697] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 12 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:49.748674 10753 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.419s	user 0.000s	sys 0.002s
W20260709 12:26:49.766868 10761 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
W20260709 12:26:49.767786 10724 consensus_meta.cc:287] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.597s	user 0.009s	sys 0.004s
I20260709 12:26:49.768210 10724 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:49.772826  5948 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:26:49.775712 10724 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:49.777514 10751 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:49.778914 10545 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:49.787096 10678 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.880s	user 0.013s	sys 0.004s
I20260709 12:26:49.787714 10678 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:49.793645 10306 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.746176 10711 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:49.799057 10739 consensus_meta.cc:287] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.580s	user 0.007s	sys 0.005s
I20260709 12:26:49.755307 10606 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:49.764288 10713 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:49.756333 10747 consensus_meta.cc:287] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.521s	user 0.013s	sys 0.000s
W20260709 12:26:49.802937 10695 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.700s	user 0.013s	sys 0.000s
I20260709 12:26:49.811510 10695 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:49.800832 10606 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:49.813602 10766 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:26:49.814623 10606 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:49.815796  5949 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:26:49.805446 10681 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.645s	user 0.009s	sys 0.004s
I20260709 12:26:49.818820 10377 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:49.819504 10681 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:49.794649 10306 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:49.806887 10712 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.696s	user 0.010s	sys 0.005s
W20260709 12:26:49.807376 10698 consensus_meta.cc:287] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.863s	user 0.016s	sys 0.000s
I20260709 12:26:49.823706 10678 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:49.810348 10713 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:49.709447 10505 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:49.739874 10697 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:49.836790  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:26:49.844813 10738 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.624s	user 0.006s	sys 0.005s
I20260709 12:26:49.760921 10710 consensus_queue.cc:237] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:49.851779  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 242800us
I20260709 12:26:49.849289 10712 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:49.931490  5870 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:49.845448 10621 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:26:49.938879  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 254276us
I20260709 12:26:49.941985  6015 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:49.944423  5840 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:49.946132  6022 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:26:49.810799 10747 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:49.907558 10738 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:49.850339 10698 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:49.949307  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:49.991724  5873 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:49.956890  5946 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:26:49.992857  5946 pending_rounds.cc:85] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 10
I20260709 12:26:49.952191 10773 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.995359 10773 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:49.937026  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:49.998368  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 61446.3us
I20260709 12:26:49.994880  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:49.958986 10695 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:50.002810 10687 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:50.003829 10687 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:50.005000 10687 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:49.959374  5947 raft_consensus.cc:2393] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 13: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:49.959015 10698 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:49.961549 10306 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:49.940903 10621 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:49.960415 10505 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:26:49.967275 10738 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:49.981369 10681 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:49.983109  6020 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:49.986097  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:50.028983  6023 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:49.987950  6019 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:50.011548  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:50.020956  5946 pending_rounds.cc:107] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:26:50.030542  6773 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:26:50.026129  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:50.029600  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:50.005367 10687 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:50.030622  5946 raft_consensus.cc:2889] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:50.036286 10621 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:49.955132 10689 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:50.017993 10426 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.052774  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:50.022027  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:50.056336  5871 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:50.055024  6015 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:50.040854  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:50.082434  5870 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:50.085417  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:50.087126  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:50.098300  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:50.051646 10739 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:26:50.032332 10738 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:50.089885 10692 consensus_queue.cc:1243] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:50.024080 10697 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:50.062456 10426 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:50.122793  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:50.109248 10689 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:50.125881  6766 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:50.084950 10683 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:50.155184 10683 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:50.156049  5944 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:50.156870 10426 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:50.159157 10773 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:50.159530  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:50.160010 10523 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:50.162461 10760 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.659s	user 0.000s	sys 0.004s
I20260709 12:26:50.135670  6015 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:26:50.123698 10751 consensus_queue.cc:1059] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:26:50.137917 10780 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:26:50.168867  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 839) took 1033 ms. Trace:
I20260709 12:26:50.144371 10783 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.114367 10687 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:50.170229 10783 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:50.171133 10602 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:50.171422 10687 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:50.171741 10602 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:50.145372  6611 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:50.148941 10777 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:50.150559 10779 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:50.165256 10676 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:50.150946 10738 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:50.151867 10786 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.018s
W20260709 12:26:50.129359  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 113090us
I20260709 12:26:50.188722  5873 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:26:50.194553 10061 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:50.135421 10689 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:50.169440 10751 raft_consensus.cc:3055] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:50.122913 10692 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:50.172966 10783 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:50.174834  6611 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:50.203483  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:50.217787  6766 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:50.218519  6766 raft_consensus.cc:606] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
W20260709 12:26:50.178665 10717 consensus_meta.cc:287] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.921s	user 0.008s	sys 0.007s
I20260709 12:26:50.184566  5943 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:50.226389  6785 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:50.226159  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:50.229246  6020 raft_consensus.cc:3055] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:50.229764  6020 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:50.230630  6020 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:50.231603  6020 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:50.176664 10779 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:50.175094  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1572) took 1192 ms. Trace:
I20260709 12:26:50.233927  5949 rpcz_store.cc:276] 0709 12:26:48.982753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.985974 (+  3221us) service_pool.cc:224] Handling call
0709 12:26:50.174780 (+1188806us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b5400673ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:50.174983 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1820684032}
I20260709 12:26:50.235229 10779 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:50.175742  5941 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:50.176391 10777 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:50.178354  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:50.179837  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:50.210685  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:50.238160 10777 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:50.238574  6015 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:50.176014 10739 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:26:50.243889 10783 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.218924  6766 raft_consensus.cc:614] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:50.218808  6762 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:50.244567 10783 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:50.220897  6022 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:50.183058  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 835) took 1224 ms. Trace:
I20260709 12:26:50.245425 10730 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:50.245801  5947 rpcz_store.cc:276] 0709 12:26:48.958554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.962273 (+  3719us) service_pool.cc:224] Handling call
0709 12:26:50.182781 (+1220508us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b54006725d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:50.182970 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2289957120}
I20260709 12:26:50.247059 10783 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:50.247634 10721 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:50.248142 10721 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:50.175341 10750 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:50.248884 10750 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.249303 10750 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:50.249135 10721 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:50.250416 10721 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:50.251015 10450 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:50.177484 10781 consensus_queue.cc:1059] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:26:50.252066  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:26:50.253756  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:50.254496  6021 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:50.169257  5945 rpcz_store.cc:276] 0709 12:26:49.135273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.135856 (+   583us) service_pool.cc:224] Handling call
0709 12:26:49.136308 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:49.140237 (+  3929us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:49.153863 (+ 13626us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.203236 (+ 49373us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b18001a55a0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb96557 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:26:49.290080 (+ 86844us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:49.341539 (+ 51459us) log.cc:844] Serialized 26 byte log entry
0709 12:26:49.378219 (+ 36680us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:49.378268 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.378351 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:50.152779 (+774428us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b18001a55a0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb96522 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:26:50.161646 (+  8867us) spinlock_profiling.cc:243] Waited 28.5 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:50.168600 (+  6954us) raft_consensus.cc:1685] finished
0709 12:26:50.168626 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.168764 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13474,"prepare.queue_time_us":17650,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":2270,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":9459,"spinlock_wait_cycles":106826240,"thread_start_us":135754,"threads_started":1,"wal-append.queue_time_us":277791}
I20260709 12:26:50.211360 10751 raft_consensus.cc:740] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:50.219660  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:50.256016  6611 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:50.260396  5865 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:26:50.260994  5865 pending_rounds.cc:85] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 7
I20260709 12:26:50.261267  5865 pending_rounds.cc:107] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:26:50.261547  5865 raft_consensus.cc:2889] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:50.262843  5872 raft_consensus.cc:2410] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:26:50.243939 10754 raft_consensus.cc:993] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: : Instructing follower 7a0204b931394fc48ffc503716e0d111 to start an election
I20260709 12:26:50.258267  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:50.249197 10689 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:50.270709  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 67375.8us
I20260709 12:26:50.278116  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:50.270495 10430 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:26:50.282451  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 287678us
I20260709 12:26:50.285410  6015 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:50.274173 10754 raft_consensus.cc:1081] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Signalling peer 7a0204b931394fc48ffc503716e0d111 to start an election
I20260709 12:26:50.300925  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:50.302923  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:50.305121 10777 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:50.306514 10777 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:50.275671 10586 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:50.312578  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:50.287571 10430 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.327870  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:50.331396 10529 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:50.328102 10794 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:50.329145 10430 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:50.335711  5990 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:50.337430 10682 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:50.308291 10689 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:26:50.337970 10682 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.338397  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:50.313666 10710 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:26:50.338429 10682 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:50.351209  5866 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:54876
I20260709 12:26:50.352015  5866 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:50.352360  5866 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:50.355934 10751 consensus_queue.cc:260] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:50.360571  5866 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:50.362848  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1270) took 1105 ms. Trace:
I20260709 12:26:50.363791  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:50.364568  6020 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:26:50.363432  5943 rpcz_store.cc:276] 0709 12:26:49.257341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.391291 (+133950us) service_pool.cc:224] Handling call
0709 12:26:50.184285 (+792994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.225391 (+ 41106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.362579 (+137188us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b54006725d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:50.362764 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1676410112}
I20260709 12:26:50.366534  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:50.365056  6020 pending_rounds.cc:85] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 6
I20260709 12:26:50.368718  6020 pending_rounds.cc:107] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:26:50.369076  6020 raft_consensus.cc:2889] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:50.431878  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: true dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:50.432629  6019 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:50.446198  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:50.430442 10770 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:26:50.463529  5866 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:50.432837 10777 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:50.482281  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1246) took 1262 ms. Trace:
I20260709 12:26:50.431704 10713 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:50.482736  6775 rpcz_store.cc:276] 0709 12:26:49.220141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.220313 (+   172us) service_pool.cc:224] Handling call
0709 12:26:49.222605 (+  2292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:49.649044 (+426439us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.482041 (+832997us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54002a0dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:50.482221 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":849651456}
I20260709 12:26:50.372695 10665 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.105s
I20260709 12:26:50.471381 10754 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.457779 10689 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:26:50.484140 10586 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:50.456459 10602 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:50.500685 10602 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:50.506242 10792 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
W20260709 12:26:50.509471  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.531983 10377 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:50.515866 10697 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:26:50.532965 10697 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:50.511776 10747 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:50.515465  5838 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:50.517877  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:50.523132  5868 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:50.525161  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:50.545047  5868 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:50.552742  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:50.520126 10755 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 1.135s	user 0.002s	sys 0.002s
I20260709 12:26:50.563232  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:50.546106  5868 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:50.570305  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 54940.7us
I20260709 12:26:50.573778  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:50.492755 10777 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:26:50.578765  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:50.489444 10713 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:50.579830  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:50.525585 10772 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:50.584522  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:50.544040 10697 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:50.559499 10592 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:50.591550  5870 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:26:50.591756  5943 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:26:50.594043  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1172) took 1255 ms. Trace:
I20260709 12:26:50.594497 10430 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:50.594746 10450 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:50.595535 10450 consensus_queue.cc:1243] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:50.559725 10721 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:50.597555 10772 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:50.598260 10721 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:50.600189 10772 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:50.602207 10721 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:50.559394 10776 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.602444 10697 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:50.603209 10776 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:50.604012 10697 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:50.604081 10430 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:50.603235  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:50.607534  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1072) took 1634 ms. Trace:
I20260709 12:26:50.607975  6018 rpcz_store.cc:276] 0709 12:26:48.973137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.016003 (+ 42866us) service_pool.cc:224] Handling call
0709 12:26:49.035355 (+ 19352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:49.044709 (+  9354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:49.044755 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:49.058119 (+ 13364us) log.cc:844] Serialized 26 byte log entry
0709 12:26:49.181694 (+123575us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:49.181738 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.181808 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:50.519438 (+1337630us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:50.607218 (+ 87780us) raft_consensus.cc:1685] finished
0709 12:26:50.607240 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.607474 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8774,"raft.queue_time_us":9754,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":13507,"spinlock_wait_cycles":8194816,"thread_start_us":87638,"threads_started":1,"wal-append.queue_time_us":220061}
I20260709 12:26:50.609340 10713 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:50.594552  6024 rpcz_store.cc:276] 0709 12:26:49.372070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.372290 (+   220us) service_pool.cc:224] Handling call
0709 12:26:49.402421 (+ 30131us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:49.402819 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:49.402851 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:49.405522 (+  2671us) log.cc:844] Serialized 26 byte log entry
0709 12:26:49.535477 (+129955us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:49.564132 (+ 28655us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.564373 (+   241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.564458 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:50.190544 (+626086us) spinlock_profiling.cc:243] Waited 9.33 ms on lock 0x7b180006a840. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:26:50.359729 (+169185us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b180006a840. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:26:50.593768 (+234039us) raft_consensus.cc:1685] finished
0709 12:26:50.593798 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.593940 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24939,"prepare.queue_time_us":39152,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":34304,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":28581,"spinlock_wait_cycles":63313664,"thread_start_us":77787,"threads_started":1,"wal-append.queue_time_us":148117}
I20260709 12:26:50.497164 10754 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:50.619062  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:50.619778 10754 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:50.621687 10796 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.289s	user 0.003s	sys 0.000s
I20260709 12:26:50.622473  5943 pending_rounds.cc:85] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 5
I20260709 12:26:50.622839  5943 pending_rounds.cc:107] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 12:26:50.623085  5943 raft_consensus.cc:2889] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:50.583089 10689 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:50.631817 10777 consensus_queue.cc:1243] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:50.633375 10717 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:50.634277 10797 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:50.634405 10717 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:50.635874 10710 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:50.636341 10710 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:50.637331 10710 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:50.638727  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.639415 10717 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:50.642266 10678 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:50.643000 10678 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:50.645134 10678 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:50.647540 10750 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:50.649950 10759 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:50.650573 10759 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.651005 10759 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:50.651458  5870 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:50.676340  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.680022  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:50.681895  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:50.586388 10747 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.682965  5944 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:50.695688  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:50.704421  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 258374us
I20260709 12:26:50.699815  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:50.670387 10776 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:50.709647  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:50.713575  5870 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:50.718701  5987 leader_election.cc:400] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12.
I20260709 12:26:50.719134  5987 leader_election.cc:403] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:26:50.721205  6019 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:26:50.709599 10681 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:50.733130  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:50.695534 10747 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:50.717860 10678 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:50.740433  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:50.708247 10750 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:50.739662 10519 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:50.727038 10721 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:50.727943 10682 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:50.760004  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 927) took 1091 ms. Trace:
I20260709 12:26:50.734483 10681 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:50.740010 10678 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.760568  5865 rpcz_store.cc:276] 0709 12:26:49.668339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.668494 (+   155us) service_pool.cc:224] Handling call
0709 12:26:49.722467 (+ 53973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.262171 (+539704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.758048 (+495877us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54005d6bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:50.759923 (+  1875us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235779456}
I20260709 12:26:50.769318  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:50.770325  5870 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:50.778384  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.753208 10698 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:50.753505 10786 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.786876  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:50.788053  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:50.790459  5866 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:50.791401  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:50.794248  5945 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:50.796227  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:50.796980  5945 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 10.
I20260709 12:26:50.765554 10678 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:50.799757  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:50.767730 10676 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:50.761245 10721 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:50.831593  5865 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:50.814469  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:50.799602 10677 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:50.838678  5940 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
W20260709 12:26:50.834663 10806 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:26:50.813429 10438 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:50.848646 10438 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:50.806305 10675 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:26:50.830741 10697 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.805900 10776 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:50.835057 10682 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:50.801343 10698 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:50.867755  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1265) took 1620 ms. Trace:
W20260709 12:26:50.873121 10805 log.cc:927] Time spent T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.164s	user 0.000s	sys 0.002s
I20260709 12:26:50.874739 10769 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:50.801733 10786 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:50.880465 10804 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:26:50.878239  5941 rpcz_store.cc:276] 0709 12:26:49.247314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.247434 (+   120us) service_pool.cc:224] Handling call
0709 12:26:50.175485 (+928051us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.236167 (+ 60682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.867535 (+631368us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b5400673ed0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:50.867688 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2053066496}
I20260709 12:26:50.818281 10681 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:50.834965 10810 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12.
I20260709 12:26:50.894694 10710 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:50.864770 10677 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:50.865674 10747 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:50.878197 10697 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:50.919776  5870 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:50.920674  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:50.885421 10776 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:50.922153  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 124083us
I20260709 12:26:50.892289 10769 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:50.927738  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:50.916431 10747 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.938638 10438 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:50.940060  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:50.942502  5870 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:50.947755 10377 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.948508 10377 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:50.950541  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:50.951012 10677 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:50.942294 10747 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:50.952353 10523 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:50.952811 10523 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:50.953284 10697 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:50.956986  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:50.959041  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:51.010249  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 49220.4us
I20260709 12:26:51.018738  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:51.021108  5945 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:50.957083 10377 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:50.960515 10697 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:51.028532  5838 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:51.031919  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:51.032956  5865 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:51.035772  5943 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:26:51.009500 10545 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.018712 10592 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:51.047043 10815 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:26:51.048920 10809 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.139s	user 0.001s	sys 0.001s
I20260709 12:26:50.957829 10523 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:51.025010 10747 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:51.018225 10772 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:26:51.043499 10611 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:51.071307  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 57912.9us
I20260709 12:26:51.072530  5913 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:51.075513  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:51.050974 10730 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:51.056593 10721 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:51.038250 10697 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:51.082710  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.084468  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:51.062031 10602 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:51.044390 10545 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:51.066412 10717 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:51.087744  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:51.088186 10674 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:51.081828 10730 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:51.078857 10786 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:51.080904 10777 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:51.082345 10698 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:51.101981  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:51.107681  5944 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:26:51.103202  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:51.082571 10769 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:51.096612 10377 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:51.118557  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:51.105655 10671 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:51.118814 10678 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:51.103622 10523 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:51.100363 10674 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:51.099016 10602 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:51.125173 10814 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:26:51.108426 10818 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.118095 10377 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.103981 10717 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:51.132990 10759 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:51.126520 10671 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:51.159124 10671 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:51.159164  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:51.127213 10674 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:51.146167 10545 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:51.142474 10818 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:51.164172 10818 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:51.166240 10717 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:51.146937 10681 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:51.127388 10776 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:51.117969 10675 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:26:51.173306 10675 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:51.173764 10675 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:51.174835 10675 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:51.176319 10675 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:51.178736 10674 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:51.153898 10697 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:51.149348 10811 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.371s	user 0.001s	sys 0.008s
I20260709 12:26:51.142622 10377 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:51.180137  5942 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:51.226548 10611 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:51.183151 10759 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:51.195312 10678 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:51.196247  6756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 966) took 2275 ms (client timeout 2168 ms). Trace:
W20260709 12:26:51.229889  6756 rpcz_store.cc:269] 0709 12:26:48.920737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.921746 (+  1009us) service_pool.cc:224] Handling call
0709 12:26:51.195949 (+2274203us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b5400007154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:51.196124 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1662794112}
I20260709 12:26:51.200500  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:51.232508  5944 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:26:51.235177  6021 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:51.151443 10602 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:51.204283  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:51.237123  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.237212 10710 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.239346  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:51.207685  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:51.211589 10779 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:51.205809  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:51.213742  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:51.241264  5870 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:26:51.241541  5874 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:51.214655 10797 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.242724 10797 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:51.243875  5867 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:51.245023  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.245153  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:51.239691 10710 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:51.246506  6762 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:51.247476  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:51.247499  6775 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:26:51.248193 10710 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:51.248576  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:51.249193 10677 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.249760 10677 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:51.250926 10779 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:51.251938 10677 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:51.249776  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.249536  5869 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:26:51.215133 10689 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.589s	user 0.015s	sys 0.000s
I20260709 12:26:51.216089  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:51.216933  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:51.261173  5943 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:51.218021  5868 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:51.262988  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:51.235296  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.265286  6022 raft_consensus.cc:2393] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:51.251960 10779 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:51.266062 10779 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:51.246567 10687 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:51.267954 10687 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:51.270398 10687 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:51.271857  6793 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:26:51.276759  6793 pending_rounds.cc:85] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 7
I20260709 12:26:51.277066  6793 pending_rounds.cc:107] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:26:51.253706  6020 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
W20260709 12:26:51.274578  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 58569.2us
I20260709 12:26:51.275184  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.270830  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:51.278290  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:51.278817  6756 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:26:51.254797  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:51.247607  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:51.279981  6778 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:51.240728  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.277360  6793 raft_consensus.cc:2889] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:51.272681  6768 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:51.235939 10061 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:51.257032  6762 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:51.237076 10717 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.270510  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:51.282370  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:51.257062  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.271541  6766 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: [14.8-15.9]
I20260709 12:26:51.285216  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:51.291090  5868 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:26:51.291594  5868 pending_rounds.cc:85] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 8
I20260709 12:26:51.291865  5868 pending_rounds.cc:107] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:26:51.292431  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:51.287983  5943 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:51.292083  5868 raft_consensus.cc:2889] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:51.231931 10671 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:51.298812  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1604) took 1467 ms. Trace:
I20260709 12:26:51.299219  5868 rpcz_store.cc:276] 0709 12:26:49.831099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.831860 (+   761us) service_pool.cc:224] Handling call
0709 12:26:49.832327 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.522987 (+690660us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b5400555da0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bfadc0 00007f39d2c851c0 00007f39d2c86231 00007f39d2c82d8f 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c
0709 12:26:51.217774 (+694787us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b58007259d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bdf0aa 00007f39d2c61447 00007f39d2c79fe7 00007f39d2c7b7de 00007f39d2c85b49 00007f39d2c86416 00007f39d2c82d8f 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4
0709 12:26:51.298530 (+ 80756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.298747 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1316361856}
I20260709 12:26:51.311686  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:51.315459  6020 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:51.316728 10061 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:51.317828 10438 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:51.319368 10061 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:51.320255  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:51.289000 10611 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:51.321376 10611 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:51.322487 10611 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:51.325255 10713 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:51.326880  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:51.335475  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.328667  6757 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:51.336619  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:51.329293  5870 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:51.335642  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:51.329258  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.339126  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:51.339627  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:26:51.327924  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.341280  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.342481  6019 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 11: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:51.336663  6022 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:51.277356 10689 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:51.275434 10678 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:51.349908  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.355234  6024 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:26:51.356071  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:51.334131 10711 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:26:51.283080 10754 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.353260  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:51.358122  6016 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:51.357504  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:51.360111  5874 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:51.361097  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:51.362150  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.362998  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.345278 10750 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:51.355031 10061 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:51.345731 10808 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:51.370849  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.347112 10786 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:51.347945 10692 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:51.375505  6019 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:51.334431 10782 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:26:51.376686  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.365130 10438 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:51.383118 10674 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:51.384879 10671 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:51.345526 10697 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:26:51.379773 10692 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:51.386634 10692 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:51.380564  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:51.335003 10717 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:51.392506  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:51.354295 10677 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:26:51.394860  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:51.395677  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:51.373440 10061 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:51.397403 10438 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:51.398046  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:51.407315  5870 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 10: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:26:51.403329  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 67094.3us
I20260709 12:26:51.416123  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.379884 10678 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:51.420915 10834 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:51.367373 10754 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:51.423970  6015 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:51.416427  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:51.426129 10759 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:26:51.389425 10808 raft_consensus.cc:697] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:51.429425 10815 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:51.445207  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:51.436645  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:51.437561  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:51.437311  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:51.409866 10061 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:51.447103  6015 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:51.442608 10834 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:51.458664 10679 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:51.460209 10828 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:26:51.462210 10377 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:51.471737  6778 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
W20260709 12:26:51.474118  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 78555.5us
I20260709 12:26:51.478462  6018 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:51.432286 10611 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.490602  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:51.485070  6017 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:51.446709 10671 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:51.444633 10815 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.506273 10815 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:51.415426 10820 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:26:51.512742 10717 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:51.513732 10834 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:51.514485 10783 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:51.514963 10834 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:51.515074 10783 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:51.515700 10834 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:51.490167 10832 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:26:51.517562 10783 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:51.520931  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.521476  6019 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:26:51.521955 10665 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:51.485673 10679 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:51.496279  6015 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:51.494091 10671 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:51.496816 10438 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:51.523818  5944 raft_consensus.cc:2368] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11.
W20260709 12:26:51.514230 10830 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:26:51.494568  6609 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:51.522850 10665 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:51.522040  6019 pending_rounds.cc:85] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 8
I20260709 12:26:51.533119  6019 pending_rounds.cc:107] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:26:51.533427  6019 raft_consensus.cc:2889] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:51.528698 10808 consensus_queue.cc:237] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:51.484032 10592 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:51.534566 10671 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:51.538329  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:51.539029  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.543201  6015 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:51.534644  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:51.544081  6021 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:51.546784  6768 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 16: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:51.549414  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:51.559718  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:51.566571  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:51.527319 10674 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:51.567430  6020 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:51.567179  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:51.521783 10730 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:51.575598  6609 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:26:51.594331  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.547854 10671 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:51.524631 10611 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:51.599282  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:51.570377 10783 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:51.601831  6611 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:51.602607  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:51.568018 10592 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:51.558435 10677 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:51.599404  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:51.617571  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:51.606343  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:51.613746 10061 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:51.619108  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:51.619247  6766 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:51.617967  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:51.621050  5915 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:51.571300 10678 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:51.614988  5870 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:51.585197 10697 consensus_queue.cc:1059] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:26:51.616017  6611 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:51.590385 10713 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:51.602346 10757 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:51.628026 10061 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:51.631706  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.615020 10783 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.639180  5915 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:51.641667  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:51.643230  5915 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:51.618397 10786 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:51.653594  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:51.623440 10776 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:26:51.627741 10836 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.155s	user 0.000s	sys 0.002s
I20260709 12:26:51.625727 10677 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:51.635825 10772 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:51.626114 10834 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:51.628472 10757 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.646281  5915 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:51.664206  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:51.714023  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 49928.5us
I20260709 12:26:51.627600 10713 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.647125 10678 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:51.652930  6611 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:51.692535 10611 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:51.649109 10783 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:51.699611 10377 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:51.649878 10721 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:51.712184 10833 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
W20260709 12:26:51.667302 10835 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.237s	user 0.000s	sys 0.002s
I20260709 12:26:51.650687 10702 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:51.650261 10624 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:51.656982 10665 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:51.719337  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 73204.1us
I20260709 12:26:51.657753 10430 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:26:51.723711 10746 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:51.738628  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.718333 10713 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:51.718119 10757 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:51.686169 10747 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:51.749665  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:51.733083 10820 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:51.755648 10679 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:51.757319 10678 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:51.787792 10678 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.788357 10678 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:51.788997  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:51.790113  6793 raft_consensus.cc:2410] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:26:51.790220  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:51.747459 10624 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.766881  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:51.798563  6023 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:51.815279  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:51.816289  6756 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:51.762154 10721 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:51.818312  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:51.818512  6793 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:51.762283 10772 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.822405  6756 raft_consensus.cc:2368] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15.
I20260709 12:26:51.773654 10746 raft_consensus.cc:697] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:51.797351 10624 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:51.853482  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 200620us
I20260709 12:26:51.868467 10779 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:51.869321 10779 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:51.872138 10779 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:51.893293  6023 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
W20260709 12:26:51.901533  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 171444us
I20260709 12:26:51.826048 10721 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:51.772927 10747 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:51.829295 10730 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:51.913924  6756 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:51.797371 10677 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:51.831513 10772 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:51.955529  6756 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:51.971283  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:51.974193 10772 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:51.957537 10721 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:51.983175 10747 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:51.925463 10746 consensus_queue.cc:237] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:52.015040  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:52.015340  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: true dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:52.018869  6775 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:52.015084  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:52.019245  6775 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:52.024920  6775 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:52.033140  6775 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:52.035391 10692 consensus_meta.cc:287] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.648s	user 0.016s	sys 0.000s
I20260709 12:26:52.035931 10692 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:52.036789 10840 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:52.037343 10840 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:52.008955 10815 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.053333 10713 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:52.053614 10783 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:52.026000 10754 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:52.056334  6768 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:52.056094 10751 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:52.053263  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 158128us
I20260709 12:26:52.056555 10750 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:52.058946 10750 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:52.060612  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:52.061131 10750 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:52.061374  5865 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
W20260709 12:26:52.067579 10855 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:26:52.068464  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:52.068990  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:52.069777  6762 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:52.071175 10061 raft_consensus.cc:3055] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:52.071583 10061 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:52.072328 10061 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.074672  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:52.075397  6793 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:26:52.076399  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:52.073489 10061 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:52.084656  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:52.085508  6773 raft_consensus.cc:2368] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15.
I20260709 12:26:52.088321  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:52.090240  6756 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:52.089268  6762 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:26:52.091863  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:52.093818  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:52.089489  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:52.098886  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:52.100181  5949 raft_consensus.cc:2368] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12.
I20260709 12:26:52.103315  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:52.111263 10797 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:52.086138 10665 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:52.086338 10842 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:52.112579 10665 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.112741 10842 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.085819 10838 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:52.114177 10838 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.115123 10665 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:52.115499 10842 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:52.116376 10838 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:52.117213  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:52.118000  5867 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:52.100798 10754 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
W20260709 12:26:52.119233  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.122684  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:52.123478  5942 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:52.126077  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:52.127686  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:52.128350  6019 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:52.128177  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:52.130223  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:52.132989  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:52.133463  6021 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:52.121419  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:52.124148  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:52.136152  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:52.141161  6775 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:26:52.144054  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:52.147225  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:52.153415  6019 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
W20260709 12:26:52.154404  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.053069 10678 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:52.158855 10061 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:52.162586 10061 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:52.165807  5838 leader_election.cc:400] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12.
I20260709 12:26:52.159968  6609 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:52.167059  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:52.167838  6020 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:52.166445  5838 leader_election.cc:403] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:52.172029 10061 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:52.174193  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:52.156334 10838 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:52.175019  5947 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:52.176113  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:52.176748  5947 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:52.177906  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:52.155038 10750 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:52.180039 10750 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:52.180469 10750 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.180819 10750 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:52.142343 10687 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.182036 10687 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.184206 10687 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:52.187280  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:52.189497 10674 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:52.190634  5866 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:52.167050 10797 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
W20260709 12:26:52.174688 10854 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 12:26:52.168224 10747 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:52.200403  5838 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:52.201941  6611 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:52.202795 10611 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:26:52.204159  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:52.204819  5871 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:52.205883 10834 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:52.208190  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:52.209035  6016 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:52.210343  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:52.211095  6016 raft_consensus.cc:2393] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:52.211824  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:52.212363  6016 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:26:52.168867 10692 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:52.218353 10772 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:52.218744 10624 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:52.219498 10772 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:52.219841 10772 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:52.209308 10808 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.220899 10624 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:52.220628 10808 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.221952  5944 raft_consensus.cc:2368] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16.
I20260709 12:26:52.223526 10808 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:52.209070  5870 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
W20260709 12:26:52.223908 10671 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.621s	user 0.007s	sys 0.006s
I20260709 12:26:52.228878 10783 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:52.269809  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:52.237299  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:52.270932  6023 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:52.202064  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:52.230222  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:52.231103 10545 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:26:52.230942 10820 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:52.234580 10611 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12.
I20260709 12:26:52.235301 10689 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:52.237275 10786 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:52.239593 10838 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:52.240156 10713 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:52.303397  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:52.240805 10678 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:52.241142  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:52.304908  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:52.305089  6793 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:26:52.241569  6757 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:52.308118  6775 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:26:52.242192  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:52.242409  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:52.245007  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:52.310078  6020 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:52.246929  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:52.249728  5990 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:52.311058  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:52.311933  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 62318.2us
I20260709 12:26:52.252933  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:52.255558  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:52.254842  5692 catalog_manager.cc:5697] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 9 to 15, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 15 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:52.314277  6018 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 11: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:52.257340  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:52.315263  6022 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:52.315786  5990 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:52.257225  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:52.259236  6017 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:52.260059 10797 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.229135  6015 raft_consensus.cc:2410] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:26:52.272197  5871 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:52.220610 10772 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:52.279477  6609 leader_election.cc:400] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16.
I20260709 12:26:52.286054  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:52.321383  6609 leader_election.cc:403] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:52.286811  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 83651.7us
I20260709 12:26:52.324576  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:52.287137  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:52.296376 10710 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:52.326655 10710 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:52.327204 10710 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:52.326224  5692 catalog_manager.cc:5697] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 12 to 15, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 15 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:26:52.276398  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:26:52.328890  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 52644.2us
I20260709 12:26:52.288782 10061 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:52.330493 10061 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.330865 10061 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:52.330940  5987 leader_election.cc:400] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11.
I20260709 12:26:52.331454  5987 leader_election.cc:403] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:26:52.282222  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:52.266453 10840 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:52.285845 10858 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.250s	user 0.004s	sys 0.000s
I20260709 12:26:52.334156  6762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:52.339054  6762 raft_consensus.cc:606] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:26:52.292076 10821 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:52.291379  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:52.293213  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:52.296409  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:52.298943  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68816.1us
I20260709 12:26:52.300973 10786 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:52.247004  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:52.325805  6023 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:52.326508  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:52.288753 10839 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:52.391150  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:52.394447  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 84790.3us
I20260709 12:26:52.397244  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:52.399365  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:52.328544  5692 catalog_manager.cc:5697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:52.334817  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:52.337491  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:52.339495  6762 raft_consensus.cc:614] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:52.389094  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 52971.2us
I20260709 12:26:52.340835  6773 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 16: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:52.326591 10772 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:52.340831  5870 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:52.414696  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:52.403733  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:52.342916  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:52.422683  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 79885.2us
I20260709 12:26:52.330924 10671 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:52.340341 10779 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:52.427867  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 103450us
I20260709 12:26:52.428931  5692 catalog_manager.cc:5697] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 7 to 11, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 11 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:52.420920 10061 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.435354  6016 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:26:52.389120 10687 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:52.392944 10821 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:52.435607  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:52.333509 10624 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:52.333765 10713 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:26:52.393028 10674 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11.
I20260709 12:26:52.405774 10815 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:52.440505  5838 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:52.485947  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 45576.5us
I20260709 12:26:52.487347  5838 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:52.413555 10665 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.434124  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:52.450588  5866 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 13: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:52.426966 10692 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:52.455345  6023 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:26:52.430907 10676 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:52.431813 10839 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:52.461356  5692 catalog_manager.cc:5697] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:52.334142 10797 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.479123 10061 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:52.341018 10824 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:52.444533 10721 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:52.501820  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 67808.8us
I20260709 12:26:52.453878 10779 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:52.498067  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:52.534807  5947 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:52.453142 10711 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:52.522642  5692 catalog_manager.cc:5697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 9 to 17, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 17 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
W20260709 12:26:52.493768 10757 consensus_meta.cc:287] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.700s	user 0.013s	sys 0.001s
I20260709 12:26:52.444375 10838 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:26:52.539095  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:52.542699 10721 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:52.462579 10821 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:52.508137 10664 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:52.500924 10689 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:52.530234  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 139228us
I20260709 12:26:52.501405 10786 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:52.539207 10711 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:52.556294  5692 catalog_manager.cc:5697] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:52.508569 10834 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:52.501066 10692 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:52.541528 10779 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:52.546263 10674 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.508991 10665 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.527711 10824 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:52.651943  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 349183us
I20260709 12:26:52.560654 10757 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:52.661496  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 122452us
I20260709 12:26:52.662220 10505 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:52.663041 10717 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:52.663331 10797 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:52.663869 10861 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.263s	user 0.000s	sys 0.002s
W20260709 12:26:52.664942  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 97910.5us
I20260709 12:26:52.508790 10842 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:52.668694 10061 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.720857 10061 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:52.688700  5947 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
W20260709 12:26:52.688818  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 253278us
I20260709 12:26:52.695273  5915 leader_election.cc:400] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15.
I20260709 12:26:52.696403 10692 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:52.724131  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:52.698227  5692 catalog_manager.cc:5697] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 4 to 13. New cstate: current_term: 13 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:52.669142 10664 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.706089  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:52.728602  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:52.732465  6016 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:52.706985  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:52.734001  6023 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:52.668318 10689 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:52.691588 10711 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:52.716053  6766 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:52.716506 10505 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:52.744705 10505 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:52.745854 10624 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:52.746657 10710 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:52.749480 10872 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:52.750222 10872 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.752453 10872 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:52.718537  6768 raft_consensus.cc:2368] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16.
I20260709 12:26:52.697088 10679 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:52.738534 10823 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:52.758025 10808 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.727339  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:52.759485  5947 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
W20260709 12:26:52.760403  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 58847.3us
W20260709 12:26:52.734828  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 216491us
I20260709 12:26:52.761849  5913 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:52.763137  5913 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:52.736507 10664 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:52.758569 10808 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:52.735142  5692 catalog_manager.cc:5697] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 7 to 14, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 14 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:26:52.765359  5838 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:52.761324 10751 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.703s	user 0.010s	sys 0.003s
W20260709 12:26:52.787773  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 45622.2us
I20260709 12:26:52.793480  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:52.794291 10879 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:52.794241  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:52.797586  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:52.703873 10674 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.797715  5692 catalog_manager.cc:5697] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 16. New cstate: current_term: 16 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:52.803527  5942 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:52.805564 10061 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:52.806489  6778 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:52.807369  6016 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:52.807843  5946 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 13: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:52.808897 10665 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:52.772468 10721 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:52.809652 10665 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:52.814476 10808 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:52.804939 10818 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:52.798949 10823 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:52.818295 10823 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.741683 10759 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:26:52.818694 10823 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:52.819198 10674 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:52.819815 10759 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:52.773612 10842 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:52.825023 10750 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.644s	user 0.009s	sys 0.002s
I20260709 12:26:52.741731 10689 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:52.825457 10879 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:52.703544 10868 consensus_queue.cc:1059] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:26:52.862259  5944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df"
dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:48330
I20260709 12:26:52.825903  5692 catalog_manager.cc:5697] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:52.862942  5944 raft_consensus.cc:606] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:26:52.863317  5944 raft_consensus.cc:614] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:52.831458  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:52.838061  5946 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:52.866472  6766 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:26:52.874353  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:26:52.877015  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 82892.1us
I20260709 12:26:52.877938  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:52.879443  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:52.840129 10863 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:26:52.884371  5692 catalog_manager.cc:5697] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:52.886572  5872 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:52.889499  5692 catalog_manager.cc:5697] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 13. New cstate: current_term: 13 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:52.891047  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:52.891641  5873 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:52.850536 10868 raft_consensus.cc:3055] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:52.825023 10820 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:52.830974 10713 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16.
I20260709 12:26:52.826157 10592 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.825134 10840 raft_consensus.cc:3055] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:52.828039 10879 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:52.910244  5942 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:52.900648  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:52.880856 10677 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:52.895910  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:52.915463  6020 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:52.896039  5840 leader_election.cc:400] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16.
I20260709 12:26:52.916574  5840 leader_election.cc:403] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:26:52.900166  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:52.899530  5692 catalog_manager.cc:5697] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 7 to 11, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 11 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:26:52.905313  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:52.907505  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:52.908831 10823 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.861006 10842 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:52.851233 10877 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:52.853806 10757 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:52.884073 10730 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:52.896205  5866 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:52.858798 10824 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:52.914119  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 176097us
I20260709 12:26:52.884492 10751 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:52.882467 10717 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:52.921231  5867 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:52.922360 10061 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:52.825620 10750 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:52.916090 10754 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:52.925415  5692 catalog_manager.cc:5697] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 7 to 12, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 12 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:52.956162  5692 catalog_manager.cc:5697] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 12, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 12 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:52.857440 10779 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.910300 10848 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:52.894999 10868 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:52.899698 10820 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:52.911000 10840 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:52.945067  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:52.946028  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:52.933043 10877 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.947513  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:52.940277 10875 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.124s	user 0.001s	sys 0.001s
I20260709 12:26:52.843276 10821 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:52.948834  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:52.958774  5941 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:52.954279 10061 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.961393  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:52.964355  5692 catalog_manager.cc:5697] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:52.976773  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:53.005903  6016 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:52.978588  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:52.979532  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:52.913411 10592 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:52.988467  6020 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:52.991214  5942 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:52.941154 10713 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.031242  6611 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:53.033331  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:52.995491  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 188289us
I20260709 12:26:53.034017  6757 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:53.023629  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:53.034690  6757 raft_consensus.cc:606] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:26:53.034337  5915 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:52.976320 10848 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:53.014392  5867 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:53.022938  5944 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:53.032020  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:53.035163  6757 raft_consensus.cc:614] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:52.930066 10847 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:26:53.035753  5692 catalog_manager.cc:5697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:53.049113 10815 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:52.962883 10717 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:53.043814  6793 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:26:52.963358 10754 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:53.094596  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:53.058650  6775 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:53.096714  5692 catalog_manager.cc:5697] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:26:53.054795  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:53.049746  5915 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:53.092115 10876 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:26:53.118258  5692 catalog_manager.cc:5697] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:53.111788  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 57073.3us
I20260709 12:26:52.996827 10877 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:53.057926 10671 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:53.052145 10519 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.052616 10821 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:53.152796  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 103203us
I20260709 12:26:53.051164 10834 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16.
I20260709 12:26:53.187908  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:53.169353  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:53.109506 10856 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:26:53.092679 10730 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:53.013332 10746 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 12:26:53.161373  5692 catalog_manager.cc:5697] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:53.113382 10882 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:26:53.178591 10750 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:53.179651 10823 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:53.155187 10678 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:53.051723 10842 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.183213 10754 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:53.057353 10747 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.103s
I20260709 12:26:53.095463 10815 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:26:53.184713  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 154465us
I20260709 12:26:53.113853 10781 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 12:26:53.190639  5915 leader_election.cc:400] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15.
I20260709 12:26:53.043824 10848 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:53.195716 10679 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:53.087917 10808 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:26:53.154765 10721 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:53.043995 10713 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:53.166875 10880 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:53.089697 10820 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:53.193724 10519 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.253372  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:53.243682 10671 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.258553  5915 leader_election.cc:403] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:53.278549  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 88164.3us
I20260709 12:26:53.279536  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:53.090067 10868 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:53.197876 10884 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:53.187513  5872 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:53.255659 10815 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:53.244339 10730 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:53.275522 10880 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:53.285686  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 254555us
I20260709 12:26:53.297273  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:53.294715  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:53.272919 10664 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.302021  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:53.302438  5944 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
W20260709 12:26:53.301268  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 95614.3us
I20260709 12:26:53.304298  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:53.273450 10713 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:53.294735  5692 catalog_manager.cc:5697] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 9 to 13, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 13 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:53.287213  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:53.309113  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:53.312505  6020 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:53.319811  6757 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:26:53.327435  5872 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:53.284565 10721 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15.
I20260709 12:26:53.334846  5692 catalog_manager.cc:5697] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:53.335510  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:53.356775  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:53.357755  6016 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:53.296526 10702 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:53.297313 10880 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:53.366827  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 79722.8us
I20260709 12:26:53.295656 10891 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:53.366185  5692 catalog_manager.cc:5697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:53.295475 10884 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:53.370808  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 60639.1us
I20260709 12:26:53.290061 10671 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:53.289968 10689 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:53.366183 10876 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:26:53.388499  6016 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:53.336741 10779 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:53.354606 10838 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.337203 10894 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:53.361224 10757 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:53.398473  5692 catalog_manager.cc:5697] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:26:53.370559 10891 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.414726 10624 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:53.415776 10624 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:53.390005 10730 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:53.420001 10664 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:53.420403 10671 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:53.403972 10890 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:26:53.411000  6785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:53.414076 10839 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.393230 10545 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:26:53.415068 10891 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:53.423207  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:53.409914 10519 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:53.427742  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.453356  6017 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:26:53.428519  5692 catalog_manager.cc:5697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 7 to 15, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 15 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:26:53.396701 10894 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.435029  5867 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:53.459312  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:53.459986  6018 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 14: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:53.437493  6785 raft_consensus.cc:606] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:26:53.390360 10754 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.417008 10624 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.435393 10702 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.455035  5692 catalog_manager.cc:5697] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:53.393165 10711 consensus_meta.cc:287] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.608s	user 0.011s	sys 0.004s
I20260709 12:26:53.456883  6762 raft_consensus.cc:2393] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 14: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:53.460930  6785 raft_consensus.cc:614] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:53.443027 10887 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.197s	user 0.003s	sys 0.001s
I20260709 12:26:53.435138 10871 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:53.462886  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:53.426024 10757 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:53.442166  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:53.438151 10781 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:53.422266 10896 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:53.472909  5867 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:53.444347  6020 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:53.398058 10838 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.513701 10838 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:53.496022  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:53.514376  6016 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:53.447963 10717 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:53.497526  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 162139us
I20260709 12:26:53.395180 10797 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:26:53.505982  6024 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:53.505002  5692 catalog_manager.cc:5697] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 10 to 13, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 13 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:26:53.460985 10894 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:53.511368 10781 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:53.524761  5692 catalog_manager.cc:5697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:53.492354 10754 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:53.515790 10871 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:53.541399  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:53.520735 10896 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:53.497701 10711 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.550086 10871 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.553547 10689 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:53.554016 10781 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:53.554817 10710 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:53.450788 10839 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.555969  5692 catalog_manager.cc:5697] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 8 to 12, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 12 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:26:53.558182  5874 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:53.559702 10711 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:53.563549 10061 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:53.564404  5692 catalog_manager.cc:5697] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 9 to 13, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 13 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:26:53.586560  6024 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
W20260709 12:26:53.588121  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 205110us
I20260709 12:26:53.543861  6016 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:26:53.585004  6611 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:53.595238 10430 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:53.500422 10702 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.610100 10430 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:53.607722 10545 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:53.613766 10545 raft_consensus.cc:697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:53.615085 10430 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:53.614959 10545 consensus_queue.cc:237] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:53.595160 10781 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:53.616307 10781 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:53.618616 10781 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:53.621282  6756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:53.632179  5990 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:53.640705 10904 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:26:53.641618 10687 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:53.642305 10687 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.642684 10687 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:53.492714 10884 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:53.646445 10754 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:53.649425 10519 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.650441 10519 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:53.652088 10754 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.652490 10519 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:53.652875 10754 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:53.658143  6024 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:53.654961  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:53.655656  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:53.659111  6016 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:53.662510 10754 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:53.656045 10848 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.663122 10839 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:53.663872  5871 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:53.669775  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:53.670699  6775 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:26:53.671767  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:53.671772  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:53.671922 10848 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:53.672405 10702 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:53.672665  6016 raft_consensus.cc:2368] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14.
I20260709 12:26:53.674345  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:53.658277 10783 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:53.649253  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:53.679729  6793 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:26:53.685261  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:53.697572  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:53.698509  6775 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:53.540396 10772 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 12:26:53.699008  5990 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:53.699021  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:53.715134  6793 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:26:53.715929  6778 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:53.721004  5840 leader_election.cc:400] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14.
I20260709 12:26:53.721668  5840 leader_election.cc:403] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:26:53.721726  6756 raft_consensus.cc:606] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:26:53.721001  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:53.740159  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:53.740947  6020 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:53.743582  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:53.744900 10847 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:53.745594 10847 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.745272  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:53.747649 10061 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.748421 10061 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:53.753536  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:53.687878 10702 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:53.683665 10783 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:53.715863 10824 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:53.760252  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:53.743708 10898 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.347s	user 0.004s	sys 0.000s
I20260709 12:26:53.794749  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:53.768659  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:53.784303  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:26:53.757438 10692 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.015s	user 0.010s	sys 0.004s
I20260709 12:26:53.805437  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:53.741856 10519 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:53.759424  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:53.739347 10505 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:53.746114 10847 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:53.797400  5871 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:53.799954  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:53.800961  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:53.801748  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.807297  5868 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:53.754809 10903 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:53.746820 10697 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14.
I20260709 12:26:53.828308  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:53.828603 10061 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:53.831046  5943 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:53.829896  5838 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:53.835425  6016 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:53.841342  5868 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:53.740975 10839 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 12:26:53.811651 10519 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.813560 10505 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.805229 10824 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:53.813509 10818 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.996s	user 0.030s	sys 0.012s
I20260709 12:26:53.849222 10687 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:53.793591 10702 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:53.835211 10821 consensus_meta.cc:287] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.642s	user 0.011s	sys 0.003s
I20260709 12:26:53.828437 10903 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.863638  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:53.800805 10896 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.868149 10824 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:53.854941 10519 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:53.856887 10818 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.039s	user 0.030s	sys 0.012s
W20260709 12:26:53.858250 10877 consensus_meta.cc:287] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.667s	user 0.014s	sys 0.000s
I20260709 12:26:53.813300 10692 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:53.882011  5865 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:53.883652 10692 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:53.855753 10505 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:53.877684  5947 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
W20260709 12:26:53.869639 10870 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 1.054s	user 0.004s	sys 0.000s
I20260709 12:26:53.866382 10697 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:53.892572  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:53.870836 10903 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:53.900216 10702 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:53.912125 10711 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:53.912794 10711 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:53.845906 10783 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:53.915180 10711 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:53.915524 10783 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:53.916986  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:53.890702 10821 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:53.918077 10821 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:53.919133 10821 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.892993 10877 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.923183 10877 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:53.924479  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:53.925222  6016 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:26:53.927604 10891 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:53.928567  5866 raft_consensus.cc:2368] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15.
I20260709 12:26:53.910986 10894 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:53.936124 10894 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:53.895805 10815 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.593s	user 0.013s	sys 0.000s
I20260709 12:26:53.937206 10815 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:53.939549 10815 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:53.941601 10687 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:53.942193  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:53.943965  5946 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:53.877844 10818 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:53.944630  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1646) took 1039 ms. Trace:
I20260709 12:26:53.945982  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:53.946836  6017 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 12: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:26:53.950682 10880 consensus_meta.cc:287] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.001s
I20260709 12:26:53.950968 10911 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:53.951969 10911 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:53.928083  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1304) took 1119 ms. Trace:
I20260709 12:26:53.953600  5941 rpcz_store.cc:276] 0709 12:26:52.808621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.831074 (+ 22453us) service_pool.cc:224] Handling call
0709 12:26:53.927860 (+1096786us) spinlock_profiling.cc:243] Waited 86.7 ms on lock 0x7b5400633154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:53.927990 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":199324544}
I20260709 12:26:53.939425  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:53.925343  6773 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:53.955902  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:53.956650  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1195) took 1760 ms. Trace:
I20260709 12:26:53.953894  5872 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:53.945075  5865 rpcz_store.cc:276] 0709 12:26:52.904970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.909099 (+  4129us) service_pool.cc:224] Handling call
0709 12:26:52.909487 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.882574 (+973087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.944400 (+ 61826us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b54004a0454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:53.944568 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2235856768}
I20260709 12:26:53.953728  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:53.960546  6022 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:53.927606  6766 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:26:53.963056  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:53.963280  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:53.963789  5943 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:53.965164  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:53.966171  5945 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:53.939291  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 994) took 1082 ms. Trace:
I20260709 12:26:53.972672  6016 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:26:53.942348 10687 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:53.977118  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:53.981885  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:53.981737  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:53.957082  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:53.945113 10818 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.970535  6778 rpcz_store.cc:276] 0709 12:26:52.857166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.874356 (+ 17190us) service_pool.cc:224] Handling call
0709 12:26:52.874818 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.721456 (+846638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.939048 (+217592us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b540021fa54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:53.939225 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1933837056}
I20260709 12:26:53.975641  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:53.967363  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:53.976135  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:53.991549  5948 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 11: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:53.976390  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:53.977170  6022 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:26:53.992698  5913 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:53.978519  5943 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:26:53.957192  6019 rpcz_store.cc:276] 0709 12:26:52.196393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.268153 (+ 71760us) service_pool.cc:224] Handling call
0709 12:26:52.271461 (+  3308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:52.271891 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.274400 (+  2509us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:52.308490 (+ 34090us) log.cc:844] Serialized 26 byte log entry
0709 12:26:52.689543 (+381053us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:52.689606 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.689728 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.868893 (+1179165us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:53.956291 (+ 87398us) raft_consensus.cc:1685] finished
0709 12:26:53.956318 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.956575 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":172,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":124,"spinlock_wait_cycles":23673600,"thread_start_us":258039,"threads_started":1,"wal-append.queue_time_us":403061}
I20260709 12:26:53.983314  5865 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 12: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:53.957237 10592 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:54.004328  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:53.990937 10882 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:54.008666 10702 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:54.009543 10702 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:53.987740  6766 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:54.010802 10702 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.986563  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:53.996615  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:54.005951 10891 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:54.017977  5872 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:53.998834  5913 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:54.027469  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:54.027849  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:54.030256  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:54.032720  6022 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:54.033123  6023 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:53.975085 10815 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.039690 10505 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:53.951267 10880 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:54.042876  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:54.051100  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:53.980502 10692 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:26:54.048609  6611 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:54.058784  6022 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:54.063342  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:54.020828 10711 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:26:53.991158 10687 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:54.041059 10882 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:54.046307 10697 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:54.044240 10914 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:54.070182  6024 raft_consensus.cc:2368] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12.
I20260709 12:26:54.094285  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:54.051487 10842 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:54.052510 10679 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:54.102375 10061 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:54.194684 10061 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:54.044153 10776 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:26:53.957165 10847 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:54.067979 10872 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:54.207800 10061 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:54.196807 10711 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:26:54.065074 10797 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.138s
I20260709 12:26:54.209616 10061 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:54.212613  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:54.048053 10815 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:54.084697 10592 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:54.081151 10914 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.255925  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:54.350164  6773 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:26:54.387953  6793 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:54.400830  5870 raft_consensus.cc:2368] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13.
I20260709 12:26:54.411466  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:54.226428 10687 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:54.211654 10880 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:54.413345  6793 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 11: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:54.337001 10909 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.194535 10851 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.198s
I20260709 12:26:54.339459 10592 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:54.389767 10921 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 12:26:54.198418 10842 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:54.427357  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:54.221107 10519 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:54.381786 10914 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:54.436039 10592 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:54.218531 10882 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:54.430539 10909 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:54.481360  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 379512us
W20260709 12:26:54.483719  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.511242 10519 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:54.496161 10880 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:54.514382 10880 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:26:54.519901  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.487743 10759 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:54.488389 10824 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:54.511288 10903 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.001s
I20260709 12:26:54.516320 10505 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:54.524447 10519 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.537873 10519 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:54.538823 10935 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:54.534112 10903 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:54.539479 10935 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:54.540337  6020 raft_consensus.cc:2368] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14.
I20260709 12:26:54.547271  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:54.543661  6017 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:54.530720 10759 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:54.592964 10759 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:54.603250  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:54.605262  5865 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 12:26:54.589301 10918 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.735s	user 0.008s	sys 0.017s
I20260709 12:26:54.606876  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:54.530756 10824 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:54.541370 10519 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:54.550630 10880 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:54.609714  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:54.567564 10772 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:54.541519 10935 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:54.559091 10851 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:54.559533 10911 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:54.541462 10903 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:54.564779 10847 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:54.617228  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:54.562736 10815 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:54.607244 10872 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:54.621781  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:54.633042  5872 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:26:54.636672  5871 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:54.616461 10783 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:54.642536  6021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.018s	sys 0.034s
W20260709 12:26:54.643075  6021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.018s	sys 0.034s
I20260709 12:26:54.645666 10674 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:54.654234  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1003) took 1006 ms. Trace:
I20260709 12:26:54.654778  6785 rpcz_store.cc:276] 0709 12:26:53.647833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.649734 (+  1901us) service_pool.cc:224] Handling call
0709 12:26:53.650137 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:53.650534 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:53.650566 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:53.651088 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.748341 (+ 97253us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:53.748389 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.748464 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:54.588521 (+840057us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:54.631888 (+ 43367us) raft_consensus.cc:1685] finished
0709 12:26:54.631914 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.654129 (+ 22215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4481,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":102,"spinlock_wait_cycles":48105344,"thread_start_us":96914,"threads_started":1,"wal-append.queue_time_us":150849}
I20260709 12:26:54.632086 10842 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:54.666648 10885 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 1.516s	user 0.003s	sys 0.002s
I20260709 12:26:54.626965 10851 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:54.623637 10772 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:54.630105 10911 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:54.642652 10872 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:54.690846  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:54.691311  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:54.675854 10687 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.654830 10783 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:54.675709 10909 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:54.712538  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:54.714092  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:54.705196 10934 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:54.706537 10872 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:54.708029 10824 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:54.708470  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:54.703677 10687 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:54.708102  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:54.723305 10938 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:54.726446 10910 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.878s	user 0.014s	sys 0.004s
I20260709 12:26:54.727074 10783 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:54.731864 10914 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:54.777721 10938 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:54.735543 10840 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:54.737057 10909 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:54.740352  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:54.728940 10592 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:54.741521  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:54.745041 10772 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:54.754056  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:54.739887  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:54.753211 10911 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:54.764333 10438 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.886320  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:54.776137 10674 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:54.815128  5866 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1233) took 1870 ms (client timeout 2078 ms). Trace:
W20260709 12:26:54.815475  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 110860us
I20260709 12:26:54.832582 10909 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:54.863195  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:54.887573  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 147846us
W20260709 12:26:54.863327  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 159014us
I20260709 12:26:54.882130  5940 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:54.890458  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:54.774425 10934 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:54.887550  5871 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:54.891975  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:54.895470 10757 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:54.897984 10438 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:54.899427 10783 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.900067 10783 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:54.900976 10438 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:54.901724 10851 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:54.902663 10840 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:54.903018 10592 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:54.873034 10872 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:54.904511 10872 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:54.907579 10872 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:54.908847 10847 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:54.911227  6757 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:54.911929  5871 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:54.914889 10914 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:54.915905 10882 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:54.887573  5866 rpcz_store.cc:269] 0709 12:26:52.944534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.944711 (+   177us) service_pool.cc:224] Handling call
0709 12:26:54.777376 (+1832665us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b54002f62d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:54.814985 (+ 37609us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2173390720}
I20260709 12:26:54.916880 10947 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.917728 10947 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:54.919160  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:54.920074  5872 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:26:54.921066 10947 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:54.922041  5871 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:54.924119  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:54.929513  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:54.938971  6023 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:54.935364  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:54.947259  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:54.947803 10687 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:54.932018 10783 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:54.951210 10909 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:54.953094  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:54.956797 10938 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:54.957830  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 69833.5us
I20260709 12:26:54.963754 10815 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:54.964730 10815 consensus_queue.cc:1243] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:26:54.965853 10772 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:26:54.967995  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:54.969887  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 77967.5us
I20260709 12:26:54.966976  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:54.974272  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 205377us
I20260709 12:26:54.976651  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:26:54.979449  6024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1221) took 1294 ms (client timeout 1649 ms). Trace:
W20260709 12:26:54.980137  6024 rpcz_store.cc:269] 0709 12:26:53.673259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.674040 (+   781us) service_pool.cc:224] Handling call
0709 12:26:54.966815 (+1292775us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b540019d554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:54.967058 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":907931904}
I20260709 12:26:54.973349  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:54.990824  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:54.982749  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:54.997560  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:54.998926  6785 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:26:55.001390  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:55.002373  6015 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:55.003315  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:54.966254 10772 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:55.006189  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:55.007076  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:55.017544  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:55.016916  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:55.017827  6766 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:55.014117  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:55.007442  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:55.022008  5987 leader_election.cc:400] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15.
I20260709 12:26:55.023891  5987 leader_election.cc:403] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:55.025807  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:55.010942  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:55.014117  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:55.026568  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:55.028540  6023 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:55.030860  5990 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:54.974635 10842 raft_consensus.cc:2764] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 12:26:55.033392  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:55.034391  6785 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:55.038662  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:55.039007  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:55.039621  6024 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:26:54.992632 10438 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:55.043835  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:55.045342  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:55.048668  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:55.049424  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:55.092743  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 61970.9us
W20260709 12:26:55.093376  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 67672us
I20260709 12:26:55.096724  6768 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:26:55.097159  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:55.098181  5871 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:55.099680  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:55.099511  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:55.102497  5948 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:26:55.104446  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:55.033011 10808 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15.
W20260709 12:26:55.094920  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 72309us
I20260709 12:26:55.113001  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:55.009900 10882 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:55.094473  5940 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:55.036749 10872 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:55.106638  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:55.119088  6015 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:26:55.122123  6017 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:55.005901 10914 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 12:26:55.105219 10903 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:55.126706 10934 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:55.102813 10776 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:55.127806 10776 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:55.010169 10759 consensus_queue.cc:1059] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 12:26:55.129981 10954 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:55.110285  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:55.130581  6785 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:55.113893  5871 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:55.122967 10687 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:55.132619 10891 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:55.124162 10872 raft_consensus.cc:697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:55.110050  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:55.134193  5872 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:26:55.133291 10891 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:55.134524 10872 consensus_queue.cc:237] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:55.136032 10891 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:55.027062 10911 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.078s
W20260709 12:26:55.111171 10929 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:26:55.032747 10937 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
W20260709 12:26:55.144436  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.103143 10519 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:55.167387  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:55.167920  6023 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:55.154987  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:55.095244  6016 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:55.178471  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:55.178861  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:55.179144  5868 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:55.185341  5872 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 14: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:55.174880 10519 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.187170 10519 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:55.191645  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:55.165995 10882 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:55.200852  6766 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:26:55.201663  6762 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:55.200280  5868 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:55.203980 10955 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:26:55.208724 10891 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:55.203979 10952 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:55.212128 10611 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:55.213335  6778 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.192s	user 0.012s	sys 0.030s
I20260709 12:26:55.118672 10772 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:55.214525 10772 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:55.174741  5865 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:55.204344 10882 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:55.219506  5870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 1658) took 1244 ms (client timeout 1528 ms). Trace:
W20260709 12:26:55.219955  5870 rpcz_store.cc:269] 0709 12:26:53.975079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.975385 (+   306us) service_pool.cc:224] Handling call
0709 12:26:55.219317 (+1243932us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b54007350d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:55.219444 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":942115200}
I20260709 12:26:55.220489 10882 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:55.202476 10953 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:55.207386 10776 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:55.212998 10611 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:55.175122 10839 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:55.222708  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1224) took 1486 ms. Trace:
I20260709 12:26:55.227397  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:55.233325  5870 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:26:55.232259  6020 rpcz_store.cc:276] 0709 12:26:53.736403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.744953 (+  8550us) service_pool.cc:224] Handling call
0709 12:26:55.222477 (+1477524us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b54006b8fd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:55.222642 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1821441408}
I20260709 12:26:55.241498  5870 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:55.246618  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:55.218147 10902 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:55.248757 10902 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:55.246253 10839 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:55.253007 10839 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:55.258592 10902 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:55.260954 10902 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.267189 10820 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:55.267915 10519 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:55.274103  6019 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:26:55.282519  5870 raft_consensus.cc:2410] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 12:26:55.282186  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 188151us
W20260709 12:26:55.275852  5840 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.196:34835 timed out after 1.649s (SENT) [suppressed 17 similar messages]
I20260709 12:26:55.283325  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:55.131814  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:55.295027  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:55.306387  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55542 (request call id 30) took 1682 ms. Trace:
I20260709 12:26:55.306957  6756 rpcz_store.cc:276] 0709 12:26:53.620941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.621132 (+   191us) service_pool.cc:224] Handling call
0709 12:26:54.697132 (+1076000us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b4c003c1bc0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d0f4fa20 00007f39d0f60a29 00007f39d0f26644 00007f39d0f4df2d 00007f39d0f4db93 00007f39d2c7dc92 00007f39d2c7d84a 00007f39d7c4b60a 00007f39d11d17d4 00007f39d11d1762 00007f39d11d1691
0709 12:26:55.303318 (+606186us) spinlock_profiling.cc:243] Waited 99.5 ms on lock 0x7b540021fa54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c7d853 00007f39d7c4b60a 00007f39d11d17d4 00007f39d11d1762 00007f39d11d1691 00007f39d11d160c 00007f39d11d08c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:55.303429 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260472320}
I20260709 12:26:55.309135 10839 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:55.309782 10839 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:55.311336  6015 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:55.315155  6778 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.294s	user 0.012s	sys 0.031s
I20260709 12:26:55.316133  6015 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:55.318104  6016 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:55.320528  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:55.329071  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:55.332041 10772 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.332835 10772 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:55.335665 10772 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:55.339530  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:55.348031 10772 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:55.350071  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:55.356595 10772 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:55.306576 10776 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:55.363436  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:55.366449 10772 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:55.367735 10519 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:55.370959 10721 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:55.374765  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 259563us
I20260709 12:26:55.365531 10776 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:26:55.383613  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 192090us
W20260709 12:26:55.376587  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 261235us
I20260709 12:26:55.385773  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:55.385092  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:55.389201  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:55.391836  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:55.393488  5990 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:55.374197 10868 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:55.390079  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:55.395066 10677 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:55.395722 10677 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.396149 10677 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:55.396492 10783 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:55.400406  6609 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:55.403095  5990 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:55.406327  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:55.407441  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:55.378033 10776 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:55.410482  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:55.411414  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:55.414296  6768 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:26:55.401430  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 125575us
I20260709 12:26:55.414254  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:55.416841  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:55.401932 10786 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:55.426797 10786 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:55.421108 10783 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:55.427271 10786 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:55.427505 10783 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:55.427728 10786 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:55.428553 10783 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:55.414969  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1289) took 1748 ms. Trace:
I20260709 12:26:55.377226 10721 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:26:55.420289 10952 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:26:55.430320  6778 rpcz_store.cc:276] 0709 12:26:53.666835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.990011 (+323176us) service_pool.cc:224] Handling call
0709 12:26:53.990261 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.990446 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:55.315858 (+1325412us) log.cc:844] Serialized 14 byte log entry
0709 12:26:55.414602 (+ 98744us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:55.414667 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:55.414723 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.414795 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.414911 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6201,"prepare.queue_time_us":505429,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":18183,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":1386269,"threads_started":2}
I20260709 12:26:55.432433 10952 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:55.438095 10937 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:55.439100 10937 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:55.392555 10815 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:26:55.441257 10937 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:55.425290 10808 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:55.444137 10808 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.444664 10872 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:55.444666 10808 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:55.410200 10903 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:55.451539  5943 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:55.441846 10963 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:55.445408 10872 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.443282  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:55.404520 10679 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:55.456089 10821 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:55.456545 10821 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:55.456933 10821 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:55.401398 10947 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:55.448052  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:55.459373  5942 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:55.449676  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:55.456573  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:55.453882  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:55.460852  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:55.461400  5943 raft_consensus.cc:2410] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:26:55.468997  5840 leader_election.cc:400] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14.
I20260709 12:26:55.470088  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:55.471032  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 47349.2us
I20260709 12:26:55.474918  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:55.472589  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:55.473013  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:55.473568 10842 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:26:55.478078  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 53833.1us
I20260709 12:26:55.479708  5865 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:26:55.452136 10962 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:55.453967 10903 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:55.451498 10519 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:55.496740 10519 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:55.499042 10519 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:55.499992 10903 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:55.483783  5868 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:55.484272 10847 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:55.484753 10882 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:55.481065 10969 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:55.470736 10963 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:55.512375 10963 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:55.514207  5941 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:55.490633 10839 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:55.515620 10947 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:55.516225 10947 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:55.517158 10969 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:55.517760 10969 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:55.460814 10872 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:55.519994 10969 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:55.487367  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:55.494194  6778 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:26:55.521045  6778 raft_consensus.cc:606] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:26:55.481861  5944 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:26:55.521759 10775 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.501008 10962 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:55.522197 10776 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:55.522737 10775 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:55.523371  5946 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:55.525738  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:55.533440  5942 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:55.537123  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:26:55.540558  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:55.541741  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:55.545802  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.522217 10702 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.492239 10953 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:26:55.524699 10847 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:55.551782 10820 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:55.529479 10947 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:55.534787 10961 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:55.493089 10677 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:55.541555 10882 raft_consensus.cc:697] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:55.559803  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:55.536775 10842 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 14 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:55.555563  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 104746us
I20260709 12:26:55.588209  5871 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:55.530467 10868 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:26:55.603819  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 119213us
I20260709 12:26:55.606359  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:55.607105  6785 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:26:55.611676  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:55.612357  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:55.613677  6022 raft_consensus.cc:2368] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12.
W20260709 12:26:55.618342 10975 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:26:55.618029 10894 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:55.597849 10868 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:55.561237 10962 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:55.573349 10775 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:55.570784 10776 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:55.633221  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:55.565582 10808 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:55.637995 10873 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:26:55.665707 10882 consensus_queue.cc:237] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:55.670478 10842 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:55.675215 10847 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:55.676033 10981 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:55.676988 10868 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:55.683791 10821 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:55.685107  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 52013.6us
I20260709 12:26:55.686385  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:55.687044 10980 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.587296 10820 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:55.641584  5840 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:55.677235 10867 raft_consensus.cc:3055] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:55.579725 10702 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:55.598405 10783 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:55.652164  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1004) took 2004 ms. Trace:
W20260709 12:26:55.696452  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 54976.5us
I20260709 12:26:55.721428  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:55.722361  5870 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:55.724112  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:55.699000  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 100242us
I20260709 12:26:55.725245  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:55.596763 10961 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:55.697250 10983 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:55.711127  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 124935us
I20260709 12:26:55.621265 10978 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 12:26:55.753321  5840 leader_election.cc:400] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12.
I20260709 12:26:55.725764  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:55.756047  6773 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:26:55.718604  6757 rpcz_store.cc:276] 0709 12:26:53.648171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.649744 (+  1573us) service_pool.cc:224] Handling call
0709 12:26:54.591232 (+941488us) spinlock_profiling.cc:243] Waited 7.05 ms on lock 0x7b580001f898. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:26:54.591507 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.911826 (+320319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.651832 (+740006us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b5400008f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:55.652063 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":750934656}
I20260709 12:26:55.771086  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:55.771392  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:55.703142 10938 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:55.772194  5942 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:55.772428  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:55.773078  5940 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:55.705144 10867 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:55.775317  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:55.778474  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.771679  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:55.785112  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1644) took 2872 ms. Trace:
I20260709 12:26:55.781073 10988 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:55.699935 10786 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:55.758198 10847 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:55.770933 10851 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:55.687958 10980 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:55.800101 10808 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:55.801788  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 892) took 1100 ms. Trace:
I20260709 12:26:55.888196  5949 rpcz_store.cc:276] 0709 12:26:54.701690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.701894 (+   204us) service_pool.cc:224] Handling call
0709 12:26:54.702239 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.723637 (+ 21398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.723684 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.723744 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:54.723791 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.801487 (+1077696us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b540055ffd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:55.801542 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.801715 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48689024}
I20260709 12:26:55.759918 10519 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:55.817437  6609 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:55.801952 10992 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:55.835826  5944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1621) took 2069 ms (client timeout 1817 ms). Trace:
W20260709 12:26:55.836598  5941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1628) took 1875 ms (client timeout 2199 ms). Trace:
I20260709 12:26:55.795763  5873 rpcz_store.cc:276] 0709 12:26:52.904281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.904504 (+   223us) service_pool.cc:224] Handling call
0709 12:26:52.904941 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.150604 (+245663us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:53.299331 (+148727us) log.cc:844] Serialized 14 byte log entry
0709 12:26:53.338155 (+ 38824us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:53.338218 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:53.338271 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.651820 (+313549us) spinlock_profiling.cc:243] Waited 2.75 ms on lock 0x7b180044ff80. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb642a6 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db
0709 12:26:53.653150 (+  1330us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:26:54.512421 (+859271us) spinlock_profiling.cc:243] Waited 9.79 ms on lock 0x7b180044ff80. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:26:55.775945 (+1263524us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400144bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:55.776006 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.776178 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69,"prepare.queue_time_us":221690,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":22326,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":8386,"spinlock_wait_cycles":598955392,"thread_start_us":186688,"threads_started":2,"wal-append.queue_time_us":38685,"wal-append.run_cpu_time_us":25502,"wal-append.run_wall_time_us":1524969}
I20260709 12:26:55.742291 10983 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:55.842427  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:55.754268 10783 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:55.845311  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:55.845836 10988 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:55.906782 10979 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:55.908573 10979 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:55.848029 10981 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:55.847786 10867 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:55.865056 10996 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.865607  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:55.866425  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 71453.5us
I20260709 12:26:55.921182  5945 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:55.922629  5840 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:55.865767 10998 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:55.879696  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:55.924111  5942 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:55.780186 10962 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.780582 10894 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:55.925875  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:55.890345  5947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1626) took 2090 ms (client timeout 1791 ms). Trace:
I20260709 12:26:55.823207  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:55.894958  5941 rpcz_store.cc:269] 0709 12:26:53.961700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.963902 (+  2202us) service_pool.cc:224] Handling call
0709 12:26:55.836429 (+1872527us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400308ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:55.836539 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3452178176}
W20260709 12:26:55.893970  5944 rpcz_store.cc:269] 0709 12:26:53.766990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.767161 (+   171us) service_pool.cc:224] Handling call
0709 12:26:55.835594 (+2068433us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b540081b854. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:55.835731 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3832832768}
I20260709 12:26:55.844640  5870 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:26:55.899086  5873 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:55.902748  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:55.899399 10983 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:55.909809  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 92514.1us
I20260709 12:26:55.914305  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:55.908942 10979 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:55.919253  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:55.854100 10851 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:55.928996 10872 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:55.926723 10981 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:55.919926 10949 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:55.883605 10847 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:55.893200 10750 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:55.911406 10821 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:55.988381 10993 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:56.050616 10993 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:56.051011 10993 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:55.992770  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:56.055154  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:56.056305  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:56.056860  5868 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
W20260709 12:26:55.995266  5947 rpcz_store.cc:269] 0709 12:26:53.799495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.799648 (+   153us) service_pool.cc:224] Handling call
0709 12:26:55.890078 (+2090430us) spinlock_profiling.cc:243] Waited 62.8 ms on lock 0x7b5400309dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:55.890224 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144471296}
I20260709 12:26:55.901214 10992 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:55.999850  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:56.059540 10969 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:56.060156 10969 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:56.060346 10992 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:56.060676 10969 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:56.061192 10990 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 12:26:56.010612 10611 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:56.010779 10702 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:56.014144 10988 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:56.012813 10983 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:56.015549  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:56.065780  6773 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
W20260709 12:26:56.066465 10977 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.514s	user 0.001s	sys 0.000s
I20260709 12:26:56.067045 10983 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:56.067257  5869 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:56.072811  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:56.018878  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 1146) took 1021 ms. Trace:
I20260709 12:26:56.032912  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:56.074936  6762 rpcz_store.cc:276] 0709 12:26:54.997283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.999104 (+  1821us) service_pool.cc:224] Handling call
0709 12:26:55.201465 (+202361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.203162 (+  1697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.996651 (+793489us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b5400003f50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:56.018755 (+ 22104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":385907968}
I20260709 12:26:56.077354  5868 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:26:56.077824  5870 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:56.078550  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:56.033537  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:56.031435  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:56.079908  5866 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
W20260709 12:26:56.065433 10903 consensus_meta.cc:287] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.002s
I20260709 12:26:56.012692  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:56.086242  6022 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 17: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:56.048060  6015 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 12:26:56.059582  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 63932.9us
W20260709 12:26:56.075639  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:56.087390  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 399437us
I20260709 12:26:56.087555  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:56.046254  5872 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 10.
I20260709 12:26:56.083814  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:56.091800  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:56.096112  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:56.096112  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:56.096377  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:56.098377  6778 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:56.096926  6773 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:26:56.100844  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:56.101492  5947 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:26:56.091145  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:56.049216 10847 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:56.016395 10980 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:56.106220  6773 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:26:56.106640  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:56.109354  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:56.110386  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:56.128329  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:56.129487  5940 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:26:56.110622  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:26:56.040220 10982 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.332s	user 0.001s	sys 0.000s
I20260709 12:26:56.113372  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:56.111989  5913 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:56.115332  5868 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:26:56.022783 10978 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.085214 10757 raft_consensus.cc:3055] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:56.042979 10851 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:56.077729 10838 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.160360  6019 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:56.121868  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:56.034443 10820 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:56.043625 10867 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:56.047797 10880 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:56.137516  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:56.168794  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:56.036680 10783 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:56.119817 10879 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:56.147696  6756 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:56.137387  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:56.046777 10981 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:56.044730 10872 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:56.149881  6773 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:56.150835  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:56.151124  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:56.186113  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:56.043457 10677 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:56.156414  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 68169.7us
I20260709 12:26:56.163385  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:56.165270  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:56.189194  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:56.106658 10983 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:56.153051 10967 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:56.155441 10775 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:56.169068  5868 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:26:56.192899  6778 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:26:56.188895  5913 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:55.925910 10962 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:56.118381  6785 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:56.112617 10818 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:56.047690 10750 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:56.168859  5948 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:56.120157 10961 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:56.173503  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:56.174878  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:56.180980  6023 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:56.120932 10786 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:56.184203  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:56.168613 10880 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:56.112985 10840 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:56.148661 10757 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:56.205797  5913 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:56.206610  6021 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:56.237918  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:56.207396  5873 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:56.208320  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:56.208516  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:56.211421  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:56.196496  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:56.219198  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:56.238941  6017 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:26:56.239540  6018 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:56.220531 10824 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:56.221904  5870 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:56.148190 10674 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.003777 10996 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:56.228917  6773 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:56.161031 10978 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:56.042387 10979 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:56.236181  6019 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:26:56.207348  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:56.239871 10840 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.245438  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:56.246117 10624 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:56.246732 10961 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:56.164284 10868 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:26:56.247787 10757 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:56.249238 10969 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:56.250114 10962 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:56.250699 10880 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:56.254562 10750 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:56.249207 10775 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:26:56.254552  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:56.302904  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 146636us
I20260709 12:26:56.170368 10838 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:56.259109  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:56.090018 10998 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:56.242517  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:56.332964  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 50247.9us
I20260709 12:26:56.332991  6757 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:56.333434  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 91056.9us
I20260709 12:26:56.333914  5913 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:56.043687 10909 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.112s
I20260709 12:26:56.246776 10943 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.122s
I20260709 12:26:56.286363 10840 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:56.230870 10871 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:56.232667  5990 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:56.280699  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:56.342654  5940 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:26:56.289170  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:56.291323  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:56.283865  5868 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:56.260003 10824 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.296247  5867 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:56.266435 10994 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:56.300737  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:56.300516  6019 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:56.301606  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 155729us
I20260709 12:26:56.332154 11002 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.357668 10868 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:56.362213 11002 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:56.367259 10996 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:56.342543 10984 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:56.368306 10984 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:56.368817 10984 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:56.376286  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.235929 10519 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:26:56.295639 10776 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:56.339571 10911 consensus_queue.cc:1059] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 12:26:56.100719 10903 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:56.275605 10823 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:26:56.347146  5869 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:26:56.332510 10884 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:56.277928 10674 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:56.337651  6768 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:56.348242  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:56.353296  5865 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 14 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:56.340333 10880 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:56.359710 10994 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:26:56.340615  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.377617 10871 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:26:56.414213  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 59700.1us
I20260709 12:26:56.415169  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:56.416283  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:56.417908  5913 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:26:56.382318  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 149732us
I20260709 12:26:56.388516  5867 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:26:56.399013  5943 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:56.424476  6024 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
W20260709 12:26:56.401160  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 97222.5us
I20260709 12:26:56.349215 10824 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:56.346760 10750 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:56.365355 11002 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:56.341389 10962 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
W20260709 12:26:56.411289  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 63097.3us
I20260709 12:26:56.408728 10757 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:56.341796 10943 consensus_queue.cc:1243] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:56.431131  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:56.448415  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:56.423022 11000 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:26:56.407948 10884 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:56.454808  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:56.455701  5945 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:56.418465 10808 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:56.423601 10961 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:56.421948 10674 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:56.420895 10883 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:56.418630 10994 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:56.441560 10823 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:56.422312 10978 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:56.444422 10838 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:56.483385  5943 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
W20260709 12:26:56.567891  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 140694us
I20260709 12:26:56.420470 11005 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 12:26:56.482966 10808 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:56.453696 10750 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:56.484637 10883 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:56.567008 10776 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.489614 10823 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:56.646884  6020 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:26:56.661032  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 215366us
I20260709 12:26:56.650746 10911 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.675911  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:56.654259 10823 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:56.650539 10776 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:56.642901 10775 consensus_queue.cc:1059] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
W20260709 12:26:56.689056  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 117801us
W20260709 12:26:56.655234 11000 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:26:56.651746 10883 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:56.652604 10969 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:56.692711  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:56.685403  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:56.705816  6768 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:56.706298  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:56.581773 11010 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 12:26:56.703126  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:56.583678 10884 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:56.690979 10978 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:56.732712 11003 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.321s	user 0.001s	sys 0.001s
I20260709 12:26:56.753829 10721 raft_consensus.cc:3055] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:56.688323 10911 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:56.759485 10943 raft_consensus.cc:3055] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:56.766675 11006 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:56.726928 10998 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:56.644945 10871 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:56.758013 10984 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:56.778245  6785 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:26:56.738426 10961 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:56.741417 10949 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:56.737305 10903 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:56.775921 10852 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.032s	sys 0.033s
I20260709 12:26:56.746264 11002 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:56.779507 11006 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:56.774513 10993 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.723s	user 0.016s	sys 0.001s
I20260709 12:26:56.780954 10884 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:56.774610 10721 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:56.796738 10852 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.032s	sys 0.034s
I20260709 12:26:56.780014 10943 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:56.795334 10949 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:56.776937 10679 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:26:56.733255 10808 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:56.867431 10949 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:56.891552 10840 consensus_meta.cc:287] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.004s
I20260709 12:26:56.892177 10840 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:56.895465 10840 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:56.878084 11006 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:56.848740 10993 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:56.898290 11006 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:56.877490 10961 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:56.906610 10943 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:56.910748 10994 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:56.912609 10851 consensus_meta.cc:287] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.742s	user 0.002s	sys 0.009s
I20260709 12:26:56.913419 10994 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:56.932323 10867 consensus_meta.cc:287] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.759s	user 0.011s	sys 0.002s
I20260709 12:26:56.901340 10993 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:56.935923 10911 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:56.920765 10880 consensus_meta.cc:287] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.502s	user 0.013s	sys 0.000s
I20260709 12:26:56.999117 10984 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:57.006927 10949 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:57.080054  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:57.097977  5914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.005s	sys 0.009s
W20260709 12:26:57.098991  5914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.006s	sys 0.009s
I20260709 12:26:57.102100  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:57.103606  5986 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.010s	sys 0.006s
W20260709 12:26:57.104133  5986 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.010s	sys 0.006s
W20260709 12:26:57.098099  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.006s	sys 0.003s
W20260709 12:26:57.105513  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.007s	sys 0.003s
W20260709 12:26:57.106237  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 659127us
I20260709 12:26:57.116816  5944 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:57.102756  5946 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:26:57.127869 10674 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.131096 10671 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:57.130748 10674 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:57.131827 10671 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:57.151261  5946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1638) took 1238 ms (client timeout 1603 ms). Trace:
W20260709 12:26:57.160398  5946 rpcz_store.cc:269] 0709 12:26:55.913158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.913902 (+   744us) service_pool.cc:224] Handling call
0709 12:26:57.150767 (+1236865us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54003084d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:57.151045 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2544831232}
I20260709 12:26:57.136529 10702 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:57.164233 10702 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:57.132141 10519 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:57.129303  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.184576 10871 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.151049  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1005) took 1014 ms. Trace:
I20260709 12:26:57.185758 10871 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:57.186200  5869 rpcz_store.cc:276] 0709 12:26:56.136838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.137041 (+   203us) service_pool.cc:224] Handling call
0709 12:26:57.150462 (+1013421us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b5400193054. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:57.150755 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389845760}
I20260709 12:26:57.143582  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 1670) took 1224 ms. Trace:
I20260709 12:26:57.133798  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1671) took 1209 ms. Trace:
I20260709 12:26:57.188387  5867 rpcz_store.cc:276] 0709 12:26:55.918835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.918991 (+   156us) service_pool.cc:224] Handling call
0709 12:26:57.143065 (+1224074us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b5400210a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:57.143286 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582882688}
I20260709 12:26:57.188563  5871 rpcz_store.cc:276] 0709 12:26:55.923896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.924706 (+   810us) service_pool.cc:224] Handling call
0709 12:26:55.925034 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.343544 (+418510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.343577 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:56.343619 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:56.343650 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:57.132784 (+789134us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b5400186d54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:57.132862 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:57.133316 (+   454us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":961838336}
I20260709 12:26:57.159373  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:57.156153 10674 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:57.137276 10747 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:57.168467 10702 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:57.171739 10786 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.950889 10851 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:57.142386  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.183584 10872 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.135211 10721 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:57.141785  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:57.184847 10519 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:57.193969 10671 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:57.217558 10949 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:57.218475 10824 consensus_meta.cc:287] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.766s	user 0.016s	sys 0.000s
I20260709 12:26:57.219303 10824 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:57.224635 10851 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:57.226344  6762 raft_consensus.cc:2393] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:57.227392 10872 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:57.229948  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.131130 10679 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.105s
I20260709 12:26:57.231582 10747 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:57.242249 10994 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.243441 10994 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:57.247575 11014 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:57.248833 11014 consensus_queue.cc:1243] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:26:57.254339 10984 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.255395 10984 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.256212 11018 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:57.257253  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.247907  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.242347 10786 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:57.299657 10786 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:57.302722 10979 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:57.304240 10994 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:57.305536  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 70628.8us
I20260709 12:26:57.305773 10824 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:57.277107 10872 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:57.310976 10941 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:57.311720 10941 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:57.314049 10984 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:57.323449 11007 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:57.324625 11007 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:57.278918 10949 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:57.327466  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.280048 10519 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:57.281298 10721 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:57.286432  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.286265 10961 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:57.287210 11025 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.259338 10747 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:57.262918 10871 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:57.291706  5990 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:57.293141  5944 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 12:26:57.278271 10624 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.985s	user 0.008s	sys 0.021s
I20260709 12:26:57.327028 10949 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.338510  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.339569 10783 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:57.340587 10949 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:57.280649 10953 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.341078 10961 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:57.337983 11025 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:57.342895  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.336599  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.340660 10783 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:57.313588 10823 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.615s	user 0.009s	sys 0.013s
W20260709 12:26:57.344805  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 112150us
I20260709 12:26:57.313541 10941 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:57.361984 10823 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:57.363109 10941 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:57.363615 11025 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:57.364545 10871 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:57.365811 10624 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:57.369055 10879 raft_consensus.cc:993] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: : Instructing follower 6add331326ca47038308a447bf798fa7 to start an election
I20260709 12:26:57.369868  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:57.371865 10873 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:57.372493 10873 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:57.372619  5915 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:57.375607 11007 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:57.376598 11007 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:57.377885 10702 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:57.378533 10702 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:57.379004 10611 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.379050 10702 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:57.367949  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1002) took 1498 ms. Trace:
I20260709 12:26:57.380043  5948 raft_consensus.cc:2410] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:26:57.388654 10611 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.392172  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.393031 10611 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:57.393493  5944 raft_consensus.cc:2393] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 15: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:57.393980 10851 raft_consensus.cc:1081] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Signalling peer 6add331326ca47038308a447bf798fa7 to start an election
I20260709 12:26:57.395494  6778 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:57.412995  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.414417 10981 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:57.379499  5913 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:57.381997  5874 rpcz_store.cc:276] 0709 12:26:55.869468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.880297 (+ 10829us) service_pool.cc:224] Handling call
0709 12:26:55.880649 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:55.881051 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:55.881079 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:55.885173 (+  4094us) log.cc:844] Serialized 26 byte log entry
0709 12:26:56.090150 (+204977us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:56.149181 (+ 59031us) log.cc:844] Serialized 14 byte log entry
0709 12:26:57.366350 (+1217169us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b18003d37e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb10c0a 00007f39d9250219 00007f39cfb98a40 00007f39cfb934e4 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287 00007f39d2c9e208
0709 12:26:57.366929 (+   579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:57.367048 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:57.367298 (+   250us) raft_consensus.cc:1685] finished
0709 12:26:57.367415 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:57.367749 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7482,"prepare.queue_time_us":47570,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":1230343,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":253,"spinlock_wait_cycles":6157568,"thread_start_us":179963,"threads_started":1,"wal-append.queue_time_us":376863}
I20260709 12:26:57.414949 10981 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:57.397890 10884 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.543s	user 0.014s	sys 0.008s
I20260709 12:26:57.401603 10842 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:57.416262 10981 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.402221 10838 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:57.382038 10871 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:57.257481 11018 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:57.417613 10838 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.420169 10871 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.364533 10823 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:57.413312 10783 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:57.416422 10842 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:57.402740 10847 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:57.385615 10824 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:57.388716  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:57.377444 10818 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:57.421465 10838 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:57.425485  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.429219  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.430246  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:57.445997  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 73584.2us
I20260709 12:26:57.453763  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1023) took 1324 ms. Trace:
I20260709 12:26:57.425450  5943 pending_rounds.cc:85] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 9
I20260709 12:26:57.422199 11025 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:57.434237  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.424618  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.461902  5946 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:26:57.464830 11025 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:57.465605 10943 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:57.447563 10847 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:57.465108  6756 raft_consensus.cc:2410] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 12:26:57.468374 10847 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:57.435425  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.470438  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:57.471740  5940 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:57.472389  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:57.470706  6766 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:26:57.472864  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:57.474419 11025 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:57.476398  5947 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:26:57.356917 10671 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.458688  6762 rpcz_store.cc:276] 0709 12:26:56.086513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.087978 (+  1465us) service_pool.cc:224] Handling call
0709 12:26:57.394802 (+1306824us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400007b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:57.410570 (+ 15768us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2446680320}
I20260709 12:26:57.459316  5943 pending_rounds.cc:107] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:26:57.435853  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.486245  5944 raft_consensus.cc:2410] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:26:57.447327  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.487558  5943 raft_consensus.cc:2889] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:57.421599 11018 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:57.422643 10998 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:57.426949 11016 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:57.498943  5941 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:57.425891 10519 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:57.464200 10842 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:57.508349  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:26:57.464929 11020 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.570s	user 0.007s	sys 0.017s
I20260709 12:26:57.422971 10996 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:57.507969  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1663) took 1639 ms. Trace:
I20260709 12:26:57.513345  6775 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:57.515862  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:26:57.516311  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 60385.3us
I20260709 12:26:57.517477  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.519372 10996 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.519412  5873 pending_rounds.cc:85] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 5
W20260709 12:26:57.522013 10776 consensus_meta.cc:287] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.823s	user 0.013s	sys 0.009s
I20260709 12:26:57.522295  5873 pending_rounds.cc:107] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:26:57.522743  5873 raft_consensus.cc:2889] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:57.522722 10776 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:57.517560  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:57.527504  6609 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:57.528834 10961 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:57.529057 10438 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:57.529767 10438 raft_consensus.cc:697] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:57.530296 10996 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:57.531068 10438 consensus_queue.cc:237] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:57.426614 10994 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:57.534080 10783 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:57.534011 10954 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:57.497985 10981 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:57.536966  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.537464  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.438318 10953 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.514756  5865 rpcz_store.cc:276] 0709 12:26:55.863261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.863455 (+   194us) service_pool.cc:224] Handling call
0709 12:26:55.863879 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.408904 (+545025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:57.478779 (+1069875us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b540069e954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:26:57.502952 (+ 24173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1116863360}
I20260709 12:26:57.507483 10818 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.534768 10783 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:57.546914 10519 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:57.548041 10757 consensus_meta.cc:287] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.089s	user 0.014s	sys 0.014s
I20260709 12:26:57.551429 11016 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:57.553721 10747 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.554286 10747 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.554438 10998 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.556625 10998 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:57.534155 10994 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:57.438225 10873 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:57.506578 10824 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:57.560907 10519 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:57.563668 10818 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:57.576300  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:57.577585 10818 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:57.582123 10953 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:57.575367  6609 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:57.582517  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:57.583451  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 1.37663e+06us
I20260709 12:26:57.585804  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:57.590186 11016 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:57.560621 11011 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:26:57.591643  6021 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e"
dest_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:34888
I20260709 12:26:57.592660  6021 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:57.593101  6021 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:57.561869 10671 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:57.573575  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:57.601265 10747 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:57.566831  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:57.568571 11018 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:57.570231  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:57.570663 10824 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:57.571708 10883 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.786s	user 0.022s	sys 0.004s
W20260709 12:26:57.614768  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 48038.1us
I20260709 12:26:57.558139 10941 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:26:57.559245  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:57.565910 10994 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:57.607887 10998 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:57.608704  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.612072  5942 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:26:57.618278  5990 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:57.558566 10783 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:57.616856 10671 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:57.651957 10953 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:57.652863 10953 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:57.655181 10953 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:57.655846 10894 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:57.656525 10894 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.658804 10894 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:57.660738 10592 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:57.661957 11018 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:57.645401 10883 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:57.668283 10883 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:57.670545 10750 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:57.672593 10998 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.674952 10998 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:57.677822 10671 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.678447 10671 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:57.674974 10775 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:57.684376 10775 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:26:57.684437  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 141321us
I20260709 12:26:57.685869 10721 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:57.685969  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:57.690526  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 63700.6us
I20260709 12:26:57.693030  5940 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
W20260709 12:26:57.697824  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 183928us
I20260709 12:26:57.671891 11035 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:57.692216 10721 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:57.649410 10994 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:57.665441 11018 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:57.685513 10775 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:57.717821 10775 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:57.717844 11016 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:57.722579 11016 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:57.723551  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:57.724493  6762 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:57.725076 11016 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:57.725888  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.726089  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.726830  6768 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:57.727196  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.727690 10998 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:57.727866  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 46785.6us
I20260709 12:26:57.728415 10998 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:57.728787  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.727169  5940 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:26:57.734757  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:57.738376  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.740527  6611 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:57.743778 10838 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.744951  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:57.709525 10994 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:57.744473 10838 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:57.751490  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:57.751233 11035 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.751715  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:57.752451  5871 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:57.752589  6020 raft_consensus.cc:2410] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
W20260709 12:26:57.753986  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 68131.2us
I20260709 12:26:57.759497  5869 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:57.761241  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.762302  5944 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:57.762439  5946 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:57.763446  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:57.764101  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.761278  5947 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:57.762221  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:57.765910  5868 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:26:57.766014  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.767643 10883 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.768427 10883 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:57.764549  6022 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:57.771047 10883 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:57.771230  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:57.771747 10883 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:57.772014  5865 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:57.772379 10883 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.775221 10776 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:57.779485 10818 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:57.783020  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.785246 11035 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:57.752635 10838 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:57.786089 11035 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:57.786262  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.786633  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:57.787020 11035 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.789382 10998 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:57.792479  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 52129.8us
I20260709 12:26:57.793577 10954 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:57.793959  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:57.794651  6022 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:26:57.796396  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:57.779008  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:57.797344  6020 raft_consensus.cc:2410] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:57.798166 10873 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:57.800127 10883 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:57.801267  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.789156  6778 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:26:57.807473  6778 pending_rounds.cc:85] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 9
I20260709 12:26:57.807842  6778 pending_rounds.cc:107] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:26:57.808068  6778 raft_consensus.cc:2889] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:57.808374  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:57.808883  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:57.810791 10980 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:57.811277 10980 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.811611 10980 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:57.812565 10702 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:57.813488 10824 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:57.814806  5874 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:57.810600  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:57.821193  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:57.821971  5870 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:57.823170  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:57.822111  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:57.824322  5874 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:57.792495 11035 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:26:57.827597 11031 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.262s	user 0.003s	sys 0.003s
I20260709 12:26:57.828212  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:57.829121  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:57.830065  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:57.831056  5873 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:57.831404  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:57.832621  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:57.834707  6017 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:57.835229  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:57.837509  5990 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:57.838025  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:57.816403 10994 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:57.848032 10823 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:57.848891 10823 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:57.838514  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:57.839663  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:57.810719  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:57.839658  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:57.749092  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:57.842437  6762 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
W20260709 12:26:57.844942  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:57.854411  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.852182  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:57.855734  6020 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:57.850095  6021 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.857309  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:57.861577  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:57.865507  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:57.865916  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.866573  5872 raft_consensus.cc:2410] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:26:57.867069  6766 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:57.866356 10880 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:57.867861 10880 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:57.868294 10880 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:57.862006  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:57.869371  5867 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:57.870425  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.871596  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:57.877230  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:57.878047  5874 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:26:57.872900  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:57.879606  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 141037us
I20260709 12:26:57.879297  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.880836  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:57.886751 10519 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:57.893841  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:57.894704  5872 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:57.896364 10702 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:57.861976 11016 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:57.897367 11016 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.897850 11016 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:57.898417  5870 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:26:57.899675  6017 raft_consensus.cc:2393] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 17: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:57.901405  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:57.902047  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:57.902803  5866 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 13: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:57.903671  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:57.904428  6785 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:57.905292  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:57.906497  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.906831  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.907346  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:57.907877 10851 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:57.908357 10851 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:57.908806  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.909075  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:57.909780 10851 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.908238  5866 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:57.910290  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:57.911545 10851 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:57.911942  6015 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:57.894729 10997 consensus_queue.cc:1059] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:26:57.906652  5874 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:57.912595  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.911938  5870 raft_consensus.cc:2410] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 12:26:57.910220  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:57.919183  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.908420  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.922827  6766 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:26:57.925060  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:57.907354  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.928339  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.923734  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:57.929569  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.931089  6021 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:57.931442  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 50655us
I20260709 12:26:57.932600  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:57.894220 10952 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:57.910645  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:57.936913  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:57.941227  6775 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 12: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:57.946099  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:26:57.937615  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 43738.1us
I20260709 12:26:57.953927  6793 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:26:57.960330 10967 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:57.964664  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.966408  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:57.967206  6756 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:57.967505  6756 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:57.968387  6756 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:57.968979  5915 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:57.969780 10750 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:57.974840 10750 raft_consensus.cc:697] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:57.975847 10750 consensus_queue.cc:237] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:57.971341 11025 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:57.894220 10519 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:57.969784 10779 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:57.986900  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:57.975648 10808 raft_consensus.cc:3055] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:57.969463  6756 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:57.986627  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:57.982409  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 49927.9us
I20260709 12:26:57.988866  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:57.988216 10871 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:57.981503  6762 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:26:57.992936  6768 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:57.994778  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:57.987844  5872 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:26:57.990770  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:58.007277  5940 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:58.007887  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:57.998191  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:57.961179 10702 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.013725 10818 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:57.978426 10952 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:57.985911 10519 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:58.030400  6775 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:58.048102  5871 raft_consensus.cc:2410] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:26:57.988957 10808 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:57.996423 11025 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:57.995842 11034 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.102s
I20260709 12:26:58.009640 10943 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:58.049024 10980 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:58.037781 10824 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:58.054562 10952 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:58.055812  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 112080us
W20260709 12:26:58.056504 11043 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:26:58.005543 10775 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:58.055831 10883 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:58.054888  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:58.027632 10997 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:58.066883  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:57.995918 11014 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:26:58.065706  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:58.061836  5865 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:58.029798 10702 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:58.095217  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:58.096431  5866 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:26:58.095057  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:58.097102  5869 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:58.097314  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:58.062880 10818 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:58.100841  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:58.103099  6022 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 16: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:58.106539  5873 raft_consensus.cc:2393] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 13: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:58.109781  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:58.110922  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:58.111876  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:58.112607  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:58.122495  5865 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:58.112263 10996 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:58.113143 11028 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:58.118822  5948 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:58.084190 11035 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:58.125313 10808 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:58.126883 10980 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:58.127712 10823 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:58.132375  5871 raft_consensus.cc:2410] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:58.133700  5868 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:26:58.094662 10775 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:58.152602  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:58.103449 10981 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:26:58.156227  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:58.105247 10997 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:58.171873  5865 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:26:58.140110 10998 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:58.215910  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:58.261499  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 44503.3us
I20260709 12:26:58.149184 11035 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:26:58.271363  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.278146  5838 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:58.278607  5865 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:58.279848  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 129799us
I20260709 12:26:58.153347 11028 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:58.284621 11028 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:58.238721 11025 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:58.288021 11035 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:58.251791 10838 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.290294 10838 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:58.216235 10998 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:58.295327 10998 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:58.295495 10997 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:58.297181  5866 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:58.231371 10967 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:58.298071 10967 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:58.317795  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.310771  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:58.324923  6768 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:58.210722 10671 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:58.293414 10838 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:58.314200 10808 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:58.289125 11035 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 12:26:58.348790 10994 consensus_meta.cc:287] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.503s	user 0.013s	sys 0.000s
W20260709 12:26:58.400497  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.417865 10994 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:58.420231  6020 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:58.354642 10818 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:26:58.423226 10994 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:58.358210 10702 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:58.438616  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:58.444463 10988 consensus_queue.cc:1059] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 12:26:58.450461 10988 raft_consensus.cc:3055] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:58.450974 10988 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:58.452095 10988 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:58.464030 10519 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:58.466820 10988 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:58.478338 10671 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:58.480465 10818 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:58.481738 10775 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:58.482383  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 43882.9us
I20260709 12:26:58.488843  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:58.436611 10997 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:58.499750  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:58.425741 11016 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:26:58.516563  5874 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.070s	sys 0.129s
W20260709 12:26:58.519459  5874 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.070s	sys 0.130s
W20260709 12:26:58.509944 10880 consensus_meta.cc:287] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.641s	user 0.016s	sys 0.000s
I20260709 12:26:58.503189 10702 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:58.501271 10981 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.496982 10980 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:26:58.576735  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 59476.4us
W20260709 12:26:58.524569 10783 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.827s	user 0.009s	sys 0.005s
W20260709 12:26:58.525810 10873 consensus_meta.cc:287] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.727s	user 0.016s	sys 0.000s
I20260709 12:26:58.516558 10671 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:58.619426  6768 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 12:26:58.638854  6756 consensus_meta.cc:287] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.649s	user 0.009s	sys 0.006s
I20260709 12:26:58.660318  6756 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:26:58.630378 10783 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:58.619135 11034 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:26:58.662091  6778 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:26:58.663587 10783 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:26:58.642037  6023 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.005s	sys 0.034s
I20260709 12:26:58.664145 10980 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:58.668885  6020 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:58.549860 10702 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:58.671753 10702 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:58.545608 10880 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:58.586814 10981 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:58.675794 10981 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:58.523608 10997 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:58.677726  6024 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
W20260709 12:26:58.680536 11048 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:58.682447 10687 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.647318 10943 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:58.647751  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.684161 10519 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:58.684247 10943 raft_consensus.cc:697] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:58.647905 10882 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:26:58.649186 10851 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.737s	user 0.012s	sys 0.005s
I20260709 12:26:58.521226 10967 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:58.650049 10671 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:58.652740 10838 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:58.638520 10824 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:58.618661 10994 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:58.683257 10687 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:58.687876 10702 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:58.685778 10943 consensus_queue.cc:237] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:58.694013 10838 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:58.700728  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 419287us
I20260709 12:26:58.695299 10776 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:58.680397  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1187) took 1112 ms. Trace:
I20260709 12:26:58.688714 10882 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:26:58.719590  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.707326  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:58.693706 10824 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:58.714905  6024 rpcz_store.cc:276] 0709 12:26:57.567888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.572732 (+  4844us) service_pool.cc:224] Handling call
0709 12:26:58.680160 (+1107428us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400160b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:26:58.680306 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2387480832}
W20260709 12:26:58.721617  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 362474us
I20260709 12:26:58.744745  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:26:58.737118  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 410647us
W20260709 12:26:58.749379  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.755110  6018 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 12: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:58.751623  5990 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:26:58.761973  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.764143  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:58.765002  6015 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 17: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:58.769140  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:58.770752  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.743420 10824 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:58.706521 10838 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:58.779446 10838 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:58.782128  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:58.783891  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:58.787603 10824 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:58.788339 10967 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:58.743641 10981 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:58.720320 10776 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:58.792006  6017 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 14: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:58.792742 10687 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:58.760289 10851 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.795506 10997 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:58.798024 10808 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:26:58.797925  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 51041.2us
I20260709 12:26:58.767442  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:58.803463  6021 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:58.820659  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.804114  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:26:58.818087  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 329330us
W20260709 12:26:58.809468 11018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.004s	sys 0.019s
I20260709 12:26:58.706595 10994 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:58.827416  5913 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:58.835671  6611 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:58.827613  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:58.830085  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:58.839864  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:58.842501  6773 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:58.810802 10981 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.846242  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:58.833523 11018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.005s	sys 0.019s
I20260709 12:26:58.700124 10671 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:58.858632  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:58.848644  5941 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:58.862425 10824 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:58.863005  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:58.863690  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 585695us
W20260709 12:26:58.877803  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.879477  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:58.872642  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:58.847098 10808 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:58.866189 10981 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:58.898898  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 63312.7us
I20260709 12:26:58.905529  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:58.907802  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:58.910604  6017 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:26:58.910992  6766 raft_consensus.cc:2393] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 17: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:58.852139 10851 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:58.914714  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 185064us
I20260709 12:26:58.917199  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:58.921268  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:58.922295  5941 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:58.924739  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:58.927178  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:58.934933  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:58.951027  6021 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:58.891516 10824 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:58.954259  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:58.897185 11025 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:58.964485  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 118310us
I20260709 12:26:58.898932 10519 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:58.962527  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:58.968528  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 109994us
I20260709 12:26:58.959600  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:58.972569  6768 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:26:58.966423  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:58.834679 10882 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:58.914093 10949 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:58.911993 11035 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:26:59.004423  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:59.016166 11075 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:58.997267 10775 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:59.022238 10775 raft_consensus.cc:697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:59.023442 10775 consensus_queue.cc:237] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:59.024408 11061 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.024833  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:59.027846 11075 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:58.980625 10941 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.029131  6021 raft_consensus.cc:2410] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:26:59.052115  6015 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:58.968458 10824 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:59.041594 11075 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
W20260709 12:26:59.061175 11060 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 12:26:59.061105 11075 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:59.062431 10824 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:59.060911 11035 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:59.063022 10994 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:59.064015 10776 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:59.064472  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:59.066417 10994 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:59.040793 10941 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:59.006256 11065 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:58.979424 11025 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:59.070644  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:59.063158 10824 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:59.073303 10882 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:59.082783  5941 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:26:59.088068  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:59.090279  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:59.073943 11062 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:26:59.095342  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 268034us
I20260709 12:26:59.102495  5913 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:59.084986 10994 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:59.109707  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1014) took 1195 ms. Trace:
I20260709 12:26:59.112025  5874 rpcz_store.cc:276] 0709 12:26:57.913905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.914030 (+   125us) service_pool.cc:224] Handling call
0709 12:26:57.925537 (+ 11507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:57.926025 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:58.522124 (+596099us) log.cc:844] Serialized 14 byte log entry
0709 12:26:59.096152 (+574028us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:59.096236 (+    84us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:59.108654 (+ 12418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.108941 (+   287us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:59.109470 (+   529us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":735839,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":30183,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":33915,"thread_start_us":1164997,"threads_started":3,"wal-append.queue_time_us":395311}
I20260709 12:26:59.036180 10808 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:59.087826 10824 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:59.089484 10880 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:59.097863 11058 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:59.134873  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:59.108141 10949 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:59.135735  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 107833us
I20260709 12:26:59.137900  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:59.136617  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 127879us
I20260709 12:26:59.112138 10996 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:59.125746  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:59.143764  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:59.117386 11087 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:59.124308 10818 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:59.156131 10880 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:59.158720 10949 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:59.159987 11083 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.160404  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1283) took 1164 ms. Trace:
I20260709 12:26:59.159090  5941 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0006f949032c4034b4f4872ee332386e"
dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:48330
W20260709 12:26:59.163254  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.161504 11084 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.165858  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:26:59.167079  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:59.170833  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:59.172732  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:59.173557  5942 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:59.038678 11056 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.135s
I20260709 12:26:59.170570 11088 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:59.154243 11065 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.160897  6016 rpcz_store.cc:276] 0709 12:26:57.903708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.904708 (+  1000us) service_pool.cc:224] Handling call
0709 12:26:57.905043 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:57.982280 (+ 77237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:57.982333 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:58.021556 (+ 39223us) log.cc:844] Serialized 26 byte log entry
0709 12:26:58.555101 (+533545us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:58.555146 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.555234 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:59.067735 (+512501us) raft_consensus.cc:1685] finished
0709 12:26:59.067763 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:59.067945 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32370,"raft.queue_time_us":68975,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"thread_start_us":453363,"threads_started":1,"wal-append.queue_time_us":663981}
I20260709 12:26:59.183207 11086 raft_consensus.cc:2764] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:26:59.184300 11086 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:59.184954 11086 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.185321 11086 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:59.041445 11074 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.136s
I20260709 12:26:59.186478 10943 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:59.187006 10943 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:59.188568  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1282) took 1285 ms. Trace:
W20260709 12:26:59.168504 10998 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.872s	user 0.017s	sys 0.000s
W20260709 12:26:59.188664  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.135895 10677 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.175s
I20260709 12:26:59.197417 10943 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:59.195606  5940 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:26:59.197948 10824 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:59.193873  5838 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:59.201480  6785 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:59.203994  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:59.175904 10941 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:59.206122  5942 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:59.204562  6773 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:26:59.211210  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:59.212137  5943 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:59.215649 11025 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:59.217905 11089 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:59.219218  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.221110  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:59.221864  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:59.222471  5866 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:26:59.222846  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:59.207816  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:59.107105 11062 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:59.196569 10851 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:59.189078  6023 rpcz_store.cc:276] 0709 12:26:57.903233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.905051 (+  1818us) service_pool.cc:224] Handling call
0709 12:26:57.905479 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:58.664732 (+759253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:58.664764 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:58.665351 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 12:26:58.905201 (+239850us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:58.905263 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.905389 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:59.188315 (+282926us) raft_consensus.cc:1685] finished
0709 12:26:59.188337 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:59.188500 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90272,"raft.queue_time_us":122723,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":17257,"thread_start_us":885943,"threads_started":2,"wal-append.queue_time_us":491126}
I20260709 12:26:59.205570 10980 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:59.247781  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:59.248070  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:59.160635 11083 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:59.253890  5871 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:59.247272  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:59.252960  5945 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
W20260709 12:26:59.256354  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 93394.6us
I20260709 12:26:59.253933  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:59.242985 11087 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:59.265137  5838 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:59.178735 11084 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:59.286461  5943 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:26:59.202880 10998 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:59.262904  5941 raft_consensus.cc:606] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:59.304271  6023 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:26:59.327977  5941 raft_consensus.cc:614] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:59.056087 11061 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:59.283124 11016 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:59.292237  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.315835  5871 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:26:59.317552  5942 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:26:59.323617  6024 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:26:59.333854  6024 pending_rounds.cc:85] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 5
I20260709 12:26:59.333901  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:59.334403  6024 pending_rounds.cc:107] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:26:59.335170  6024 raft_consensus.cc:2889] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:59.336608  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:59.337579  5869 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
W20260709 12:26:59.345326  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.195720 11065 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:59.347045 11095 raft_consensus.cc:2764] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:26:59.348260  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 83206.8us
I20260709 12:26:59.351655  5945 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:59.351804  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:59.352474  5942 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 12.
I20260709 12:26:59.304260 10943 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:59.366226  5867 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:59.366891 10838 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.587s	user 0.009s	sys 0.007s
W20260709 12:26:59.371815  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:59.373144  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 191352us
I20260709 12:26:59.375586  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:59.379853  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:26:59.399231  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 48606.7us
I20260709 12:26:59.357239 10998 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:59.376472 11063 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.213s	user 0.003s	sys 0.003s
I20260709 12:26:59.416332  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:59.416345 11025 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:26:59.416918  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 251148us
I20260709 12:26:59.419847  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:59.425364  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:59.407411  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:59.404909 10980 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:59.432168  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:59.432598  5838 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:59.405184 11028 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:59.439568  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:59.440564  5870 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:59.417557 10981 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.007s
I20260709 12:26:59.433763  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:59.441555  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:59.404232 11016 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:26:59.428351 10941 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:59.451218  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:59.452159  5869 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:26:59.430200 10818 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:59.429234 11074 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:59.449040  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:59.457221  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:59.459170  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:59.460397  5871 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 13: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:59.432202 10851 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:26:59.465220  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.456784  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:59.469748  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:59.467165  5866 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:59.470705  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:59.472445 11074 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:59.472643  6020 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:26:59.473064  5838 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:59.474675 11087 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:59.475976 11035 consensus_queue.cc:1059] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:59.446951 11066 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:59.477263 11066 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.478227  6024 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:59.475800  5838 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:59.443034 10750 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:26:59.453778 10808 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:59.445905 10981 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:59.461781 10998 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:59.461686 10818 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.486305  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:59.487850  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:59.475912 11087 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:26:59.489727  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:59.483530 11088 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:59.477705 11066 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:59.475800 11062 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:59.477553 11035 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:59.495604  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:59.478569 11090 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:59.478670 10783 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:59.498714  6020 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 12: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:59.485603 10996 raft_consensus.cc:3055] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:59.485737 11016 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.501897  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:59.506739  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 84146.2us
I20260709 12:26:59.497184 10838 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:59.509804 11089 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:59.510385 11089 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.510820 11089 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:59.513691 10988 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:59.486784 10851 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:26:59.494661 10702 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.795s	user 0.016s	sys 0.000s
I20260709 12:26:59.493706 10818 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:59.521497  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 45785.4us
I20260709 12:26:59.495373 11088 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:59.497603 11065 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:59.526795 10955 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:59.519260 11062 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:26:59.473733 11074 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:59.529991 11096 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:26:59.536767  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:26:59.534633  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:26:59.538393  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 111818us
I20260709 12:26:59.514992 10996 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:59.540866  5838 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:59.542851  6020 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:59.523039 10988 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:59.516882 10783 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:59.548141  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:26:59.517845 11016 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:59.541808 10955 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:59.517355 11090 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:59.555219  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:59.556241 10808 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:59.560885  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:59.562959 10808 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:59.556241 11101 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:59.572255  5838 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:59.575842  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:26:59.548852 11065 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:26:59.563225 10671 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:59.583786  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:26:59.585839  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:59.586586  5942 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:26:59.572016 10808 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:59.572899 11101 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:59.573324 11056 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:59.594245  5869 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:26:59.560161 11086 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:59.588374 11087 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:59.597798 11101 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:59.599946 11102 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.069s	user 0.001s	sys 0.001s
I20260709 12:26:59.595064 11106 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:59.613523  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:59.614859  5838 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:59.615339  5942 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:26:59.615950 11025 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:26:59.617901 11082 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.494s	user 0.004s	sys 0.000s
I20260709 12:26:59.623812  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:26:59.560690 10851 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:59.624616  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.622270  5874 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:59.598407 11056 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.625999 11056 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:59.626032 11081 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.502s	user 0.004s	sys 0.000s
I20260709 12:26:59.627782 11084 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:59.623648 11025 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:59.565745 10996 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:59.634860  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:59.561026 11088 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:59.642103  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:59.644289  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.645315  5838 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:59.654342  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:59.654598  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:59.640139 11062 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:59.658455  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:59.658569  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:59.659540  6793 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:26:59.659863 11088 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:59.660130 10949 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.651585 10750 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:59.660737 10949 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:59.661435 10750 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:59.662724  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:59.663594 10750 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:59.663728 11058 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:59.664032 10880 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:59.664518 10880 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:26:59.665126 10880 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:59.665241 11069 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:59.665784 11069 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:26:59.663148 10949 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:59.666745 11069 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:59.668148 11069 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:59.664481  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:59.622681 11106 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:26:59.632316 11084 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.700448  5870 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:26:59.689323  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:59.704663  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:59.707238  6775 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:26:59.707885  6775 pending_rounds.cc:85] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 6
I20260709 12:26:59.708166  6775 pending_rounds.cc:107] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:26:59.708444  6775 raft_consensus.cc:2889] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:59.710502  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:26:59.714591  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:59.706216 10882 consensus_meta.cc:287] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.005s
I20260709 12:26:59.668814 11073 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:26:59.664220 10750 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:59.719172  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:59.724663  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:59.720402  5865 raft_consensus.cc:2410] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:26:59.706125 11095 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:59.709779 11058 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:26:59.729388  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 84215.9us
I20260709 12:26:59.711982 11057 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:59.711108 11087 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:59.715828 11028 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:59.747399  5869 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:26:59.714439 10998 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:59.716374 10996 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:59.760226  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 125724us
I20260709 12:26:59.762527  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:26:59.762945  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.712180 11084 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:26:59.763821  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.747449 11057 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.730765 10988 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:59.753521 10818 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:59.733553 10750 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:59.748824 10671 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:59.752447 10783 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:26:59.762542 11098 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
W20260709 12:26:59.769655  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 59249.9us
I20260709 12:26:59.774576  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:26:59.776849  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.757977 11016 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:26:59.780957 11107 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:26:59.769512 11090 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:59.782212 10941 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:59.753314  5691 catalog_manager.cc:5697] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 9 to 13, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 13 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:59.787189 11084 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:59.788703 11089 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:59.789825 10671 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:59.791108 11089 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:59.790926 10671 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:59.761121 11106 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:59.792783 11106 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:59.799508 10750 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:59.729416 11073 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:59.812752  5990 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:59.774447  5840 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:26:59.818533  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 44150.4us
I20260709 12:26:59.819769  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:59.766669 11057 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:59.825282  6020 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
W20260709 12:26:59.826717  6611 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.2.4.254:41049, user_credentials={real_user=slave}} blocked reactor thread for 47577.9us
I20260709 12:26:59.830688  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:59.831475  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:59.832182  6019 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 17: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:59.832598  6023 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:26:59.830310 11087 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:59.842443  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:59.843178  5990 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:26:59.830367  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:59.847512  5990 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:59.850971  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:26:59.852046  6785 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:26:59.855150  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:26:59.848387 10824 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:59.808138 11113 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:59.858777  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:59.780068 11028 raft_consensus.cc:697] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:59.835099 10980 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:59.835963 11089 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:59.871290  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:59.863282 11062 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:59.874305  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:26:59.860229 11086 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:59.883314  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:59.841977 10880 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:59.885159 11035 raft_consensus.cc:3055] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:59.886493 11028 consensus_queue.cc:237] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:59.848651 11101 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:59.888459 11025 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:26:59.888962  5871 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:59.868942 11110 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:59.914496  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 67052.6us
W20260709 12:26:59.904683  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 93535.6us
I20260709 12:26:59.908622  6019 raft_consensus.cc:2393] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 14: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:26:59.909408  5869 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 11.
I20260709 12:26:59.894101 11058 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:59.884622 10818 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:26:59.908766  6611 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:26:59.901594 10687 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:59.923899  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:59.900104 11089 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:26:59.924592  6785 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:59.925444  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:26:59.931713 10880 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:59.912782 11035 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:26:59.913053 11084 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:59.950670 11115 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:59.952632  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:26:59.955703 11083 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:26:59.958073 11058 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:59.942762 11056 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:26:59.968093  6785 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:26:59.968927 11056 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:59.911849 11066 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:59.947477 10687 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:26:59.970535 10687 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:26:59.971526 11066 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:59.972478  6778 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 13: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:26:59.944917 11089 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:59.974789  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:26:59.978492  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:59.979770  5870 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:59.983897  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:59.984863  5870 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 11.
I20260709 12:26:59.988353  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:26:59.989010  5871 raft_consensus.cc:2368] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14.
I20260709 12:26:59.993003  5868 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:59.975940 10677 raft_consensus.cc:2764] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:26:59.957424 11084 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:26:59.986018 11086 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:26:59.989900 10996 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:00.017601 11086 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:26:59.999236 11056 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:00.026644 10996 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:00.027032 11086 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:26:59.986971 11035 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:00.014571 10808 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:00.042332 11035 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:00.043998 10808 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:00.046106 11084 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:00.048328 11057 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:00.050508  5867 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:27:00.051257  5867 pending_rounds.cc:85] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 6
I20260709 12:27:00.051677  5867 pending_rounds.cc:107] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:27:00.052028  5867 raft_consensus.cc:2889] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:00.056684 11057 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:00.059448 11101 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:00.060534  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:00.063665  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:00.100711  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:00.101580  6775 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:00.064631  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:00.064915  6766 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:00.066758 10981 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:00.067158  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:00.066754  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:00.105226  6017 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:27:00.105911  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:00.108633 10677 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:00.108982  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.109324 10677 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:00.111011 11073 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:00.111637 10677 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:00.074226  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:00.114673  6018 raft_consensus.cc:2368] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14.
I20260709 12:27:00.074774  5941 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:00.074994  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:00.075253  5942 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:27:00.116608  5942 pending_rounds.cc:85] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 8
I20260709 12:27:00.116461  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:00.116904  5942 pending_rounds.cc:107] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:27:00.117166  5942 raft_consensus.cc:2889] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:00.117234  5948 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:27:00.075971  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:00.072494  5691 catalog_manager.cc:5697] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 9 to 11, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 11 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:00.087620  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:00.119232  5913 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:00.088774  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:00.089797  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:00.120278 10818 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:00.091058  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:00.120815 10818 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.121295 10818 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.120347  5945 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:27:00.028163 10996 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:00.094287 10776 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.122887 10776 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:00.123194  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:00.123925  5945 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:00.125250 10776 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:00.069366  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:00.126837  6785 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:00.112831 11083 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:00.122027  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:00.095073 10824 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:00.125803  5943 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:00.125824  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:00.149119  6778 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:00.111833 11061 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:00.128576  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:00.157200  5940 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:27:00.129613  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:00.130322  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:00.158197  5947 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.158430  5874 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:27:00.126013  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:00.162281  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 217357us
I20260709 12:27:00.162091  6785 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:00.165701  5990 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:00.168606 11056 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:00.174099 11056 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:00.177802  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:00.179248  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:00.179768  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 53843.3us
I20260709 12:27:00.180202  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:00.180891  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 147675us
I20260709 12:27:00.181408  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:00.182168  6773 raft_consensus.cc:606] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:27:00.182787  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:00.183391  5869 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:00.180054  5873 raft_consensus.cc:2393] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 13: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:00.185601  5870 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:00.186964  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:00.179248  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:00.190081  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:00.191087  5865 raft_consensus.cc:2368] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 17. Current term is 18.
I20260709 12:27:00.193559  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:00.194446  6020 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:00.196673  5948 raft_consensus.cc:2368] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16.
I20260709 12:27:00.188174  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:00.198832  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:00.200246  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:00.200974  5870 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 11.
I20260709 12:27:00.201936  5940 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:00.202078  5913 leader_election.cc:400] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14.
I20260709 12:27:00.202565  5913 leader_election.cc:403] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:00.204644  5871 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:00.190718  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:00.206168  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.208009  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:00.211179  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:00.215108  5913 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:00.216737  5913 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:00.217020 10941 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:27:00.217528  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.217748 11073 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14.
I20260709 12:27:00.218752  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:00.220118 11124 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:27:00.221079  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:00.221789  5874 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.189373 11089 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:00.185367 10824 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:00.225317 11089 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:00.231683  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.226487  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:00.232940  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:00.238029  5874 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:27:00.238166  5942 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:00.226137  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:00.246351  5865 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:00.248521  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:00.249661 11073 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:00.250283 11073 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:00.250756 11073 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:00.251047  5946 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: [13.9-15.10]
I20260709 12:27:00.251839  5868 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:00.255172  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:00.168414 10776 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
W20260709 12:27:00.261001  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 72986us
I20260709 12:27:00.262382 11016 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:00.262614  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:00.240769 11065 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:00.208701 11025 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:27:00.264595 11057 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:00.267036 11057 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.169332 11083 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:00.267506 11057 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:00.269954 10943 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:00.270955 11120 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:00.271718 11058 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:00.272274 11120 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:00.272773 11120 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:00.273530  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:00.241758  6611 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:00.280445  5943 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:00.280915 10677 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:00.281597 10677 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.281961 10955 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:00.282582 11120 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:00.282061 10677 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:00.286640  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:00.287416  5941 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:00.288228  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:00.289105  6773 raft_consensus.cc:606] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:00.289516  6773 raft_consensus.cc:614] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:00.228422  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:00.239319  5870 raft_consensus.cc:2393] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 14: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:00.231190  5940 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 12. Current term is 13. Ops: []
W20260709 12:27:00.291625  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 126056us
I20260709 12:27:00.285313  6762 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:00.287453  5865 raft_consensus.cc:2376] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17. Re-sending same reply.
I20260709 12:27:00.295650  5869 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:00.297710 10824 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:00.297063 10988 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.305606 10988 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:00.308388 10988 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:00.309233  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:00.299346  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:00.312728  6020 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
W20260709 12:27:00.313241  5692 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50588 (request call id 57) took 15862 ms (client timeout 15000 ms). Trace:
I20260709 12:27:00.302316 10671 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:00.314299  5691 catalog_manager.cc:5697] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:27:00.303751  5871 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:00.293879 11083 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:00.318140 11061 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.318403  6793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:00.318907 11061 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:00.299675  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:00.322402 10981 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:00.322894 11061 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:00.323379 10981 consensus_queue.cc:1243] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 18.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:00.316461  5691 catalog_manager.cc:5697] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 4 to 8, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 8 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:27:00.324083 11061 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:00.324551 11061 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:00.325670 11061 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:00.327185 11061 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:00.298149  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 56847.7us
W20260709 12:27:00.329677  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 96856.7us
I20260709 12:27:00.330502  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:00.330749  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:00.339951  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:00.343063  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.346189  5838 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:00.352124  5874 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:27:00.355842  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:00.357204  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:00.333884 11120 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:00.363058 10941 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:00.365283 11116 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:27:00.365715  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 112112us
W20260709 12:27:00.366302  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 225476us
I20260709 12:27:00.354269 11075 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:00.373394  5915 leader_election.cc:400] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14.
I20260709 12:27:00.335458 11083 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:00.374261  6773 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:00.354688 11089 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:00.375376  6778 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:00.381670  5947 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
W20260709 12:27:00.374671  6609 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.767s (SENT) [suppressed 14 similar messages]
I20260709 12:27:00.384007  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:00.493779  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:00.492587  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:00.500119  5949 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:00.492774  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:00.496959  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:00.367450 11025 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:00.360817 11106 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:00.503526  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 128872us
I20260709 12:27:00.349252 11035 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 12:27:00.504990  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 121085us
W20260709 12:27:00.497740  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.374215 10996 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:00.525738  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:00.526700  6017 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 12. Current term is 13. Ops: []
W20260709 12:27:00.528050  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.547067  6024 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:27:00.528215  6023 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:00.533413  5945 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:00.537509  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:00.503818 11084 raft_consensus.cc:2764] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:27:00.541553  6018 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:00.544390  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:00.391420 11116 raft_consensus.cc:697] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:27:00.568171  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.535490  5874 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:00.555625  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:00.556667  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:00.532325 11065 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:00.570911  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.360406  5691 catalog_manager.cc:5697] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 6 to 10, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 10 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:00.580412  5691 catalog_manager.cc:5697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 8 to 13, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 13 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:27:00.586403  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:00.582505  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.512873 10823 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 12:27:00.589931  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:00.313804  5692 rpcz_store.cc:269] 0709 12:26:44.450473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.452835 (+  2362us) service_pool.cc:224] Handling call
0709 12:27:00.312988 (+15860153us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:54.213220 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:54.214002 (+   782us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:54.214054 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:26:54.214079 (+    25us) tablet.cc:662] Decoding operations
0709 12:26:54.217122 (+  3043us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:54.217235 (+   113us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:54.217289 (+    54us) tablet.cc:685] Acquiring locks for 31 operations
0709 12:26:54.670187 (+452898us) tablet.cc:701] Row locks acquired
0709 12:26:54.670214 (+    27us) write_op.cc:260] PREPARE: finished
0709 12:26:54.670406 (+   192us) write_op.cc:270] Start()
0709 12:26:54.670562 (+   156us) write_op.cc:276] Timestamp: P: 1783600014670352 usec, L: 0
0709 12:26:54.670587 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:26:54.734236 (+ 63649us) log.cc:844] Serialized 11987 byte log entry
0709 12:26:55.144373 (+410137us) op_driver.cc:464] REPLICATION: finished
0709 12:26:55.481946 (+337573us) write_op.cc:301] APPLY: starting
0709 12:26:55.492066 (+ 10120us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:56.040674 (+548608us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:56.040781 (+   107us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:56.936475 (+895694us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:56.937052 (+   577us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=62,key_file_lookups=62,delta_file_lookups=0,mrs_lookups=0
0709 12:26:56.937129 (+    77us) write_op.cc:312] APPLY: finished
0709 12:26:56.946563 (+  9434us) log.cc:844] Serialized 267 byte log entry
0709 12:26:56.947296 (+   733us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:56.947695 (+   399us) write_op.cc:454] Released schema lock
0709 12:26:56.947792 (+    97us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:56.948631 (+   839us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:56.948674 (+    43us) write_op.cc:454] Released schema lock
0709 12:26:57.431512 (+482838us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b1800404ec0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb96522 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
Metrics: {"mutex_wait_us":336964,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":310215,"apply.run_cpu_time_us":55183,"apply.run_wall_time_us":1478429,"cfile_cache_hit":126,"cfile_cache_hit_bytes":133831,"num_ops":31,"prepare.queue_time_us":327416,"prepare.run_cpu_time_us":34071,"prepare.run_wall_time_us":656930,"raft.queue_time_us":198899,"raft.run_cpu_time_us":29508,"raft.run_wall_time_us":248053,"replication_time_us":473617,"spinlock_wait_cycles":43948416,"thread_start_us":668670,"threads_started":4,"wal-append.queue_time_us":684613,"wal-append.run_cpu_time_us":16103,"wal-append.run_wall_time_us":2260180}]]}
I20260709 12:27:00.510718 11025 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.503485 11138 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.223s
W20260709 12:27:00.598855  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 252753us
I20260709 12:27:00.599565  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:00.642457  5838 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:00.642657  6778 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:00.583180 10941 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:00.648897  6024 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:00.643891  5838 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:00.613874 11106 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:00.597687 10818 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:00.678186  5944 raft_consensus.cc:2368] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 12. Current term is 13.
I20260709 12:27:00.679149 10818 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:00.641398 11025 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:00.673343  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:00.691361  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.647970 11089 raft_consensus.cc:2764] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:27:00.670763 10823 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:00.674484 10996 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:00.692167  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:00.738113  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 94295.1us
W20260709 12:27:00.740108  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 237061us
I20260709 12:27:00.741367  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:00.748243  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:00.707484 11137 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:27:00.750722  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:00.732914  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 59700.3us
I20260709 12:27:00.753131 10783 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.761289 10783 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:00.764030 10783 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:00.770706  5870 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:27:00.771373  5870 pending_rounds.cc:85] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 9
I20260709 12:27:00.771641  5870 pending_rounds.cc:107] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:27:00.771914  5870 raft_consensus.cc:2889] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:00.772360  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 230308us
I20260709 12:27:00.774412  5868 raft_consensus.cc:3055] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:00.774889  5868 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:00.775718  5868 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:00.776849  5868 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.766937 11116 consensus_queue.cc:237] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:00.779223  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:00.780637  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:00.781344 10954 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:00.767292 11089 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:00.782294 10954 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:00.708089 10823 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:00.693369 10988 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 12:27:00.791098  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:00.791443  5838 leader_election.cc:400] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16.
I20260709 12:27:00.792024  5838 leader_election.cc:403] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 12:27:00.798827  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 106760us
I20260709 12:27:00.803351  5870 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 12:27:00.810240  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 63858.8us
I20260709 12:27:00.811137  5838 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:27:00.790931 11073 consensus_meta.cc:287] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.007s
I20260709 12:27:00.851042  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:00.859236  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:00.860795  6609 leader_election.cc:400] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 17. Current term is 18.
I20260709 12:27:00.861052  6609 leader_election.cc:403] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:27:00.861989 11090 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:00.862736 10955 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:00.863529 11090 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 17. Current term is 18.
I20260709 12:27:00.788309 10687 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:00.801259 11065 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:00.800668 10808 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.818552 10702 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:00.877324  5868 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:00.883278  5872 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:00.792284 11016 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 12:27:00.847234 10783 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:27:00.881841 11057 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.614s	user 0.017s	sys 0.007s
W20260709 12:27:00.894508  5946 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.029s	sys 0.043s
I20260709 12:27:00.870401  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:00.924916  5869 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:27:00.926587  5946 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.032s	sys 0.043s
I20260709 12:27:00.939608  5869 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:27:00.893175  5838 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:00.815130 11035 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16.
I20260709 12:27:00.912406  5941 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:00.916503  5872 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:00.960769  5945 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:00.918246 11056 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:27:00.874629  5691 catalog_manager.cc:5697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 6 to 9, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 9 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:27:00.870826 10943 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:27:00.873409 11139 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 12:27:00.906895 10783 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:00.972514  5913 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:00.949776 11057 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:00.999019 11057 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:01.000730 11120 consensus_meta.cc:287] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.625s	user 0.017s	sys 0.004s
I20260709 12:27:01.002094  6020 raft_consensus.cc:2410] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
W20260709 12:27:01.004303 10677 consensus_meta.cc:287] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.718s	user 0.013s	sys 0.000s
I20260709 12:27:01.010296  6024 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:01.014009  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:01.016343 11090 raft_consensus.cc:993] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: : Instructing follower 6add331326ca47038308a447bf798fa7 to start an election
I20260709 12:27:01.016227  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:01.016975 11090 raft_consensus.cc:1081] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Signalling peer 6add331326ca47038308a447bf798fa7 to start an election
I20260709 12:27:01.018271  6017 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:27:01.019845  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:01.020654  6019 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.868456 11073 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:01.036258  6017 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc"
dest_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:34888
I20260709 12:27:01.037289  6017 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:01.037819  6017 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.902655 10808 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:01.004994 10677 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:01.052810  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 80416.6us
I20260709 12:27:00.978217 10981 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:00.951010 11123 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.125s
W20260709 12:27:01.066080  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 205432us
W20260709 12:27:01.068517  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 214042us
I20260709 12:27:00.899751 10687 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:01.084231  6023 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:01.045655 11035 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:01.093771  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:01.097074  6793 raft_consensus.cc:606] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:01.107647  6793 raft_consensus.cc:614] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:00.986876 11061 consensus_queue.cc:1059] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
W20260709 12:27:01.104261  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 211173us
I20260709 12:27:01.062942 11084 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:01.067433 10981 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.091641  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:01.112147  5944 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:27:01.117256  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:01.125180  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:00.931454 10823 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:01.126747  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:01.093851 11057 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.134667  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:01.144472  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.069010 10702 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:27:01.150978  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:00.967059  5691 catalog_manager.cc:5697] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 3 to 8, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 8 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:27:01.122778 10981 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:01.160259  5944 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:01.097021 11114 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 12:27:01.174333  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 71573us
I20260709 12:27:01.131047 11084 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:01.184638  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 45137.5us
I20260709 12:27:01.166685 10671 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:01.120019 11131 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:27:01.185922  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:01.120579 11035 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:01.146687 10967 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:01.215348  6017 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:01.167084 11139 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:27:01.175261 11090 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:01.154492 10955 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:01.182314  5949 consensus_meta.cc:287] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.678s	user 0.014s	sys 0.008s
I20260709 12:27:01.267318  5949 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:01.148001 11057 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:01.112147 10783 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:01.240900  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:01.183986 11156 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:27:01.260862  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 75078.8us
I20260709 12:27:01.268354  5943 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:01.274456  6017 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:01.219636 11159 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.288458  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:01.245306 10671 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:01.289355  6016 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:01.305148  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:01.343758  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 192879us
I20260709 12:27:01.345099  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:01.270419 11120 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:01.287065 11138 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.294476 11151 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.266454 11090 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:01.367545  6019 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:27:01.352614 11116 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:01.274953 10955 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:01.354607 10783 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:01.374245 11090 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:01.359193 11114 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 12:27:01.345160 11159 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:01.381915 10808 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:01.467080 11057 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:01.461817 11073 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:01.518800 11161 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 12:27:01.523512 10955 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:01.524277 11159 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:01.382848 11151 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:01.528326 11151 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:01.537135 11140 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.670s	user 0.004s	sys 0.004s
W20260709 12:27:01.509794  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 163129us
I20260709 12:27:01.467366 10677 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:01.540401  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:01.385648 11138 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:01.548277  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1667) took 1368 ms. Trace:
I20260709 12:27:01.550055  5946 rpcz_store.cc:276] 0709 12:27:00.171011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.252293 (+ 81282us) service_pool.cc:224] Handling call
0709 12:27:00.252748 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:00.253243 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:27:00.933798 (+680555us) log.cc:844] Serialized 14 byte log entry
0709 12:27:01.100183 (+166385us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:01.100836 (+   653us) log.cc:844] Serialized 26 byte log entry
0709 12:27:01.101033 (+   197us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:01.101915 (+   882us) log.cc:844] Serialized 14 byte log entry
0709 12:27:01.107711 (+  5796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.107968 (+   257us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:01.538829 (+430861us) raft_consensus.cc:1685] finished
0709 12:27:01.538857 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:01.539025 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10145,"prepare.queue_time_us":1368,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":83,"raft.queue_time_us":22631,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":16307,"spinlock_wait_cycles":20631168,"thread_start_us":806311,"threads_started":2,"wal-append.queue_time_us":179372}
I20260709 12:27:01.466113 11157 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.138s
I20260709 12:27:01.508183 10954 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:01.360316 11084 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:01.569334  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:01.574618  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:01.574841 11172 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:01.575501 11172 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:01.575908 11172 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:01.578001 10677 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:27:01.578318 11163 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 12:27:01.578734 10677 raft_consensus.cc:697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:01.579919 10677 consensus_queue.cc:237] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:01.580801  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55542 (request call id 34) took 1263 ms. Trace:
I20260709 12:27:01.582661 11151 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:01.538692 10671 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:01.557833 11069 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:01.586182 11069 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:01.586653 11069 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:01.370445  5691 catalog_manager.cc:5697] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 4 to 10, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 10 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
W20260709 12:27:01.592480 11165 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.335s	user 0.001s	sys 0.000s
W20260709 12:27:01.595645 11145 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.601s	user 0.007s	sys 0.007s
W20260709 12:27:01.588907 11160 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:27:01.581290  6793 rpcz_store.cc:276] 0709 12:27:00.317452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.318230 (+   778us) service_pool.cc:224] Handling call
0709 12:27:01.580530 (+1262300us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b5400007654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c7d853 00007f39d7c4b60a 00007f39d11d17d4 00007f39d11d1762 00007f39d11d1691 00007f39d11d160c 00007f39d11d08c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:01.580691 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1788411776}
I20260709 12:27:01.607563 11172 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:01.617945 11090 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:01.620939 10981 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:01.578437 11035 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:01.633844 11090 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:01.633998 10981 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:01.625954  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 1202) took 1070 ms. Trace:
I20260709 12:27:01.635094  6768 rpcz_store.cc:276] 0709 12:27:00.555066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.557602 (+  2536us) service_pool.cc:224] Handling call
0709 12:27:00.558090 (+   488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:00.558750 (+   660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:00.558797 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:00.559477 (+   680us) log.cc:844] Serialized 26 byte log entry
0709 12:27:00.885719 (+326242us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:00.885768 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.885843 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:01.595016 (+709173us) spinlock_profiling.cc:243] Waited 58.2 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:01.598089 (+  3073us) raft_consensus.cc:1685] finished
0709 12:27:01.598123 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:01.625785 (+ 27662us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":871,"raft.queue_time_us":1592,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":6211,"spinlock_wait_cycles":133839104,"thread_start_us":325853,"threads_started":1,"wal-append.queue_time_us":434256}
I20260709 12:27:01.567740 10954 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:01.637552 10954 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:01.636298 11172 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:01.631054 11138 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:01.653200  6768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:01.655670  6768 raft_consensus.cc:606] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:27:01.656147  6768 raft_consensus.cc:614] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:01.652350 11150 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.582s	user 0.005s	sys 0.008s
W20260709 12:27:01.651684 11166 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.267s	user 0.000s	sys 0.000s
I20260709 12:27:01.661144 11155 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:01.660058  6016 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
W20260709 12:27:01.663247  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.011s	sys 0.038s
W20260709 12:27:01.639191 11025 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.958s	user 0.011s	sys 0.010s
I20260709 12:27:01.590909  5691 catalog_manager.cc:5697] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 7 to 8, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 8 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:01.672360  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.011s	sys 0.038s
W20260709 12:27:01.678480  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.665565 11155 raft_consensus.cc:3055] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:27:01.683401  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 549695us
I20260709 12:27:01.663662 11156 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:01.691398  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.671566 10823 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:01.685428  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.683097 11172 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:01.699313  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1279) took 1202 ms. Trace:
I20260709 12:27:01.681900 11171 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:01.701025  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:01.687479 11155 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:01.700698  6020 rpcz_store.cc:276] 0709 12:27:00.496513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.496684 (+   171us) service_pool.cc:224] Handling call
0709 12:27:01.697111 (+1200427us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b540041e454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:01.699232 (+  2121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1148465024}
I20260709 12:27:01.712607 11157 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.713462 11157 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:01.716917 11157 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:01.696672 11035 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:01.684466 11025 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:01.724318  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:01.698218 11154 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.698546 10818 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.701678 10824 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.731248  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:01.737733  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:01.738399  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:01.757251  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:01.758395  5941 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:27:01.718032 11173 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.721597 11073 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:01.787397  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 74592us
I20260709 12:27:01.778481  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:01.701363 11172 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:01.718885 10981 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:01.733505  5872 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.007s	sys 0.014s
I20260709 12:27:01.788753  5941 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:01.720029 11177 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:01.794878 11167 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.797948  5691 catalog_manager.cc:5697] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 7 to 11, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 11 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:01.818619  5872 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.007s	sys 0.014s
I20260709 12:27:01.733412 11035 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:01.718366 11171 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:01.777316 10824 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:01.754918 10818 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:01.829948  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:01.824828  5941 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:27:01.746791 11154 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:01.816701 11075 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.867165  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:01.911298  5949 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:27:01.816012 11138 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:01.916261  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 126654us
I20260709 12:27:01.910044  5946 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:01.899482 11090 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.927641  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:01.928555  5868 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:27:01.929819  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:01.929966  5941 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:27:01.931365  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:01.827100 10981 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:01.939172  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:01.939023  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:01.940938  5873 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:27:01.938858  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:01.938872  5838 leader_election.cc:400] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 12. Current term is 13.
I20260709 12:27:01.943845  5838 leader_election.cc:403] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:01.945573  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:01.946424  5872 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:01.918922 11061 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:01.915835 10996 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:01.957842  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:01.964349  5871 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:01.955850  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:01.916080 11075 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:01.966500  5868 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:01.793188 11173 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:01.967519  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:01.969288  5865 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:01.970152  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:01.831430 11167 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:01.968127  5874 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:27:01.935117 11183 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.938613 11192 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:01.981096 10776 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:27:01.992671  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 65712.6us
I20260709 12:27:01.825942 11177 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:01.982486 11172 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:01.989642 11025 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:01.938182 11147 consensus_queue.cc:1059] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:27:01.998597 11035 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:01.821478 11155 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:01.998821 11171 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:01.936479 11090 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:01.999433 11182 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:27:02.001108 10981 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:01.923787 10954 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:01.984086 10677 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:01.959462 11069 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:02.015106  6785 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:01.830981 11156 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:01.996616 10776 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:01.966858 11061 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:01.967846 10996 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:01.995718 11191 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:02.028900  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:02.057737  6766 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:27:02.047260  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:02.037032  5868 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:02.040016  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:02.013614 11035 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 12. Current term is 13.
I20260709 12:27:02.011176 11195 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:02.080785 11154 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:02.084944 10954 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:02.086184 11189 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:27:02.025599 11177 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:02.088791 11069 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:02.026296 11056 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.089564 11056 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:02.090683 10824 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:02.001989 11192 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:02.096299  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 165015us
I20260709 12:27:02.099882  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:02.102298  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.075692  6609 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:02.018949 11155 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:02.088928 11190 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:27:01.996222 11183 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:02.129175  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 53597.8us
I20260709 12:27:02.105782 10880 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.132303 10880 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:02.133831 11173 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:02.079044 10988 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:02.138700 11090 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:02.139490 10818 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:02.141688 11167 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:02.106228 11195 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:02.145843  6766 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:02.150871  5691 catalog_manager.cc:5697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 3 to 10. New cstate: current_term: 10 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:27:02.161972  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:02.167907  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:02.161970  6609 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:02.176689 11016 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:02.176934 11199 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:02.177724 11158 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:02.178212 10818 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:02.176784 11073 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:02.179086 10818 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:02.179191 11073 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:02.181563 10943 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:02.182344 10943 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:02.183907  5943 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
W20260709 12:27:02.174014 10783 consensus_meta.cc:287] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.681s	user 0.012s	sys 0.003s
I20260709 12:27:02.186023 11167 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.160989 10967 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:02.186720 10996 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:02.186666 11167 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:02.182624  5946 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:02.188581  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.189693  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.190021  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:02.034077 10677 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:02.191910  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:02.198844  5941 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:02.193179  6766 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:02.195761  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:02.200434  5949 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:02.192436  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:02.181324 11073 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:02.206614  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:02.211302  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:02.211536  5874 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:02.212018  5941 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:02.212611  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:02.178360 10880 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:02.236279 10687 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:02.125784 11075 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:02.197000  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.185262 10943 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:02.217016  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:02.244294  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:02.226619  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:02.245169  5872 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:27:02.245391  5941 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:02.195093  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:02.245657  5874 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
W20260709 12:27:02.246359  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 51392.7us
I20260709 12:27:02.206126  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:02.238989  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 46634us
I20260709 12:27:02.248060  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:02.181329 10818 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:02.250480  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:02.247684  5944 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:02.247213  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:02.256762  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:02.204555 11115 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:02.274971  5838 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:02.276023  5838 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:02.276930  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:02.277865  5944 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:02.260571  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:02.280941  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:02.281716  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:02.282312 10988 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:02.282402  6762 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:02.283586 11203 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:02.284508 11195 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:02.284844  5872 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:02.286662  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:02.201519 10967 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:02.218988 11073 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:02.172544 11151 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:27:02.284238 11203 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:02.287377 11173 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:27:02.289593 11073 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:02.290302 11173 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:02.290769 11173 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.291131 11173 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:02.292565 11073 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:02.293535 11201 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:02.294068 11201 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:02.294842 11203 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:02.294965 11201 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:02.296375 11201 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:02.298072 11056 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:02.286787 11075 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.281711  5874 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:02.302258 11075 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:02.303484 10823 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.605s	user 0.014s	sys 0.001s
I20260709 12:27:02.303798  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:02.304653  6756 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:02.305666 11075 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:02.280782  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:02.206269 11167 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:02.306911  6785 raft_consensus.cc:3055] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:02.307307  6785 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:02.308580  5838 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:02.308816  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:02.079563 11191 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:02.309759  5871 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:27:02.315248  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.286552 11183 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:02.289094  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:02.287847  5872 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:02.269613  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:02.319355  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:02.321295  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:02.321728  5838 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:02.325958  5947 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:02.329591  5838 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:02.252856 10783 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:02.346570  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:02.348091  5913 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:02.275978 10687 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:02.198913 11199 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:02.349403  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:02.314961 11089 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.450s	user 0.008s	sys 0.007s
I20260709 12:27:02.320592  6785 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:02.273376 10880 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:02.376222  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:02.354143  5913 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:02.329524  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 60041.3us
W20260709 12:27:02.377194  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 100974us
I20260709 12:27:02.370787  6016 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 14: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:02.379666  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:02.340058  5949 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:02.355288  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:02.326448  6778 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:27:02.254115 11090 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:02.166072 10776 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:02.374365  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:02.289147 10967 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.344843  5867 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:02.368045  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.372287  6785 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:02.382136  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:02.328203 11035 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:02.382243  5838 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:02.322924 10996 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:02.347579 10988 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:02.361733 11167 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:02.398685  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:02.402884  5872 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:02.403290  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:02.390479  6611 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:02.352185 11159 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:02.407191  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.408077  6024 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:02.376279 10824 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:02.376739 10880 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.415238  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.362879 10687 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:02.371826 11089 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.553s	user 0.009s	sys 0.007s
W20260709 12:27:02.497095  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 79758.9us
I20260709 12:27:02.402320 11177 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:02.393415  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:02.409691  6018 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:02.400331  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:02.153573  5691 catalog_manager.cc:5697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 5 to 6, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 6 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:02.401206  5942 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
W20260709 12:27:02.400997  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 46957.3us
W20260709 12:27:02.492832  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 110434us
I20260709 12:27:02.414791  6611 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:02.394284 11151 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:02.395371 10671 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:02.497017  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:02.408072 11167 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.397846 10967 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:02.407833 11159 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.506588  5873 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:02.376148 11183 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.558053 11183 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:02.520018  6785 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:27:02.394708 11035 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.394610 11069 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:02.386270 11192 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:02.500357  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:02.400827 10996 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.413069 10943 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:02.582901 10943 raft_consensus.cc:697] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:02.584084 10943 consensus_queue.cc:237] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:02.584820  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:02.503893  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 121836us
I20260709 12:27:02.407990 11180 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.155s
I20260709 12:27:02.492312 11200 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:02.548033  5874 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
W20260709 12:27:02.550832  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46645.9us
I20260709 12:27:02.551090 11176 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:02.532521 11187 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:02.551513 11177 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:02.552140 11211 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:02.552992 10941 raft_consensus.cc:3055] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:02.552198  5691 catalog_manager.cc:5697] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 7 to 10, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 10 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:02.554160  5913 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:02.622318  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.622284  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:02.623546  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 69477.6us
I20260709 12:27:02.503480 10687 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:02.581985 11208 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:02.498353 10880 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:02.503480 11089 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:02.566497  5873 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:27:02.651836  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:02.580194  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:02.652925  5947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1671) took 1536 ms (client timeout 1992 ms). Trace:
I20260709 12:27:02.407801 11115 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:02.495358 10824 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.572101 11035 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:02.534042 10671 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.543507 11210 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:27:02.581956 10996 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:02.588835  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.389175 11195 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.132s
I20260709 12:27:02.589682  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.522179 11151 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.589779  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.598213  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.596282  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:02.603605  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:02.544620 11167 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:02.564652 11159 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:02.509397 11056 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
W20260709 12:27:02.591506 11145 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:27:02.609393  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:02.611454  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.612435  5942 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:27:02.626276  6018 raft_consensus.cc:2393] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 18: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:02.590942 11155 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:02.496984 11172 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:02.625495  5913 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:02.713426  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 88014.6us
I20260709 12:27:02.714491  5913 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:02.716096  5913 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:02.630640 10677 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:02.632332  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.639026  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:02.654439  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:02.657100  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:02.653725  5947 rpcz_store.cc:269] 0709 12:27:01.116791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.116968 (+   177us) service_pool.cc:224] Handling call
0709 12:27:02.652713 (+1535745us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400674654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:02.652855 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2718350592}
W20260709 12:27:02.660503  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 245827us
I20260709 12:27:02.652843  5873 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:02.617015 11116 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:02.631227 11205 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 12:27:02.612342 10941 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:02.667665  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.638980 11178 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:27:02.668622  6019 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:02.665328 10824 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:02.594588 11069 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:02.634411 11183 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:02.692582  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 96397.1us
I20260709 12:27:02.693785  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:02.694906  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:02.661963 11173 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:02.779361  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.622831 11174 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 12:27:02.691774 11131 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:27:02.758858 11073 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:02.616153 11187 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:02.756166  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 117225us
W20260709 12:27:02.753494  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.797026  5838 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:02.623116 11211 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:02.759922  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:02.764322  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.627342 11025 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:02.809756  5872 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:02.774176 11191 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:02.764446  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:02.811087  6785 raft_consensus.cc:2410] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 19 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 12:27:02.775156 11075 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:02.768078  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:02.652802 11200 consensus_queue.cc:1059] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:27:02.701494 11084 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:02.700492  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.777191  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:02.775520  5949 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:27:02.776305  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:02.782166  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:02.783460  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 88702.5us
I20260709 12:27:02.758634 11154 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:02.786337  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:02.690279 10671 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:02.630796 11177 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:02.710915 10988 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:27:02.763209 11172 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:02.800554  6778 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:02.654742 11208 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:02.634521  5691 catalog_manager.cc:5697] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:27:02.696790 11151 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:02.668566  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:02.814678  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:02.814826  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:02.851357  6785 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:02.821251 11201 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:02.823800  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:02.790210 10954 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:27:02.831018  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 53920.3us
I20260709 12:27:02.787950 11199 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:02.759706 11155 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:02.792470 11073 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:02.771370 11116 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:02.813577 11090 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:02.811381 11215 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:02.787163 11016 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:27:02.883622 11016 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:02.870054 11073 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:02.899266 11155 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:02.853451  5941 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
W20260709 12:27:02.811028  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 105555us
I20260709 12:27:02.819141 11075 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:02.860540  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:02.832149 10823 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:02.943702  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 83304.1us
I20260709 12:27:02.944953  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:02.945596  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:02.827005 11210 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:02.825394 11084 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:02.865892  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:02.834103 11025 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:02.811944 11158 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:27:03.010512  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:02.839365 11174 raft_consensus.cc:3055] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:02.786028 11089 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:02.900098 11061 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:02.902165 11206 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:27:02.782276 11115 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:02.872256 11090 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:02.908092 11215 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:02.908478 11172 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:02.909325  5691 catalog_manager.cc:5697] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 3 to 10, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 10 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:27:02.913704 11211 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:02.915176 11069 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:02.916486 11173 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:02.919000 11187 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:02.919498 11208 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:02.864033 11224 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:02.846704 11191 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:02.921550  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:02.893155 11222 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:02.840338  6611 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:03.070868 11224 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:03.070951 11222 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:02.913683  6017 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:27:02.940820  5942 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:02.807437 10941 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:02.947113  6609 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:02.948007  5838 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:02.948472 11197 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:02.859115 11201 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:02.949643 11075 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:02.962365  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:02.972196  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:03.005391  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 60572.6us
I20260709 12:27:02.907291 11218 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.024431 11146 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:02.955446 11230 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:27:03.019299 10671 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:02.907313 11116 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:03.029812  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1311) took 2250 ms. Trace:
I20260709 12:27:03.004448 10981 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:02.970397 11139 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:02.970796 11084 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:03.019570 10818 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:03.037580  6757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.005s	sys 0.007s
I20260709 12:27:03.047533 11155 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.020462 11154 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.019685 11174 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:02.969058 10955 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 12:27:03.089095 11187 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:02.973814 10687 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:03.065264  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:03.071414  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 231138us
I20260709 12:27:03.116572  6611 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:02.931269  5990 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:03.080359  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 133307us
W20260709 12:27:03.081652  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 133704us
W20260709 12:27:03.117568  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 186387us
I20260709 12:27:03.119138  6016 raft_consensus.cc:2368] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15.
I20260709 12:27:03.091256 11173 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:03.091028 10988 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:03.088263  6023 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:27:03.093183  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 257166us
I20260709 12:27:03.093429  6018 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:27:03.122143  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:03.098170  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:03.100502  6757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.005s	sys 0.007s
I20260709 12:27:03.103469  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:03.103865  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:03.104051  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.011621 11210 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:03.073523 11224 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:03.095150  5866 rpcz_store.cc:276] 0709 12:27:00.769013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.769708 (+   695us) service_pool.cc:224] Handling call
0709 12:27:00.774516 (+  4808us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:01.079984 (+305468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:01.080018 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:01.080561 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 12:27:01.235140 (+154579us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:02.413911 (+1178771us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b18000818e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d667b3c7 00007f39d2c8d9a7 00007f39d2c9d0bf 00007f39d2c9d038
0709 12:27:02.654631 (+240720us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.978244 (+323613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.978378 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:02.978448 (+    70us) raft_consensus.cc:1685] finished
0709 12:27:03.019539 (+ 41091us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.019803 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31620,"prepare.queue_time_us":52481,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":68,"raft.queue_time_us":42155,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":40767,"spinlock_wait_cycles":408821120,"thread_start_us":949425,"threads_started":4,"wal-append.queue_time_us":579912,"wal-append.run_cpu_time_us":2873,"wal-append.run_wall_time_us":1249574}
I20260709 12:27:03.056887  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:03.133942 11210 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:03.134305 11204 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.135111 11204 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:03.136365 10671 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:03.112762 11069 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.138028  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.138923  5872 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:03.121589  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:03.121589  5990 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:03.074504 11222 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:03.129741 11233 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:03.089726 11201 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:03.079746 11191 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:03.156359  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.140825  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.182355  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.105571 11218 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:03.159641 11204 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:03.160477 11229 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:27:03.161016 10776 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:03.163357 10687 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:03.164219  6024 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:03.151813 10955 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:03.228016 11174 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:03.137519 11069 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:03.274992 11174 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:03.276981 11069 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:03.151504 11159 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:03.279712 11159 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:03.166643  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:03.280603 10880 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.610s	user 0.011s	sys 0.000s
W20260709 12:27:03.170710  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 48665.5us
I20260709 12:27:03.139654  5948 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:03.105605 11197 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:03.121788 11187 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.198621  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.290982  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1400) took 1429 ms. Trace:
I20260709 12:27:03.291661  6768 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:03.139318 11176 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:03.218628 11226 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:27:03.228016 11116 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:03.107457 10818 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:03.110487 11155 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:03.303597  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 133291us
I20260709 12:27:03.174989 11173 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:03.175575 10671 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:03.234190  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.304839  6785 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:03.221275  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.175302 10677 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:03.255357 11215 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:03.261811  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.306869  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:03.113319 11154 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:03.322409  6785 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:03.323948  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:03.320945  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:03.327494  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.193787  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:03.281044  5942 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:03.106153 11178 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.281735  5873 raft_consensus.cc:2368] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15.
I20260709 12:27:03.282449  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1385) took 1066 ms. Trace:
I20260709 12:27:03.230261 11158 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:03.331039  6766 rpcz_store.cc:276] 0709 12:27:02.239862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.242514 (+  2652us) service_pool.cc:224] Handling call
0709 12:27:03.280768 (+1038254us) spinlock_profiling.cc:243] Waited 8.28 ms on lock 0x7b580001f898. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:27:03.281031 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.281289 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.281312 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:03.281362 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:03.281392 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.281608 (+   216us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.282364 (+   756us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19044736}
W20260709 12:27:03.286459  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 128434us
I20260709 12:27:03.334437  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:03.287895  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:03.289196  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1379) took 1193 ms. Trace:
I20260709 12:27:03.291647  5940 rpcz_store.cc:276] 0709 12:27:01.862171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.862464 (+   293us) service_pool.cc:224] Handling call
0709 12:27:01.862985 (+   521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:01.930005 (+ 67020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:01.930037 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:01.930425 (+   388us) log.cc:844] Serialized 26 byte log entry
0709 12:27:02.011998 (+ 81573us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:02.012056 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.288838 (+1276782us) spinlock_profiling.cc:243] Waited 66.5 ms on lock 0x7b54006a3bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:03.288890 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:03.288940 (+    50us) raft_consensus.cc:1685] finished
0709 12:27:03.288956 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.290890 (+  1934us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2016,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":17497,"spinlock_wait_cycles":153020288,"thread_start_us":73441,"threads_started":1,"wal-append.queue_time_us":88807,"wal-append.run_cpu_time_us":1948,"wal-append.run_wall_time_us":1002099}
I20260709 12:27:03.262619 11139 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:03.301744  6768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:03.337463  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:03.313472 10818 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:03.314813 10967 consensus_meta.cc:287] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.743s	user 0.017s	sys 0.000s
I20260709 12:27:03.305838 10671 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.318764 11233 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:03.294697 11187 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:03.335435  6793 rpcz_store.cc:276] 0709 12:27:02.095574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.096033 (+   459us) service_pool.cc:224] Handling call
0709 12:27:02.125585 (+ 29552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.138382 (+ 12797us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.138413 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:02.138454 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:02.138487 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.288926 (+1150439us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b54002c24d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:03.288972 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.289130 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28847872}
I20260709 12:27:03.335878  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:03.335902  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 142248us
I20260709 12:27:03.327823 10988 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:03.355865  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:03.356823  6015 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 12: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:03.303591 11116 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:03.187341 11061 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:03.334805  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:03.339569  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51755.5us
I20260709 12:27:03.347373 11232 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:03.385973  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 51266.8us
I20260709 12:27:03.348841 10671 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:03.287166 10880 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:03.341378  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:03.326356 11167 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.614s	user 0.010s	sys 0.005s
I20260709 12:27:03.338102  6768 raft_consensus.cc:606] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:03.350986  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:03.394773  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 43880.5us
I20260709 12:27:03.352764  5942 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:03.338388  5940 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:03.352137  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.398861  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:03.340193 11089 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:03.358376  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.340756 11211 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:03.330366 11215 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:27:03.365350 11221 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.520s	user 0.002s	sys 0.000s
W20260709 12:27:03.370361 11238 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 12:27:03.339319 11178 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:03.370787 11075 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:03.371706 10824 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.002s
I20260709 12:27:03.373780 11197 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:03.374981 11035 consensus_meta.cc:287] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.679s	user 0.013s	sys 0.000s
I20260709 12:27:03.338996 10677 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:03.340552 11131 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:27:03.347852 11158 raft_consensus.cc:697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:03.393211  6021 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:03.393782  6768 raft_consensus.cc:614] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:03.400323  5838 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:03.384980 10996 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.684s	user 0.011s	sys 0.002s
I20260709 12:27:03.381593 11159 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:03.392086 11073 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:03.379494 10988 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:03.541538 10988 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:03.543231 11239 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.309s	user 0.002s	sys 0.000s
I20260709 12:27:03.376629 11204 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:03.390798 11199 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 12:27:03.544039 11204 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:03.545315 11245 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:27:03.546542  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:03.546931 11204 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:03.548022  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:03.393910 11065 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.442358  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:03.456928  6020 raft_consensus.cc:2368] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 17. Current term is 18.
I20260709 12:27:03.381460 11192 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:03.393402 11146 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:03.476603  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:03.347239 11139 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:03.407063 11222 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:03.409561 11211 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:03.575285 11146 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:03.402407 11243 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:03.580061  5946 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:27:03.581425 11192 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:03.470692 11151 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.007s
I20260709 12:27:03.582545  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:03.583117 11151 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:03.347729 10967 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:03.526197  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 125939us
I20260709 12:27:03.526784 11156 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:03.527426 11069 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.528335  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.528810 11154 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:03.534466 11155 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:03.537078 10996 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:03.480324 10702 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:03.538314 11159 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:03.496417 11035 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:03.539289 11158 consensus_queue.cc:237] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:03.542730 10677 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:03.548079  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:03.550460  5949 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:03.441846  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:03.551721  5940 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:03.404000 11167 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:03.559968 11204 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:03.560487 11227 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:03.561831 11241 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:27:03.481760 10824 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:03.570309 11199 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:03.565659  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:03.570031  5947 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:03.570940  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.575784 11211 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:03.384446 11061 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:03.691958  5870 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:03.583235  5946 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:03.659503  6768 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:27:03.538786 11174 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:03.695399  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:03.641129 11224 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.666189  5838 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:03.555090 11191 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:03.446156 11215 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:03.670094  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:03.553809 11218 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:03.780848  5944 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:03.781296  6768 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:03.677112  5868 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:03.583868 11192 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:03.563203  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1324) took 1168 ms. Trace:
I20260709 12:27:03.688050  5871 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:27:03.385337 11173 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:03.539336 11075 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:03.703830  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:03.704308  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.674525 11146 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:03.700431  5838 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:03.673439 11156 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:03.766486  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:03.585297 11178 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:03.678510 11139 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:03.575304 11065 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:03.786801  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 292815us
I20260709 12:27:03.788787  5869 rpcz_store.cc:276] 0709 12:27:02.396050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.522666 (+126616us) service_pool.cc:224] Handling call
0709 12:27:02.554107 (+ 31441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.554505 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.554537 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:02.555164 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 12:27:02.694391 (+139227us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:02.694432 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.694504 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:03.364590 (+670086us) spinlock_profiling.cc:243] Waited 4.64 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:03.553249 (+188659us) raft_consensus.cc:1685] finished
0709 12:27:03.553276 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.563094 (+  9818us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13742,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":22443,"spinlock_wait_cycles":10663680,"thread_start_us":138546,"threads_started":1,"wal-append.queue_time_us":234738}
I20260709 12:27:03.665522 11069 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:03.666689 10702 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:03.808279  6768 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:27:03.765266 11155 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:03.807065  5865 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:27:03.798337 11073 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:03.830547  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1391) took 1542 ms. Trace:
I20260709 12:27:03.833873 10981 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:03.854928 10981 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:03.854988  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.855854  6768 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:03.856267 11151 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:03.855931 10981 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:03.856904  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.835162  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1398) took 2003 ms. Trace:
I20260709 12:27:03.802340 11191 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:03.863286  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:03.864256  5869 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:27:03.840118 10941 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:03.842298  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 441712us
I20260709 12:27:03.813711  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:27:03.685961 11204 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:03.584787 11090 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:03.825307 10880 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:03.851868  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.802894 11177 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.698076 11199 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:03.836024  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1291) took 1251 ms. Trace:
I20260709 12:27:03.850812 10783 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:03.853717  6775 rpcz_store.cc:276] 0709 12:27:02.288052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.289180 (+  1128us) service_pool.cc:224] Handling call
0709 12:27:02.289481 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.404952 (+115471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.405005 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:02.405073 (+    68us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:02.405125 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.830218 (+1425093us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540021fa54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:03.830282 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.830465 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264798080}
I20260709 12:27:03.767887 11224 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:03.750030 11174 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:03.814078 11192 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:03.884065  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:03.850705 11155 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:03.911530  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:03.913920  6021 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:03.858479  5945 rpcz_store.cc:276] 0709 12:27:01.832164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.832462 (+   298us) service_pool.cc:224] Handling call
0709 12:27:01.860067 (+ 27605us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:01.932189 (+ 72122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:01.932223 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:01.957740 (+ 25517us) log.cc:844] Serialized 26 byte log entry
0709 12:27:02.380489 (+422749us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:02.380546 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.834679 (+1454133us) spinlock_profiling.cc:243] Waited 68.1 ms on lock 0x7b5400833954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:03.834737 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:03.834797 (+    60us) raft_consensus.cc:1685] finished
0709 12:27:03.834811 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.835091 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6259,"raft.queue_time_us":22792,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":8136,"spinlock_wait_cycles":156530048,"thread_start_us":376345,"threads_started":1,"wal-append.queue_time_us":681333}
I20260709 12:27:03.856916 11151 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:03.686547 11227 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:03.871359  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:03.878214  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:03.889899  6756 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:03.894747  6775 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:03.899955  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:03.892812  6017 rpcz_store.cc:276] 0709 12:27:02.584324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.589339 (+  5015us) service_pool.cc:224] Handling call
0709 12:27:03.835834 (+1246495us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54006b9ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:03.835970 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562914560}
I20260709 12:27:03.924239  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.855046 11246 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:03.928656  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:03.931461  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:03.932714  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:03.933511  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 117724us
I20260709 12:27:03.935526  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:03.925275  6775 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:03.910650 10783 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:03.938349 10702 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:03.944936 11065 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:03.945796 11224 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:03.946055  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:03.949841 11220 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:03.951067  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1326) took 1443 ms. Trace:
W20260709 12:27:03.951385 11254 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:27:03.951561  5868 rpcz_store.cc:276] 0709 12:27:02.508173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.638139 (+129966us) service_pool.cc:224] Handling call
0709 12:27:02.638436 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.782093 (+1143657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.950766 (+168673us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54007350d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:03.951001 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2387962624}
I20260709 12:27:03.904237 11180 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.920698 11155 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:03.913187 11242 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:03.969573  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.970670  6756 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:03.880856 11191 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:03.931100 11090 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:03.972977  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 272632us
I20260709 12:27:03.954891 11208 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:03.977648 11035 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:03.988744  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:03.989966  5874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 1751) took 1671 ms (client timeout 1665 ms). Trace:
W20260709 12:27:03.990501  5874 rpcz_store.cc:269] 0709 12:27:02.318575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.318717 (+   142us) service_pool.cc:224] Handling call
0709 12:27:03.989736 (+1671019us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54001d0454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:03.989874 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383343488}
W20260709 12:27:03.979207 11257 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:27:03.904409 11177 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:03.986560 11262 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:03.939584 11154 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:27:03.992401  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 49869.9us
I20260709 12:27:03.929543 11151 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:03.963620 10671 consensus_meta.cc:287] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.564s	user 0.009s	sys 0.004s
I20260709 12:27:04.010134 10981 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.963637 11224 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:04.011541 11251 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:03.936389 11246 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:03.939821 11156 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:04.014871 11261 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:04.016106 11167 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:04.018646 11191 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:03.920323 11192 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:04.093619 11192 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:04.025714  5873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1060) took 1215 ms (client timeout 1561 ms). Trace:
W20260709 12:27:04.094722  5873 rpcz_store.cc:269] 0709 12:27:02.810301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.814288 (+  3987us) service_pool.cc:224] Handling call
0709 12:27:03.993459 (+1179171us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b5400027e54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:04.025599 (+ 32140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":989956864}
I20260709 12:27:04.002013  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1251) took 1247 ms. Trace:
I20260709 12:27:04.097530  6019 rpcz_store.cc:276] 0709 12:27:02.754718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.776436 (+ 21718us) service_pool.cc:224] Handling call
0709 12:27:03.835990 (+1059554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.836198 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.836222 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:03.836266 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:03.836298 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.836357 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.001666 (+165309us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54006b9ed0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:04.001905 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2435800960}
I20260709 12:27:04.099879  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:04.027310  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 91908.9us
I20260709 12:27:04.103153  6778 raft_consensus.cc:2410] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 12:27:04.105329  6020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1246) took 1732 ms (client timeout 1640 ms). Trace:
W20260709 12:27:04.029379  6016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1247) took 1630 ms (client timeout 1824 ms). Trace:
W20260709 12:27:04.106329  6016 rpcz_store.cc:269] 0709 12:27:02.399343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.399675 (+   332us) service_pool.cc:224] Handling call
0709 12:27:03.996625 (+1596950us) spinlock_profiling.cc:243] Waited 715 ms on lock 0x7b54001612d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:04.029293 (+ 32668us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1644279040}
I20260709 12:27:04.107487  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 977) took 1515 ms. Trace:
I20260709 12:27:04.108049  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.109150  6016 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:04.047740  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:03.963372 11180 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:04.011073  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:04.045387 11089 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:04.054244  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 169199us
I20260709 12:27:04.054807  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.055958  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:03.983140 11199 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:04.062654 11208 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.066737  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1061) took 1253 ms. Trace:
I20260709 12:27:04.059779 10981 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:04.051678 11201 consensus_meta.cc:287] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.852s	user 0.012s	sys 0.005s
I20260709 12:27:03.984184  5946 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:27:04.141160  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:04.116014 11256 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:27:04.116717 11266 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.108050  5949 rpcz_store.cc:276] 0709 12:27:02.592546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.831376 (+238830us) service_pool.cc:224] Handling call
0709 12:27:02.831782 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.668915 (+837133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.107155 (+438240us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b5400632254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:04.107408 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1652228864}
W20260709 12:27:04.105940  6020 rpcz_store.cc:269] 0709 12:27:02.373294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.373935 (+   641us) service_pool.cc:224] Handling call
0709 12:27:04.105021 (+1731086us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540026fa54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:04.105231 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2454733568}
I20260709 12:27:04.118145  5873 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:27:04.053465  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.119903  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.120844  6756 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:04.121609  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:04.044516 10823 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.121836 11159 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:03.980808 11242 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:04.087116 11251 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:04.055826 10671 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:04.083546 11224 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:04.127143 11192 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:04.174078 11192 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:04.046226 11262 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:04.128024  6021 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:04.129153  6793 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:04.135746  5874 rpcz_store.cc:276] 0709 12:27:02.812960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.991750 (+1178790us) service_pool.cc:224] Handling call
0709 12:27:04.066136 (+ 74386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.066369 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:04.066398 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:04.066445 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:04.066480 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.066536 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.066654 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:04.137137  6609 leader_election.cc:400] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15.
I20260709 12:27:04.139006 11180 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:04.139544 11084 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.026s	user 0.014s	sys 0.005s
W20260709 12:27:04.123008 11257 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:27:04.143069  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:04.109416  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:04.060546 11035 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:04.194190  6022 raft_consensus.cc:2368] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19.
I20260709 12:27:04.122635 10824 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:04.162195  5942 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:27:04.127501 11261 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:04.172022  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.175942  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:04.178164  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1063) took 1358 ms. Trace:
I20260709 12:27:04.199092  5873 rpcz_store.cc:276] 0709 12:27:02.820117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.096136 (+1276019us) service_pool.cc:224] Handling call
0709 12:27:04.096737 (+   601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:04.177860 (+ 81123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.178101 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:04.200668  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:04.125878  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:04.180949  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 43961.5us
W20260709 12:27:04.204095  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.204188 11251 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:04.185420  6793 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:27:04.205780 11195 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:04.192812  5947 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:04.181411 10823 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:04.214221  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:04.170573 11246 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:04.165565 11227 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:04.145862 11208 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:04.209532  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:04.163829 10981 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:04.211455  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:04.215997  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:04.183470 11262 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:04.217459  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:04.221450  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.219745  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:04.208184  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 82577us
W20260709 12:27:04.227485  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.145933 11201 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:04.235494  5867 raft_consensus.cc:2393] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 14: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:27:04.236788 11187 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.881s	user 0.010s	sys 0.001s
I20260709 12:27:04.207600 11035 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:04.157681 11228 consensus_queue.cc:1059] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 12:27:04.229254 11061 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:04.227734  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.228477 11242 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:04.237569  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:04.237932  5865 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
W20260709 12:27:04.238323  5948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 987) took 1230 ms (client timeout 1523 ms). Trace:
I20260709 12:27:04.251006  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.239930  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1328) took 1636 ms. Trace:
I20260709 12:27:04.246431 11208 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:04.237330 11187 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:04.244324 11151 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:04.259665 11192 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:04.240717  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.261866  6766 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:04.261999  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.257231 10687 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:04.263866 10687 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:04.263934  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1053) took 1485 ms. Trace:
I20260709 12:27:04.264541  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:04.265028 10687 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:04.265955  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1325) took 1853 ms. Trace:
I20260709 12:27:04.266253 10687 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:04.266476  5871 rpcz_store.cc:276] 0709 12:27:02.412765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.593235 (+180470us) service_pool.cc:224] Handling call
0709 12:27:02.593670 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:03.789103 (+1195433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.265618 (+476515us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54002f62d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:04.265867 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2510407936}
I20260709 12:27:04.267588 11155 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:04.268241  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:04.268486 11204 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:04.207768 10941 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.217231 11199 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:04.270183  5871 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 12.
I20260709 12:27:04.270776 11199 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.271220 11199 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:04.185137 11266 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:04.271450  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:04.278048  5871 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:27:04.264493  6766 rpcz_store.cc:276] 0709 12:27:02.779535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.698772 (+919237us) service_pool.cc:224] Handling call
0709 12:27:03.797468 (+ 98696us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.262394 (+464926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.263661 (+  1267us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b54002a12d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:04.263863 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1067430656}
I20260709 12:27:04.243678  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.279258  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:04.263338  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:04.245990  6016 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:04.256531  5872 rpcz_store.cc:276] 0709 12:27:02.603503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.168357 (+564854us) service_pool.cc:224] Handling call
0709 12:27:03.168677 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.250462 (+ 81785us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.250512 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:03.250576 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:03.250630 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.239684 (+989054us) spinlock_profiling.cc:243] Waited 81.4 ms on lock 0x7b54001446d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:04.239744 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.239871 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":187242368}
I20260709 12:27:04.280794  5871 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:27:04.283885  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:04.285547  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 156689us
W20260709 12:27:04.256215  5948 rpcz_store.cc:269] 0709 12:27:03.007682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.048326 (+ 40644us) service_pool.cc:224] Handling call
0709 12:27:04.238132 (+1189806us) spinlock_profiling.cc:243] Waited 7.61 ms on lock 0x7b5400602a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:04.238262 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17494272}
I20260709 12:27:04.295768  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:04.242475 11228 consensus_queue.cc:1243] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:04.322914  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:04.191713 11084 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:04.324971  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.322774  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:04.324927  5873 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:04.324227  6018 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 12:27:04.330081  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 207216us
I20260709 12:27:04.331115  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:04.261413 11187 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:04.332154  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:04.337044  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1065) took 1481 ms. Trace:
I20260709 12:27:04.332958  5871 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:04.334619  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:04.337471  5867 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 9.
I20260709 12:27:04.239570 11201 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:04.331161  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:04.343874  5872 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:04.343972  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:04.243878 11246 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:04.346140  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:04.347010  5873 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:04.239079 10671 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:04.340337  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1064) took 1500 ms. Trace:
I20260709 12:27:04.348261  5871 rpcz_store.cc:276] 0709 12:27:02.839807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.320297 (+1480490us) service_pool.cc:224] Handling call
0709 12:27:04.320782 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.339983 (+ 19201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.340268 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:04.242383 11035 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:04.339083  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:04.273569 11224 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:04.341213  6015 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:27:04.266387 11261 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:04.338601  5869 rpcz_store.cc:276] 0709 12:27:02.855721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.336137 (+1480416us) service_pool.cc:224] Handling call
0709 12:27:04.336461 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.336679 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:04.336708 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:04.336756 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:04.336793 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.336852 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.336985 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:04.270231 10941 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:04.332283  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:04.323280 10824 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.352252  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:04.365391  5949 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:04.353643  6766 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:04.351619  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:04.366536  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:04.336794 11178 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:04.363567  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.369513  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:04.364574  5867 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:04.374265  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1066) took 1500 ms. Trace:
I20260709 12:27:04.380653  5867 rpcz_store.cc:276] 0709 12:27:02.874369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.338630 (+1464261us) service_pool.cc:224] Handling call
0709 12:27:04.361096 (+ 22466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.373877 (+ 12781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.374186 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:04.382339  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1760) took 1006 ms. Trace:
I20260709 12:27:04.346060 11246 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.387605  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1753) took 1013 ms. Trace:
I20260709 12:27:04.389251  5873 rpcz_store.cc:276] 0709 12:27:03.373779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.360653 (+986874us) service_pool.cc:224] Handling call
0709 12:27:04.387053 (+ 26400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.387265 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:04.387291 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:04.387339 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:04.387374 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.387432 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.387548 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:04.350669 11204 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:04.387668  5869 rpcz_store.cc:276] 0709 12:27:03.375636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.372714 (+997078us) service_pool.cc:224] Handling call
0709 12:27:04.373204 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.373457 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:04.373498 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:04.373556 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:04.373602 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.382086 (+  8484us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.382259 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:04.341041 11065 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:27:04.395160  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1414) took 1003 ms. Trace:
I20260709 12:27:04.406607  5867 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:04.409089  5872 raft_consensus.cc:2410] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:27:04.341667 11192 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:27:04.377339 11201 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:04.384748 10981 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:04.409052  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1761) took 1033 ms. Trace:
I20260709 12:27:04.412465  5949 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:04.414413  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.415522  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.407886  5948 rpcz_store.cc:276] 0709 12:27:03.391823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.393955 (+1002132us) service_pool.cc:224] Handling call
0709 12:27:04.394471 (+   516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.394718 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:04.394758 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:04.394820 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:04.394866 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.394937 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.395087 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:04.420228  5867 rpcz_store.cc:276] 0709 12:27:03.375861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.381741 (+1005880us) service_pool.cc:224] Handling call
0709 12:27:04.400882 (+ 19141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.408625 (+  7743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.408962 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:04.420555  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1762) took 1029 ms. Trace:
I20260709 12:27:04.423183  5869 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:04.424053  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:04.431967  5872 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:27:04.422989  5873 rpcz_store.cc:276] 0709 12:27:03.391290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.390260 (+998970us) service_pool.cc:224] Handling call
0709 12:27:04.420028 (+ 29768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.420223 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:04.420246 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:04.420291 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:04.420322 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.420380 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.420493 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:04.427173 11235 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.204s	user 0.000s	sys 0.009s
W20260709 12:27:04.428449  6778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1052) took 1661 ms (client timeout 1927 ms). Trace:
I20260709 12:27:04.423003 11155 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:04.429203  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.421432  6018 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:04.397044 11246 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:04.396219 11204 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:04.433871  6778 rpcz_store.cc:269] 0709 12:27:02.767345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.921221 (+153876us) service_pool.cc:224] Handling call
0709 12:27:04.428193 (+1506972us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540015efd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:04.428366 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2379639680}
I20260709 12:27:04.408311 11061 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:04.445498  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:04.376356 10824 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:04.455217  6778 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:27:04.457098  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:04.457995  5873 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:04.420608 11201 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:04.447782  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:04.461455  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:04.452486  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:04.465405 10981 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:04.465886 10981 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.466231 10981 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:04.473775  6018 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:04.478899  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.480031  6017 raft_consensus.cc:2410] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:27:04.481406 11228 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:04.483160 11228 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:04.484339 11228 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:04.477389 10824 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:04.490775  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:04.493531 11120 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:04.495797 11120 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:04.502854  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1329) took 1864 ms. Trace:
I20260709 12:27:04.503543  5866 rpcz_store.cc:276] 0709 12:27:02.638057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.366838 (+728781us) service_pool.cc:224] Handling call
0709 12:27:03.367294 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.796917 (+429623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.796969 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:03.797031 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:03.797070 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.502470 (+705400us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54004f5bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:04.502545 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.502763 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":987213056}
I20260709 12:27:04.509778  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 1228) took 1272 ms. Trace:
I20260709 12:27:04.510303  6768 rpcz_store.cc:276] 0709 12:27:03.237285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.870399 (+633114us) service_pool.cc:224] Handling call
0709 12:27:03.870730 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.065351 (+194621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:04.065407 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:04.065478 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:04.065526 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.509479 (+443953us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b5400003554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:04.509529 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.509706 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446821120}
I20260709 12:27:04.505559  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:04.511869  5866 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:27:04.512445  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:04.512794  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:04.420387 11035 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
W20260709 12:27:04.515292  6762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1051) took 1759 ms (client timeout 1644 ms). Trace:
W20260709 12:27:04.515744  6762 rpcz_store.cc:269] 0709 12:27:02.755716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.767646 (+ 11930us) service_pool.cc:224] Handling call
0709 12:27:04.515069 (+1747423us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400007654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:04.515216 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2839484160}
I20260709 12:27:04.515507  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:04.517680 11281 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:04.518129 11281 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:04.518553 11281 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:04.521013  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.522621 11266 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:04.525936  5872 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:27:04.525868  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:04.450613 11204 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:04.557574  5873 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:04.545233  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:04.560729  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:04.547073 11276 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:04.524945 11201 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:04.622749  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 60831.9us
W20260709 12:27:04.545651 11211 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.778s	user 0.008s	sys 0.011s
I20260709 12:27:04.631155  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:04.623919  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:04.666589  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:04.628718 11255 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:04.668138 10967 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:04.621732 11276 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:04.666648 11177 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:04.677595  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.683183 11195 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:04.685176 11215 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:04.688010 11116 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:04.690102 11264 raft_consensus.cc:3055] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:04.712872  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:04.637090 10824 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:04.674336 11275 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:04.634095 11224 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:04.724192  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.640187  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:04.727670  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 87600.4us
I20260709 12:27:04.702185 11192 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:04.698436  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:04.732184  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 453920us
I20260709 12:27:04.644575 11211 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:04.707459  6756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.012s	sys 0.010s
I20260709 12:27:04.715381  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:04.714395  6775 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:04.716722  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:04.718273  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.720672  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:04.738058  6762 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 18: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:27:04.739013  6756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.012s	sys 0.010s
I20260709 12:27:04.700552  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:04.706040 11276 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:04.746613  5943 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:27:04.747653  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:04.748469  6768 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:04.748677  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:04.744735  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.713467 11264 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:27:04.744542  6773 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 1212) took 2432 ms (client timeout 1947 ms). Trace:
I20260709 12:27:04.755158  5913 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:04.758170  6773 rpcz_store.cc:269] 0709 12:27:02.299124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.308617 (+  9493us) service_pool.cc:224] Handling call
0709 12:27:03.137858 (+829241us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b580001f898. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:27:04.722105 (+1584247us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b540004bfd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:04.731674 (+  9569us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1970058752}
I20260709 12:27:04.761487  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:04.761989  5913 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:04.762898  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:04.704592 11255 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:04.768314  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:04.764541 11178 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:04.753471  5868 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:04.744766  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:04.776274  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:04.752302  5944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1683) took 2392 ms (client timeout 1666 ms). Trace:
I20260709 12:27:04.722430 11275 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.713033 11195 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:04.763095  5940 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:04.751456  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:04.763365  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:04.721845 11224 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:04.739362 11215 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:27:04.771767  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 325615us
I20260709 12:27:04.795064  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:04.798875  6768 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 14: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:04.799093  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.777420  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:04.779160  5868 raft_consensus.cc:2410] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:27:04.755339  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.723090 10824 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:04.777858  5944 rpcz_store.cc:269] 0709 12:27:02.350962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.351206 (+   244us) service_pool.cc:224] Handling call
0709 12:27:04.742555 (+2391349us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54008aff54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:04.742775 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3227803008}
W20260709 12:27:04.793730 11262 consensus_meta.cc:287] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.006s
I20260709 12:27:04.770507 11084 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:04.787001  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1402) took 1456 ms. Trace:
I20260709 12:27:04.789070  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:04.813099  6611 leader_election.cc:400] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15.
I20260709 12:27:04.766325 11065 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:04.794665  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:04.742269 11192 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:04.796867 10823 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.557s	user 0.006s	sys 0.007s
I20260709 12:27:04.826555  6785 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:27:04.804749  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:04.811182  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:04.719339 10981 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:04.814329  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:04.852444  5840 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:04.797372 11177 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:04.822670  6611 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:04.809618 11215 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:04.820668 11035 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:04.827421  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1763) took 1351 ms. Trace:
I20260709 12:27:04.800423 11224 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:04.789491 11275 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:04.825738  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:04.876999  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 51359us
I20260709 12:27:04.867754  5869 rpcz_store.cc:276] 0709 12:27:03.475903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.396769 (+920866us) service_pool.cc:224] Handling call
0709 12:27:04.420686 (+ 23917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.430912 (+ 10226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.827170 (+396258us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b54002f9ed0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:04.827350 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54201600}
I20260709 12:27:04.803362 11178 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:04.830992  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:04.890552  5868 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:04.854806 11176 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:27:04.849709  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 45061.5us
I20260709 12:27:04.904711  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:04.906193  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:04.852787  5944 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:04.789829 11180 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:04.826468  6766 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:04.907826  6766 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:04.821187  6757 rpcz_store.cc:276] 0709 12:27:03.331949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.246301 (+914352us) service_pool.cc:224] Handling call
0709 12:27:04.262698 (+ 16397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.262943 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:04.444666 (+181723us) log.cc:844] Serialized 14 byte log entry
0709 12:27:04.786203 (+341537us) log.cc:844] Serialized 14 byte log entry
0709 12:27:04.786439 (+   236us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:04.786490 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:04.786544 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.786708 (+   164us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.786881 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5575,"prepare.queue_time_us":148097,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":6541,"raft.queue_time_us":6805,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":155,"thread_start_us":514692,"threads_started":2}
W20260709 12:27:04.871142  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.831853 11065 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:04.855245 11177 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:04.895646  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:04.830508  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:04.896514 11220 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:04.851714 11195 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:04.850481 10823 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:04.932041  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1397) took 1630 ms. Trace:
I20260709 12:27:04.898469 11208 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:04.898221  6611 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:04.829056 11262 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:04.965088 11061 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:04.912843 10967 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:04.966693 10967 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:04.967254 11262 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:04.968600 11252 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:04.900810 11192 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:04.902347 11255 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:04.810190 11264 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:04.898219 11287 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:27:04.970271 11264 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:04.893324  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:04.972644  5869 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:27:04.972760  5870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 1752) took 2653 ms (client timeout 3344 ms). Trace:
I20260709 12:27:04.973942  5869 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:27:04.975101 11195 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:04.975772  5869 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 12:27:04.913327 11065 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:04.973197  5870 rpcz_store.cc:269] 0709 12:27:02.319024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.319151 (+   127us) service_pool.cc:224] Handling call
0709 12:27:04.972537 (+2653386us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400027954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:04.972672 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3079018880}
I20260709 12:27:04.979825  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1059) took 2170 ms. Trace:
W20260709 12:27:04.963357  5942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1688) took 1954 ms (client timeout 2115 ms). Trace:
I20260709 12:27:04.942787 11180 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:04.943838  6766 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:04.986478  5874 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:04.986997  6766 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:04.965104  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 67012.6us
I20260709 12:27:04.988648  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:04.893093 11285 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:27:04.982969 11220 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:04.958739 11286 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 12:27:04.981904  5942 rpcz_store.cc:269] 0709 12:27:03.009364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.402583 (+393219us) service_pool.cc:224] Handling call
0709 12:27:04.963073 (+1560490us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b54003098d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:04.963266 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1132989824}
I20260709 12:27:04.862320 11155 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:04.912348 10954 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:04.911504 11176 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:04.978868  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:05.027973  5947 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:04.877189 11234 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:05.045156 11234 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.045678 11234 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:04.977737  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:05.048601  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:05.050249 11215 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:04.943387 11174 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:05.054908  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1410) took 1666 ms. Trace:
I20260709 12:27:05.055450  5943 rpcz_store.cc:276] 0709 12:27:03.388059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.867184 (+479125us) service_pool.cc:224] Handling call
0709 12:27:04.742924 (+875740us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:04.754014 (+ 11090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.054601 (+300587us) spinlock_profiling.cc:243] Waited 837 ms on lock 0x7b54008aff50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:05.054813 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1924147584}
I20260709 12:27:04.982802 11180 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:04.961076  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:04.996675  6611 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:04.983680  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.985562  6756 rpcz_store.cc:276] 0709 12:27:03.301950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.173245 (+871295us) service_pool.cc:224] Handling call
0709 12:27:04.173675 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:04.174087 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:04.174121 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:04.174623 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 12:27:04.739891 (+565268us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:04.740061 (+   170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.740208 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:04.931724 (+191516us) raft_consensus.cc:1685] finished
0709 12:27:04.931748 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.931967 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1063,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":9637,"thread_start_us":564676,"threads_started":1,"wal-append.queue_time_us":645223}
I20260709 12:27:05.070475  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:04.914557 11243 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.144s
I20260709 12:27:04.967362 10967 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:05.002177  6016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.052s	sys 0.045s
W20260709 12:27:05.079453  6016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.052s	sys 0.045s
I20260709 12:27:05.000598  5868 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:27:05.080687  5872 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: [12.7-13.8]
I20260709 12:27:05.001420  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:04.969189 10824 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:05.007375  5990 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:04.913827  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:04.933183 11292 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:27:04.892062 11147 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:04.932251 11283 log.cc:927] Time spent T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Append to log took a long time: real 0.390s	user 0.004s	sys 0.000s
I20260709 12:27:04.874960 11035 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.998323 11252 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:04.997707 11177 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:04.961653 10783 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:04.938158 11069 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:04.915326 11199 consensus_meta.cc:287] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.644s	user 0.015s	sys 0.000s
I20260709 12:27:05.131920 11199 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:05.030231  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.134263 11199 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:05.103418 11035 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:05.134955 11199 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:27:05.032524 11291 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:27:05.033788  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:05.036471 10981 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:05.006342 11220 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:05.137964  5948 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:05.138595  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 993) took 1196 ms. Trace:
I20260709 12:27:05.138994  5942 rpcz_store.cc:276] 0709 12:27:03.942663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.137622 (+1194959us) service_pool.cc:224] Handling call
0709 12:27:05.138057 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.138246 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:05.138273 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:05.138317 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:05.138351 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.138413 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.138523 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:05.140604 10941 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:05.141219 10941 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:05.044322  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:05.141779 10941 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:05.143384  5947 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:05.144554 10941 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:05.145035  5947 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 16. Ops: []
I20260709 12:27:05.145988 11211 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:05.146629  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1339) took 1041 ms. Trace:
I20260709 12:27:04.980280  5865 rpcz_store.cc:276] 0709 12:27:02.809938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.932094 (+1122156us) service_pool.cc:224] Handling call
0709 12:27:04.237725 (+305631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.273709 (+ 35984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.979579 (+705870us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b54002f99d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:04.979753 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":639475584}
I20260709 12:27:05.149077 10967 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:05.053336  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1400) took 1737 ms. Trace:
I20260709 12:27:04.973090 10823 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:05.018389 11155 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:04.911392  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:05.151440  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1764) took 1594 ms. Trace:
W20260709 12:27:05.152120  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 240833us
I20260709 12:27:05.152089  5874 rpcz_store.cc:276] 0709 12:27:03.557066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.513695 (+956629us) service_pool.cc:224] Handling call
0709 12:27:04.513988 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.987126 (+473138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.151160 (+164034us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b5400144bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:05.151373 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1085870848}
I20260709 12:27:05.084156  5944 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:05.153232  6609 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:05.154275 11069 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:27:05.084794  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 77510.2us
I20260709 12:27:05.154832 11174 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:05.155941  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:05.156388 11276 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:05.156885 11069 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:04.956947 11204 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:05.151482 11155 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:05.158197  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:05.158651 11255 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:05.094847 10996 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:05.091732 11131 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:04.894441 11281 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:05.099680  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 103094us
I20260709 12:27:05.160624 11131 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:05.161590 11131 raft_consensus.cc:3055] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:05.162114 11131 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:05.162161 11281 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:05.162439  6611 leader_election.cc:400] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19.
I20260709 12:27:05.163007  6611 leader_election.cc:403] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:05.163313 11131 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:05.058394 11176 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:04.952898 11246 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:05.105441  6766 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:27:05.198717  6766 pending_rounds.cc:85] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 7
I20260709 12:27:05.199018  6766 pending_rounds.cc:107] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:27:05.199471  6766 raft_consensus.cc:2889] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:05.081746 11265 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:05.080155 11227 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.141321  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1755) took 1767 ms. Trace:
I20260709 12:27:05.213088  5947 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:05.213269  5871 rpcz_store.cc:276] 0709 12:27:03.374354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.360757 (+986403us) service_pool.cc:224] Handling call
0709 12:27:04.361166 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:04.361589 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:27:04.361616 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:04.362124 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 12:27:04.745037 (+382913us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:04.745088 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.745168 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:05.141084 (+395916us) raft_consensus.cc:1685] finished
0709 12:27:05.141107 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.141253 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":428,"raft.queue_time_us":3169,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":7355,"thread_start_us":382555,"threads_started":1,"wal-append.queue_time_us":486754}
I20260709 12:27:05.023277 10954 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:05.150970  6793 rpcz_store.cc:276] 0709 12:27:03.315612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.206609 (+890997us) service_pool.cc:224] Handling call
0709 12:27:04.207023 (+   414us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:04.218502 (+ 11479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:04.218567 (+    65us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:04.247286 (+ 28719us) log.cc:844] Serialized 52 byte log entry
0709 12:27:04.697103 (+449817us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:04.706595 (+  9492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.706694 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:05.053028 (+346334us) raft_consensus.cc:1685] finished
0709 12:27:05.053056 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.053261 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10988,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":23038,"thread_start_us":353742,"threads_started":1,"wal-append.queue_time_us":513008}
I20260709 12:27:05.162358 11204 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:05.166281  5872 rpcz_store.cc:276] 0709 12:27:04.104944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.971635 (+866691us) service_pool.cc:224] Handling call
0709 12:27:05.080423 (+108788us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:05.081080 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.146414 (+ 65334us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54006db5d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:05.146555 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249338112}
I20260709 12:27:05.167060  6022 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:05.101039  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:05.167608  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:05.162931  5949 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:05.225734  5946 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.184823  5944 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []
W20260709 12:27:05.195770  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.202333  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 991) took 1390 ms. Trace:
I20260709 12:27:05.230078  5948 rpcz_store.cc:276] 0709 12:27:03.811577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.442147 (+630570us) service_pool.cc:224] Handling call
0709 12:27:04.456721 (+ 14574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.138545 (+681824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.201918 (+ 63373us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b540058fa54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:05.202239 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1566113024}
I20260709 12:27:05.167153 11131 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:05.233371  5943 raft_consensus.cc:2393] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:05.236333 10967 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:05.204810  5941 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:27:05.175509 11243 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:27:05.204836  5874 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:27:05.207319  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:05.097764 11147 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:05.210306  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1409) took 1822 ms. Trace:
W20260709 12:27:05.218991 11296 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:27:05.159125  5867 raft_consensus.cc:2410] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
W20260709 12:27:05.220054 10687 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.950s	user 0.015s	sys 0.000s
W20260709 12:27:05.214257  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.243633  5940 rpcz_store.cc:276] 0709 12:27:03.387745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.796995 (+409250us) service_pool.cc:224] Handling call
0709 12:27:04.239176 (+442181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.791283 (+552107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.175976 (+384693us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b5400602a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:05.209961 (+ 33985us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b5400602a50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:05.210213 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2220212352}
I20260709 12:27:05.246018  5948 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:05.250339  5872 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:05.221858 10954 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:05.166028 11201 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:05.199873 11252 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:27:05.293983 11276 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:05.221653 11227 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:05.298694  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:05.299940  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:05.256192  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1691) took 1595 ms. Trace:
I20260709 12:27:05.279105 11246 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:05.199700 11065 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:05.244503 11147 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:05.321862  5949 rpcz_store.cc:276] 0709 12:27:03.661320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.442674 (+781354us) service_pool.cc:224] Handling call
0709 12:27:04.453937 (+ 11263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.234398 (+780461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.255973 (+ 21575us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b5400633154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:05.256134 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1629927808}
I20260709 12:27:05.265050 11234 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:05.329391  5946 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:05.305222 11178 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:05.331923  6018 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:05.285359 11035 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:05.338691  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:05.301821 11265 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
W20260709 12:27:05.351591  5913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.010s	sys 0.029s
W20260709 12:27:05.361455  5913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.010s	sys 0.030s
W20260709 12:27:05.361603  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 303960us
I20260709 12:27:05.371546  6609 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:27:05.362207  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 562420us
I20260709 12:27:05.382068  5867 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:05.388197  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:05.389761  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:05.394605  5913 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:05.419428  5840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.001s	sys 0.031s
W20260709 12:27:05.420066  5840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.001s	sys 0.031s
W20260709 12:27:05.420344  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 520865us
I20260709 12:27:05.420823  5872 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:05.421062  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:05.422176  5913 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:05.427881  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:05.407673 10967 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:05.409582 11265 consensus_queue.cc:1243] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 12:27:05.427990 11224 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.552s	user 0.009s	sys 0.003s
I20260709 12:27:05.427924 11069 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:05.406215 10823 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:27:05.427806 11246 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:05.431272 10981 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:05.425503 11211 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:27:05.449576 10671 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.018s	sys 0.007s
I20260709 12:27:05.462100  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:05.433243 11220 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:05.455713 11275 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.545s	user 0.006s	sys 0.006s
W20260709 12:27:05.466982  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.451516 11065 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:05.429471 10783 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:27:05.468475  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:27:05.481765  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.454371 11204 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:27:05.483654 11178 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:05.483836  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 61792.6us
I20260709 12:27:05.429901 11260 consensus_queue.cc:1059] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:27:05.489531 10955 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:05.484678 10981 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:05.484685 11246 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:05.489601 11174 consensus_queue.cc:1059] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
W20260709 12:27:05.493119 10671 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.018s	sys 0.008s
W20260709 12:27:05.502279 11290 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 12:27:05.503109 11295 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:27:05.471899 11192 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:05.469688 11200 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.469466 10967 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:05.506441  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:05.508792  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:05.513536  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:05.479694 11208 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.010s	sys 0.015s
I20260709 12:27:05.486646 11201 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:05.544875  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:05.522945  5866 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:27:05.519251 11227 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:05.478638 11155 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:05.479702 11224 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:05.556043  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:05.557143  5944 raft_consensus.cc:2410] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:27:05.559598  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:05.560567  5947 raft_consensus.cc:2393] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:05.530308 11234 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:05.522645  5869 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.008s	sys 0.020s
W20260709 12:27:05.577380  5869 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.008s	sys 0.020s
I20260709 12:27:05.577857  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:05.546032  6608 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.029s	sys 0.026s
W20260709 12:27:05.578742  6608 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.029s	sys 0.026s
I20260709 12:27:05.524250 10981 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:05.553041  5949 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:05.510293 11275 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:05.586452 11192 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:05.588543 11275 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:05.588660 11061 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:05.589974  5946 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:05.590303  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1766) took 2004 ms. Trace:
I20260709 12:27:05.590819  5866 rpcz_store.cc:276] 0709 12:27:03.585611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.976288 (+1390677us) service_pool.cc:224] Handling call
0709 12:27:04.976661 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:05.551020 (+574359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.589995 (+ 38975us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54004f60d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:05.590233 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1251731200}
I20260709 12:27:05.575142  5940 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17. Ops: []
W20260709 12:27:05.545219 11208 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.011s	sys 0.015s
I20260709 12:27:05.592480  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1333) took 1489 ms. Trace:
I20260709 12:27:05.524372 11219 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
W20260709 12:27:05.538358  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.016s	sys 0.008s
I20260709 12:27:05.580992  5944 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:05.583750 11224 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:05.571671 11065 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:27:05.555444 11035 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:05.590018  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:05.604641  5948 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:05.593080  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:05.593956  5870 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:05.592932  5873 rpcz_store.cc:276] 0709 12:27:04.103026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.640285 (+537259us) service_pool.cc:224] Handling call
0709 12:27:04.706443 (+ 66158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.706693 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:05.082173 (+375480us) log.cc:844] Serialized 14 byte log entry
0709 12:27:05.592097 (+509924us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:05.592159 (+    62us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:05.592205 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.592294 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.592420 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30108,"prepare.queue_time_us":853338,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"thread_start_us":813056,"threads_started":2,"wal-append.queue_time_us":479269}
W20260709 12:27:05.594468  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.605576  5947 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
W20260709 12:27:05.596387  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.017s	sys 0.008s
W20260709 12:27:05.581686  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.585297 11265 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:05.610133  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 579485us
I20260709 12:27:05.611970 11260 raft_consensus.cc:3055] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:05.612592 11260 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:05.620745  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1346) took 1177 ms. Trace:
I20260709 12:27:05.621356  5870 rpcz_store.cc:276] 0709 12:27:04.443645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.979989 (+536344us) service_pool.cc:224] Handling call
0709 12:27:04.980569 (+   580us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.605852 (+625283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.620464 (+ 14612us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b540018a954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:05.620678 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1410127744}
I20260709 12:27:05.622028  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1335) took 1130 ms. Trace:
I20260709 12:27:05.622701  6016 rpcz_store.cc:276] 0709 12:27:04.491669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.491860 (+   191us) service_pool.cc:224] Handling call
0709 12:27:04.492360 (+   500us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:04.492833 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:27:04.492883 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:04.493559 (+   676us) log.cc:844] Serialized 52 byte log entry
0709 12:27:05.079973 (+586414us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:05.080018 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.080129 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:05.621758 (+541629us) raft_consensus.cc:1685] finished
0709 12:27:05.621781 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.621955 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":522,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":585901,"threads_started":1,"wal-append.queue_time_us":834594}
I20260709 12:27:05.595359 11156 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:05.585937 11201 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:27:05.554786 11200 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:05.624905 10824 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:05.640678  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:05.646523  5947 raft_consensus.cc:2410] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:27:05.642303  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1412) took 2253 ms. Trace:
I20260709 12:27:05.647576  5945 rpcz_store.cc:276] 0709 12:27:03.388688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.941126 (+552438us) service_pool.cc:224] Handling call
0709 12:27:03.974100 (+ 32974us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:03.984923 (+ 10823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.984974 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:04.021946 (+ 36972us) log.cc:844] Serialized 26 byte log entry
0709 12:27:04.274815 (+252869us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b18003b74c0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:27:04.770159 (+495344us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:04.770207 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.770288 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:05.629315 (+859027us) raft_consensus.cc:1685] finished
0709 12:27:05.629352 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.642183 (+ 12831us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14327,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":160,"spinlock_wait_cycles":8187264,"thread_start_us":495108,"threads_started":1,"wal-append.queue_time_us":1314661}
I20260709 12:27:05.594271 11299 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:05.677227  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1347) took 1185 ms. Trace:
I20260709 12:27:05.677356  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:05.678138  5946 raft_consensus.cc:2368] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16.
W20260709 12:27:05.687158  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 150541us
I20260709 12:27:05.667779 11299 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:05.681474  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1344) took 1257 ms. Trace:
I20260709 12:27:05.675598 11275 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.693475 11299 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:05.693825 10671 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:05.613602 11260 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:05.623760 11265 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:05.697455  5838 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:05.698410  5947 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:05.685499 10996 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:05.699107  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:05.702247  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 193554us
I20260709 12:27:05.709836  5840 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:05.702512  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.640077 11246 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:05.614352 10967 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:05.695144  5869 rpcz_store.cc:276] 0709 12:27:04.424449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.976921 (+552472us) service_pool.cc:224] Handling call
0709 12:27:04.977207 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.977467 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:27:05.577855 (+600388us) log.cc:844] Serialized 14 byte log entry
0709 12:27:05.681081 (+103226us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:05.681165 (+    84us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:05.681224 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.681298 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.681414 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":582825,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":647,"spinlock_wait_cycles":5212544,"thread_start_us":679327,"threads_started":2}
I20260709 12:27:05.716526  5840 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:05.716719  5838 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:05.709903  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:05.716414  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 995) took 1575 ms. Trace:
I20260709 12:27:05.720398  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1349) took 1227 ms. Trace:
I20260709 12:27:05.723409  5865 rpcz_store.cc:276] 0709 12:27:04.492679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.151786 (+659107us) service_pool.cc:224] Handling call
0709 12:27:05.152048 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.152251 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:05.596908 (+444657us) log.cc:844] Serialized 14 byte log entry
0709 12:27:05.719879 (+122971us) log.cc:844] Serialized 14 byte log entry
0709 12:27:05.720044 (+   165us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:05.720086 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:05.720141 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.720209 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.720332 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":410922,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"thread_start_us":566522,"threads_started":2}
I20260709 12:27:05.701478 11252 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:05.690294 10823 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:27:05.719336  5942 rpcz_store.cc:276] 0709 12:27:04.140774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.139930 (+999156us) service_pool.cc:224] Handling call
0709 12:27:05.140264 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.140486 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:05.530919 (+390433us) log.cc:844] Serialized 14 byte log entry
0709 12:27:05.715809 (+184890us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:05.715909 (+   100us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:05.715978 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.716074 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.716309 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13783,"prepare.queue_time_us":526895,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":51,"raft.queue_time_us":13971,"thread_start_us":559462,"threads_started":2}
I20260709 12:27:05.694196 11275 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:05.731427  5940 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:27:05.712168 11301 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:05.694293 10671 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:05.715394 11061 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:05.715405 11211 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:05.737907  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:05.699872 10954 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:05.714955 11201 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:27:05.718029 11208 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:27:05.712415 11147 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:05.801349  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.770946  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:05.770291  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:05.783036  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:05.795320  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:05.813511  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:05.807741  6762 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:27:05.808917  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:05.771080  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:05.815089  6775 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:05.816062  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:05.816537  5990 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.650s (SENT) [suppressed 11 similar messages]
I20260709 12:27:05.816859  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:05.810089  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:05.817364  5946 raft_consensus.cc:2368] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16.
I20260709 12:27:05.817859  6793 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.818750  5873 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:05.793910  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:05.780861  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:05.820185  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 160119us
I20260709 12:27:05.777908  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:05.746785  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:05.832026  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:05.815382  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:05.835338  6762 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:05.835444  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:05.722500 11178 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:05.820598  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:05.827351  6766 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:05.821570  5945 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:27:05.729357 11252 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:05.843801  5838 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:05.766726  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:05.820070  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:05.819270  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:05.820104  5869 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:05.823716  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:05.770761  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:05.848335  6773 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 18: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:27:05.755991 11131 consensus_meta.cc:287] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.002s
I20260709 12:27:05.747100 11220 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:05.845404  5949 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:27:05.851672  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:05.738328 11243 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19.
I20260709 12:27:05.810819 11318 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:05.854542 10954 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:05.737217 11069 raft_consensus.cc:3055] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:05.739437 11155 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:05.763049 10783 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.762624 11192 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:05.849364  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:05.809069 11211 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:05.850246  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:05.862943  6773 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:05.779891 11322 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:05.868080  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 151610us
I20260709 12:27:05.737002 11320 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:05.873620  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 60204.6us
W20260709 12:27:05.850651 11176 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.003s
I20260709 12:27:05.850978 11234 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:05.883965  5949 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:27:06.046648  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:05.927999 11192 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:05.825564 11224 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
W20260709 12:27:06.017194 11195 consensus_meta.cc:287] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.041s	user 0.021s	sys 0.002s
I20260709 12:27:05.861018 10954 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:05.878847 10671 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:05.829450 11215 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:27:05.870718 11200 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:05.859208 11156 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:05.850528 11180 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 12:27:06.057027 11301 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:06.012038  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:05.856061 11276 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:27:06.017690  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:05.853348 11255 consensus_queue.cc:1059] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
W20260709 12:27:05.883256 11306 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.185s	user 0.001s	sys 0.001s
I20260709 12:27:06.015175  5948 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:06.026770  5944 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:06.046537  6762 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:06.011457  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:06.017971  6609 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:05.884745 10783 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:06.066026  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.119210  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:06.120802  5946 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
W20260709 12:27:06.123719  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.867015 11069 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:06.068318 11291 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.179s	user 0.002s	sys 0.004s
I20260709 12:27:06.048610 10824 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:06.049088 11330 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:06.158640  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 306785us
I20260709 12:27:06.157200 11084 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:05.856683 11303 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:06.174819 11156 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:05.942384 11252 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:06.179888  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:06.218441  6773 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:06.219745  6773 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:06.020457 11322 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:06.197435  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 310480us
I20260709 12:27:06.224910  5987 leader_election.cc:400] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16.
I20260709 12:27:06.225404  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:06.226778  5840 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:06.187667 11333 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:06.049379 11320 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:06.211903  6768 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:06.122084 11176 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:06.176560 10671 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:06.200896  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:06.180976 11035 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:27:06.221179  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:06.190109  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 172222us
I20260709 12:27:06.151906 11234 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:06.241195 11333 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:06.250901 11234 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:06.174485  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 114732us
W20260709 12:27:06.199728 11317 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.422s	user 0.001s	sys 0.000s
W20260709 12:27:06.260455  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.161509 10954 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:06.200210 11327 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:06.200294 11276 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:27:06.218022 11325 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.368s	user 0.000s	sys 0.002s
I20260709 12:27:06.199421 11084 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:06.250995 11326 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.463s	user 0.005s	sys 0.000s
I20260709 12:27:06.256443  6018 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:27:06.257179  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:06.214781  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.677765  5868 rpcz_store.cc:276] 0709 12:27:04.492118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.080709 (+588591us) service_pool.cc:224] Handling call
0709 12:27:05.081059 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.081259 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:05.530315 (+449056us) log.cc:844] Serialized 14 byte log entry
0709 12:27:05.676858 (+146543us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:05.676916 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:05.676961 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.677030 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.677171 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":479263,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":28,"raft.queue_time_us":204,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":5872,"thread_start_us":571594,"threads_started":2}
I20260709 12:27:06.212273 11330 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:06.267740  6756 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:06.208557 11275 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:06.241938 11252 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:06.230173 11192 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:06.262689 11319 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.394s	user 0.000s	sys 0.001s
I20260709 12:27:06.304930 11322 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:06.281952 11339 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.303406 10954 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:06.305081 11341 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:06.251924 11333 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:06.360539 11339 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:06.288097 11332 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:06.362885  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:06.363219  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.342120  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:06.324728  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:06.242161 11147 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:06.315112  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:06.283452  6785 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 20: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:06.380367  6766 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:06.293965  6775 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:06.245718 11035 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:06.392593  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:06.210826 10824 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:06.302462 11340 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 12:27:06.284374 11291 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:27:06.319195 11335 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260709 12:27:06.310431 11328 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:27:06.288589 11276 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:06.435298  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:06.437294  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:06.357807 11262 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:06.312193 11155 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:06.333141 11260 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:06.447655  5867 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:27:06.219669 11303 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:06.427697  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 156285us
W20260709 12:27:06.464258 11177 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.289s	user 0.012s	sys 0.004s
I20260709 12:27:06.345916 11195 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.113s
I20260709 12:27:06.434300 11147 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:27:06.475412  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:06.475883  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:06.481050  5869 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:06.415856 11333 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:06.471735 10823 raft_consensus.cc:2764] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:27:06.447579  6793 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.629s	user 0.014s	sys 0.000s
I20260709 12:27:06.541709  6793 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:06.353960 11200 consensus_queue.cc:1059] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 12:27:06.487303  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:06.468441  6766 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:06.526198  5870 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:06.258805 11069 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:06.530493  5990 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.011s	sys 0.014s
W20260709 12:27:06.546375  5990 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.011s	sys 0.014s
W20260709 12:27:06.546837  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 730220us
I20260709 12:27:06.458149 11262 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:06.550940  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:06.535957  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.014s	sys 0.010s
W20260709 12:27:06.553145  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.014s	sys 0.011s
I20260709 12:27:06.421262 11035 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:06.487308 11325 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:27:06.538408  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:06.509720 10981 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.924s	user 0.017s	sys 0.003s
I20260709 12:27:06.459022 10783 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:06.478763 11155 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:06.445700 11338 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:06.534317  6768 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:06.547857  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:06.379297 11301 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:06.545153 11291 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:27:06.535085 11336 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:06.569535  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:06.568449  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 339149us
I20260709 12:27:06.421005 11332 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:06.571039  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:06.572467  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:06.600137  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:06.578338  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:06.601460  6018 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:06.579919  6024 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:27:06.557713 11069 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:06.585741  6766 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:06.531494 11177 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:06.587805  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:06.588696  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:06.590049  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.471292 11084 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:06.599300  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:06.576143  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 289427us
I20260709 12:27:06.603885  6762 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 12:27:06.607546  6766 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: [16.10-17.11]
I20260709 12:27:06.619839  6020 raft_consensus.cc:2393] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 14: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:06.621883  5840 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:06.608958  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44631.3us
I20260709 12:27:06.570868 11275 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.570997 11330 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:06.621309  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:06.575610 10941 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:06.633001 11334 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 12:27:06.634028  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:06.634594 11195 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:06.637768 11355 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.630352  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:06.626902  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:06.639225 11177 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:06.640288 10783 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:06.641016 10783 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:06.642524 11265 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:06.628664  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:06.633744 11275 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:06.646196  6017 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:06.608763 11336 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:06.638454 11355 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:06.554515 11276 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:06.648289 11276 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:06.642019 10783 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:06.654611 10824 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:06.655040 11208 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:06.625979  6793 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:27:06.573297 11338 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:06.600847 10996 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.889s	user 0.013s	sys 0.002s
I20260709 12:27:06.640128 11177 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:06.660238  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:06.655681 11320 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:06.575223 10981 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:06.665572  6793 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:06.666069  6793 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:06.669490  6768 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:06.660843 11338 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:06.673764 11298 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:06.674551 11303 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:06.674896 11275 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:06.663446 10824 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:06.660809 11155 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:06.676331 11155 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:06.676718 11355 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:06.677315 11355 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:06.663992 10996 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:06.677829 11355 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:06.678128 11299 consensus_meta.cc:287] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.984s	user 0.015s	sys 0.000s
I20260709 12:27:06.624969 11323 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:06.641036 11084 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:06.728484 11332 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:06.655767 11208 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:06.732316 10824 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.732838 11355 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:06.735107 11224 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:06.735702 11224 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:06.738003  5868 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:06.739488 11336 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:06.699656 11339 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:06.700994  6793 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:06.672044 11176 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:06.646229 11180 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:27:06.733094 10824 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:06.619369 11262 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:06.736824 11224 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:06.776127  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.789072 11255 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:06.794288 11346 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:06.795042 11346 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:06.797175 11298 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:27:06.797987  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 197857us
I20260709 12:27:06.797940 11298 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:06.799894 11131 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:06.680325 11265 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 12:27:06.806308 11227 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:06.806999 11227 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:06.810599  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 1236) took 1040 ms. Trace:
I20260709 12:27:06.812635  6757 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:06.683308  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.811244  6785 rpcz_store.cc:276] 0709 12:27:05.769816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.769948 (+   132us) service_pool.cc:224] Handling call
0709 12:27:06.810270 (+1040322us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b5400003f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:06.810506 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1041720064}
I20260709 12:27:06.800099  5987 leader_election.cc:400] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16.
I20260709 12:27:06.829597  5987 leader_election.cc:403] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:27:06.790073  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:06.840296  6768 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:06.801440 11265 raft_consensus.cc:3055] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:27:06.789336 11352 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:27:06.850736 11265 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:06.704963 11299 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:06.856506  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1710) took 1133 ms. Trace:
W20260709 12:27:06.856285  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 56356.2us
I20260709 12:27:06.858952  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:06.858088  5947 rpcz_store.cc:276] 0709 12:27:05.723287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.733830 (+ 10543us) service_pool.cc:224] Handling call
0709 12:27:05.820296 (+ 86466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:05.826202 (+  5906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:05.826247 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:05.826733 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 12:27:06.444608 (+617875us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b18004037e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:27:06.580989 (+136381us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:06.581600 (+   611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.581830 (+   230us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:06.856176 (+274346us) raft_consensus.cc:1685] finished
0709 12:27:06.856198 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.856436 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4509,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":7669,"spinlock_wait_cycles":24648448,"thread_start_us":136135,"threads_started":1,"wal-append.queue_time_us":949655}
I20260709 12:27:06.858748  6793 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:06.862650  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:06.865651  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.869562  5945 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:06.869930  5944 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:27:06.871193  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:06.871999  5941 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:27:06.874176  5945 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
W20260709 12:27:06.881011  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 257870us
I20260709 12:27:06.882292  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:06.899686  6768 raft_consensus.cc:2368] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 16.
I20260709 12:27:06.900499  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:06.882714  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:06.902462  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.891386  6018 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:06.658223 10941 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:06.890863 11201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.034s	user 0.004s	sys 0.015s
I20260709 12:27:06.906013  5840 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:06.930994 11368 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:06.931150 11265 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:06.932371 11360 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:06.932931 11360 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:06.935173 11360 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:06.935707 11360 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.936138 11360 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:06.910337  5946 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:27:06.873284 10967 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:06.939352 10824 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:06.940377 10671 consensus_meta.cc:287] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.687s	user 0.017s	sys 0.000s
I20260709 12:27:06.859655 11224 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:06.920019  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:06.941494  6785 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:06.833644 11255 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:06.922923 11208 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:06.952144  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:06.925283  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:06.952633  6766 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:27:06.928369 11195 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:06.929994 11201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.080s	user 0.005s	sys 0.015s
W20260709 12:27:06.938199 11252 consensus_meta.cc:287] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.555s	user 0.011s	sys 0.011s
I20260709 12:27:06.940076 11362 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:06.946795  5868 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:06.922006  6775 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: [14.8-16.9]
I20260709 12:27:06.944285  6015 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
W20260709 12:27:06.949872  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 281519us
I20260709 12:27:06.923393 11211 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:06.925024 11227 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:06.905640  6609 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:06.980559  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:06.983471  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:06.983918  5866 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:06.955351  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:06.956429  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:06.958045  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:06.986651  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:06.976915  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:06.987439  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 82103.5us
I20260709 12:27:06.963821  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:06.988777  5874 raft_consensus.cc:2393] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:06.896637 11147 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:06.893507 11361 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:06.972445  5947 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:06.908599 11180 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16.
I20260709 12:27:06.980559  6773 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:27:07.003909  6773 pending_rounds.cc:85] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 10
I20260709 12:27:07.004225  6773 pending_rounds.cc:107] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:27:07.004454  6773 raft_consensus.cc:2889] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:06.955055  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:06.987739  5865 raft_consensus.cc:2410] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 19 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:27:06.959690  6793 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:27:07.004782  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:07.005264  6793 pending_rounds.cc:85] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 9
I20260709 12:27:07.005597  6793 pending_rounds.cc:107] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:27:07.005956  6793 raft_consensus.cc:2889] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:06.985351  5869 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:27:06.991513  5872 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:06.994846  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:07.000946  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:07.002274  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.009196  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:07.013392  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 115731us
I20260709 12:27:07.014267  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:07.015202  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1780) took 1227 ms. Trace:
I20260709 12:27:07.016834  5915 leader_election.cc:400] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 16.
I20260709 12:27:07.015868  5871 rpcz_store.cc:276] 0709 12:27:05.787764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.787889 (+   125us) service_pool.cc:224] Handling call
0709 12:27:05.794468 (+  6579us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:05.815777 (+ 21309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:05.815814 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:05.823313 (+  7499us) log.cc:844] Serialized 52 byte log entry
0709 12:27:06.377371 (+554058us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b18004107a0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:27:06.570484 (+193113us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:06.570529 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.570601 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.014710 (+444109us) raft_consensus.cc:1685] finished
0709 12:27:07.014748 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.015079 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20758,"raft.queue_time_us":67685,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":5098,"spinlock_wait_cycles":5319040,"thread_start_us":192858,"threads_started":1,"wal-append.queue_time_us":1037208}
I20260709 12:27:07.018256  5915 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:07.018597 10824 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:07.019449 11358 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:06.997732 11336 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:07.020345 11336 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.020889 11336 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:07.021724  5915 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:07.024917  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:07.033120 11371 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:07.033252  5990 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:07.035413 11370 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:07.033213 11377 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:07.039198  5865 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:07.042018  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:07.043143  5865 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:27:07.043186 11308 raft_consensus.cc:3055] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:07.044118 11308 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:07.002072 11211 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:07.005501 11227 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:07.068470  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:07.069557  5871 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:07.003768 11131 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:07.079957  6768 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:07.006763 11065 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:07.080385  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:07.041234 11377 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:07.045385 11308 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:07.007539 11361 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:27:07.087833  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 78694.2us
I20260709 12:27:07.048508 11350 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:07.002511 11147 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:07.053263 11346 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:07.113710  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:07.102104 11065 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:07.121203 11350 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:27:07.125459  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.104753 11377 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:07.080247 10981 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:07.080646 11358 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:27:07.134080 11178 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.287s	user 0.000s	sys 0.025s
I20260709 12:27:07.100633 11131 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:07.120775 11356 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.086045 11343 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:07.116889  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:07.012581 11347 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 12:27:07.036355 11370 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:27:07.204267  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 168486us
I20260709 12:27:07.034379 11371 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:06.981439 11341 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 12:27:07.113849 11298 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:07.213143  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 99537.1us
W20260709 12:27:07.233119  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 116357us
I20260709 12:27:07.218098  5871 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:27:07.197829 11337 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:07.210814 11350 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:07.108071 11366 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.119s
W20260709 12:27:07.214700 11338 consensus_meta.cc:287] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.540s	user 0.024s	sys 0.000s
I20260709 12:27:07.235167  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:07.216251 11368 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:07.233695 10823 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.246794 11131 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:07.255453  5840 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:07.218240 11345 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:07.249151  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1413) took 1379 ms. Trace:
I20260709 12:27:07.193890 11359 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.126s
I20260709 12:27:07.218255 11178 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:07.259565  6757 rpcz_store.cc:276] 0709 12:27:05.869442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.872962 (+  3520us) service_pool.cc:224] Handling call
0709 12:27:06.812324 (+939362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:06.813241 (+   917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.248909 (+435668us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b5400003f50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:07.249074 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2159398656}
I20260709 12:27:07.234234 11299 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:07.282303 11381 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:27:07.286048 10941 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:07.266010 11361 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:07.248971 11338 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:07.296725 11338 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:07.254662 10981 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:07.261369 10823 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:07.302171 10823 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:07.305106 11156 consensus_meta.cc:287] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.075s	user 0.011s	sys 0.001s
I20260709 12:27:07.224007 11356 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:07.307698 11368 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:07.308019 11065 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:07.266893 11255 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:07.267993 11346 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:07.321485 11260 consensus_meta.cc:287] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.811s	user 0.014s	sys 0.012s
W20260709 12:27:07.331648 11069 consensus_meta.cc:287] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.707s	user 0.033s	sys 0.016s
W20260709 12:27:07.330170 11367 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.318s	user 0.000s	sys 0.005s
I20260709 12:27:07.239817 11343 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:07.318143 11345 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:07.261870 11362 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:27:07.324112 11301 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:07.363552  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:07.355139 11035 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.785s	user 0.019s	sys 0.007s
I20260709 12:27:07.361145 11366 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:07.374070 11260 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:07.374953 11084 consensus_meta.cc:287] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.646s	user 0.013s	sys 0.016s
I20260709 12:27:07.375340 11343 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:07.376438 11276 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.728s	user 0.027s	sys 0.011s
I20260709 12:27:07.383512 11356 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:07.352744 11201 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:07.384526 11201 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:07.325026 11156 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:07.361171 10981 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.386907 11201 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:07.386547 10981 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:07.387368 11156 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:07.389276 10981 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:07.379632 11035 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:07.355966 11368 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:07.403753  6756 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
W20260709 12:27:07.408577  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:07.409202  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:07.402007 11155 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.725s	user 0.017s	sys 0.007s
I20260709 12:27:07.377804 11301 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:07.390528 11260 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:07.390875 11366 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:07.413195 11350 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:07.427996 11371 consensus_queue.cc:1243] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:07.426363 11345 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:07.444509 11366 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:07.441545 11368 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:07.451776 11350 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:07.456027 11377 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:07.506294  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 269357us
I20260709 12:27:07.505686  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:07.510744  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:07.513010  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:07.516260  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.518322 10783 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:27:07.516314  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.519980 10783 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:07.531484 11255 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.533483 11275 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:07.531642 11260 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:07.536307 11276 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:07.456019 11336 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:07.538398 11377 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:07.539887 11343 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:07.533011 11255 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:07.534085 11275 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:07.551925 11035 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:07.552933  6017 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:07.555316 11361 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:07.555922 11361 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:07.556224  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:07.557155 11255 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:07.552826 11336 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:07.557158  6017 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:07.535429 11224 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:07.562453 11084 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.574821 11084 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:07.559435 10783 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:07.577715  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:07.557499 11361 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:07.584944 11220 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:27:07.589452  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 55792.4us
I20260709 12:27:07.561033 11275 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:07.593355 11386 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:07.594568  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:07.594882  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1421) took 1404 ms. Trace:
W20260709 12:27:07.596149 10954 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:27:07.596338 11275 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:07.600071 11211 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.600641 11147 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:07.604599  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:07.585597 11220 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:07.582011 11276 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:07.610633 11084 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:07.611517  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:07.596530 11065 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.613776 11065 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:07.618638 11346 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:07.619238 11346 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.619642 11346 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:07.625447  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.627255  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:07.632804 11224 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:07.607722 11276 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:07.617369  5990 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:07.595683  6773 rpcz_store.cc:276] 0709 12:27:06.190085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.220679 (+ 30594us) service_pool.cc:224] Handling call
0709 12:27:06.220955 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:07.004817 (+783862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.594068 (+589251us) spinlock_profiling.cc:243] Waited 759 ms on lock 0x7b54002a0154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:07.594575 (+   507us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1745952640}
W20260709 12:27:07.642298  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.643419 11360 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:07.645752 11262 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:07.653143  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:07.653714  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:07.654624  5943 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:27:07.655040  5946 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:07.656162  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:07.657653  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:07.658361  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:07.658450  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:07.658718  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:07.654013 11370 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:07.657483  5943 raft_consensus.cc:2410] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:27:07.661939 11370 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:07.663925 11370 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:07.666463  5949 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:27:07.658672  5940 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:07.659770  5873 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:07.669358 11276 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:07.670887  5943 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:07.665417 11370 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:07.672377  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 45521.9us
I20260709 12:27:07.601061 11211 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:07.683689  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:07.688282 11338 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:07.688997 11338 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:07.691237 11338 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:07.692737 11303 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:07.693074 11338 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:07.718840  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:07.719985  5871 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:07.695825 11220 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:07.698113 11361 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:07.630301 11308 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:07.699288  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:07.700186  6015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 1340) took 1865 ms (client timeout 2215 ms). Trace:
I20260709 12:27:07.701921 11265 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:07.728823  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:07.731292  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:07.732177  5944 raft_consensus.cc:2410] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 12:27:07.730962  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 371127us
I20260709 12:27:07.669500 11301 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:07.704031 11215 raft_consensus.cc:3055] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:27:07.704399  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 87241.9us
I20260709 12:27:07.741845 11215 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:07.743188 11388 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:27:07.743933 11385 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.306s	user 0.003s	sys 0.000s
I20260709 12:27:07.744480  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:07.705188 11376 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:07.744524 11343 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:07.743690 11215 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:07.695849 11061 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.747911 11215 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:07.729825  6015 rpcz_store.cc:269] 0709 12:27:05.834872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.835215 (+   343us) service_pool.cc:224] Handling call
0709 12:27:07.699447 (+1864232us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54006b99d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:07.699844 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2548348160}
I20260709 12:27:07.738021  5941 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:07.738899 11180 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:07.740840  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:07.710227 11224 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:07.751772  5942 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:07.723340  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:07.749847  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:07.753170  5948 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:27:07.753388  5871 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:27:07.752975 11343 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:07.747716 11061 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:07.755604 11386 raft_consensus.cc:3055] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:07.756207 11386 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:07.749922  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:07.751070  6016 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:07.758203 11386 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:07.758075  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:07.764423  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:07.765813  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:07.766989  6757 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 19: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:07.766767  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:07.767691 11200 raft_consensus.cc:3055] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:07.768144 11200 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:07.768602  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:07.764631  6778 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:27:07.741340  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:07.769604  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:07.770884  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:07.799115  5865 raft_consensus.cc:2393] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:27:07.799983 11227 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.608s	user 0.017s	sys 0.007s
I20260709 12:27:07.802057  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:07.802227  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:07.803253  6757 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:27:07.804659 11211 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:07.804983  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:07.774173  5945 raft_consensus.cc:2410] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:27:07.806085  6778 raft_consensus.cc:2376] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15. Re-sending same reply.
I20260709 12:27:07.807161 11200 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:07.808712 11200 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:07.808828  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:07.810477  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:07.811055  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.810781  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:07.814467  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:07.815107  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.816097 11234 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.816812  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:07.817436 11069 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:07.818011  5868 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 14: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:07.818194 11069 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:07.783171 11065 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:07.819399 11065 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.820003 11065 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:07.820730 11069 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:07.820811  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:07.822888 11065 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:07.789044 11061 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:07.840183  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:07.841202  6773 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:07.777011  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:07.842728 11349 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:07.843205 11349 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:07.843575 11246 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:07.843700  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 66882us
I20260709 12:27:07.844316 11246 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:07.844806  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:07.843595 11349 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:07.847198 11303 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:07.847708 11303 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.848071 11303 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:07.849490  6609 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:07.850564 11220 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:07.780609 11386 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:07.789485  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:07.782922 11276 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:07.861593  5944 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:07.790153  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:07.793371  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:07.807763  6768 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:07.794137  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:07.816814 11234 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:07.787735  6756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1085) took 1521 ms (client timeout 1814 ms). Trace:
I20260709 12:27:07.794512 10671 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:07.765836  6762 raft_consensus.cc:2393] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:27:07.791056  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 50101.2us
I20260709 12:27:07.839341  6018 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:27:07.856585  5940 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:27:07.792610  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:07.824108  6022 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:07.853236 11377 raft_consensus.cc:3055] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:07.864727  6018 pending_rounds.cc:85] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 7
W20260709 12:27:07.871608  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:07.863749  6756 rpcz_store.cc:269] 0709 12:27:06.266395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.392157 (+125762us) service_pool.cc:224] Handling call
0709 12:27:07.787200 (+1395043us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b5400006254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:07.787463 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2149917056}
I20260709 12:27:07.873328  5869 raft_consensus.cc:2393] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:07.876035  5868 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:07.878507 11195 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.879307 11195 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:07.870738  6018 pending_rounds.cc:107] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:27:07.885366  6018 raft_consensus.cc:2889] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:07.883766  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:07.866784 10954 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:07.907728  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.902683  6773 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:27:07.870695 11377 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:07.897322  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:07.915187  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:07.919677 11389 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.459s	user 0.003s	sys 0.000s
I20260709 12:27:07.919806  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:07.922057  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:07.923225  5872 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:07.912443  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:07.925689  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 76285.2us
I20260709 12:27:07.887920 11065 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:07.905380 11195 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:07.940840  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:07.938220  5865 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:07.944568  5867 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:07.923208 11234 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:07.951061  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:07.952200  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:07.953068  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.954007  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:07.956261  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:07.958487  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:07.959821  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:07.966101  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:07.966382  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:07.967339  6757 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:07.967437  6021 raft_consensus.cc:2410] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 17 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:27:07.968477  6778 pending_rounds.cc:85] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 11
I20260709 12:27:07.972527  6778 pending_rounds.cc:107] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:27:07.972812  6778 raft_consensus.cc:2889] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:07.976259  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:07.977828  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 50258.1us
I20260709 12:27:07.981187 11234 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.981778  6016 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:27:07.982107 11234 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:07.984998 11234 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:07.949874 11246 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:07.957370 11195 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:07.991611  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.988198  5869 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:07.996335  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:08.002293  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:08.005524  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:07.956163 11065 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:08.010757  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:08.014168  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:08.014647  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:08.015349  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:08.032868  6017 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:08.016031  5870 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:27:08.005283 11361 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.002825  6018 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
W20260709 12:27:08.023010 11394 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:27:07.938140  6762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
W20260709 12:27:08.038585  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 94882.9us
I20260709 12:27:08.037777  6762 raft_consensus.cc:606] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:27:08.040350 11361 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:08.055459  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.065965  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.065999  6785 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:08.052156  6775 raft_consensus.cc:2410] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:27:08.078228  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:08.080017  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:08.081698  5838 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:08.081940  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:08.082911  6021 raft_consensus.cc:2410] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:27:08.083248 11360 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:08.084918  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:08.085142  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:08.083341  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:08.085661  6015 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:27:08.086465 11211 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:08.086946 11211 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:08.087106  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:08.087255  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:08.087405 11211 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:08.088200 11227 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:08.088696  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:08.089076 11227 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:08.089951  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:08.090937  6021 raft_consensus.cc:2410] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:27:08.091950 11360 consensus_queue.cc:1059] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:08.093309 11360 raft_consensus.cc:3055] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:08.093794  6773 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:08.094309  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:08.095348 11227 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:08.097426 11360 raft_consensus.cc:740] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:08.098815 11360 consensus_queue.cc:260] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:08.049569 11361 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:08.059943 11065 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:08.102622 11227 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:08.104027 10671 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.104640 10671 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:08.107208 10671 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:08.100788 11360 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:08.110580  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:08.111781 11361 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:08.112208 11361 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.112565 11361 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:08.112964  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:08.113903  6768 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:08.117488  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:08.118589  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:08.118963  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:08.122360  6024 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:27:08.124157  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 110132us
I20260709 12:27:08.000067 11195 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:08.126611  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:08.051263  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:08.081511  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:08.138445  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 87401.9us
I20260709 12:27:08.142230  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:08.101104 11065 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:08.145285  5867 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:27:08.146518  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:08.152383  6793 raft_consensus.cc:2393] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 17: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:08.152797  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:08.144647 11246 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:08.159343  6775 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:08.166165  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:08.159879  6762 raft_consensus.cc:2376] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16. Re-sending same reply.
I20260709 12:27:08.168869  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 1250) took 1218 ms. Trace:
I20260709 12:27:08.169584  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:08.169984 11156 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:08.173138  6785 rpcz_store.cc:276] 0709 12:27:06.950635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.950829 (+   194us) service_pool.cc:224] Handling call
0709 12:27:08.168585 (+1217756us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b5400005854. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:08.168784 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2112229632}
I20260709 12:27:08.007082 11377 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:08.183039 11349 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:08.194146  6766 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 14: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:08.199553  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:08.202546  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:08.167627 11065 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:08.208604  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:08.209540 11356 consensus_queue.cc:1059] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:27:08.171976 11246 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:08.214960  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 133580us
I20260709 12:27:08.218276  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:08.219612  6017 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:27:08.246831  6021 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 12: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:08.220850  6015 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:27:08.247999  6015 pending_rounds.cc:85] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 10
I20260709 12:27:08.248286  6015 pending_rounds.cc:107] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:27:08.248544  6015 raft_consensus.cc:2889] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:08.225979 11178 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:08.249817  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 69048.9us
W20260709 12:27:08.250435  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.190734 11349 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:08.251397 11349 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:27:08.253304 11262 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.607s	user 0.012s	sys 0.002s
I20260709 12:27:08.228188  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:08.228617  6773 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
W20260709 12:27:08.243304  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 325241us
I20260709 12:27:08.227442 11195 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:08.253830  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.254302  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:08.255146 11358 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:08.255780  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:08.257464  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:08.273459  6766 raft_consensus.cc:2410] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 12:27:08.244772  6775 raft_consensus.cc:2376] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17. Re-sending same reply.
I20260709 12:27:08.275475  5940 raft_consensus.cc:2410] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
W20260709 12:27:08.278928  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.280635  6768 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:08.265110 11361 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:08.287352  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:08.287837  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:08.288743  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:08.291956 11195 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.292850  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:08.295271  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:08.296264 11377 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:08.296914 11199 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:08.266736 11246 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:08.297904  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:08.280077 11358 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.299149 11156 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:08.299335 11178 raft_consensus.cc:3055] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:08.312889  6785 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:08.298455  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:08.320708  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:08.322414  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:08.361174  5947 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:08.325594  5944 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:08.326193 11262 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.327463 11361 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:08.329948  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:08.335812  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:08.328188 11195 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:08.345860  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:08.346318 11303 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:08.318215 11343 consensus_queue.cc:1059] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 12:27:08.347962  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.349368  6793 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:08.362394  6766 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:27:08.285898 11227 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:08.362821  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 63238.8us
I20260709 12:27:08.370906  6016 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
W20260709 12:27:08.411489  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 51329.9us
I20260709 12:27:08.414767 11156 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:08.415464  6762 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:08.414155  6775 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:08.314671 11199 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.423172  6773 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:08.425817 11262 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:08.416028 11061 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:27:08.428164 11200 raft_consensus.cc:993] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: : Instructing follower 6add331326ca47038308a447bf798fa7 to start an election
I20260709 12:27:08.444042  6024 raft_consensus.cc:2410] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:27:08.447224  6015 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:08.448401  6023 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:08.362790 11178 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:08.359642 11358 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:08.462242  6757 raft_consensus.cc:2368] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21.
I20260709 12:27:08.314286 11356 consensus_queue.cc:1059] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.139s
I20260709 12:27:08.447646 11199 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:08.464293  5838 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:08.441907 11220 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.389616 11069 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:08.473949  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:08.462951 11200 raft_consensus.cc:1081] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Signalling peer 6add331326ca47038308a447bf798fa7 to start an election
W20260709 12:27:08.466609 11346 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.846s	user 0.007s	sys 0.017s
W20260709 12:27:08.511593  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 180942us
I20260709 12:27:08.512084  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:08.524814  6785 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:27:08.510488  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:08.534343  5944 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
W20260709 12:27:08.535041 11400 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:27:08.537817 11402 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:27:08.547765 11370 consensus_queue.cc:1059] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:08.549882  6762 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 12:27:08.555686  5943 raft_consensus.cc:2410] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
W20260709 12:27:08.557869  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 358527us
I20260709 12:27:08.462540 11227 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:08.508549 11343 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:08.500922 11156 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:08.580658  5948 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:08.528261 11349 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.585851  6773 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:27:08.593436 11338 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.875s	user 0.004s	sys 0.014s
W20260709 12:27:08.593838  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 80689.6us
I20260709 12:27:08.602528 11195 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:08.587420 11069 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:08.508051 11346 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:08.626083  5945 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:27:08.625926 11361 raft_consensus.cc:3055] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:08.515959 11220 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:08.629060  5692 catalog_manager.cc:5697] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 6 to 13, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 13 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:27:08.473014 11065 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:08.645352  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 73248.1us
W20260709 12:27:08.646945  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 79585.9us
I20260709 12:27:08.573755 11343 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.606036 11246 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:08.638098 11361 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:08.657007 11220 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:08.654585  5913 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:08.665024  6020 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 21: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:08.590480 11349 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:08.658509 11343 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:08.696682  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.696147  5947 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
W20260709 12:27:08.731034  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.734699  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 263584us
I20260709 12:27:08.712519 11303 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:08.709688 11211 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.622s	user 0.013s	sys 0.003s
W20260709 12:27:08.747803  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 287967us
I20260709 12:27:08.752030  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:08.736447  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.757143  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 209938us
I20260709 12:27:08.642338 11178 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:08.756588  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:08.757524 11262 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:08.777029  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.777614  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.775375  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 128946us
I20260709 12:27:08.767563 11265 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:08.785905  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:08.790378  5867 raft_consensus.cc:2410] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:27:08.793972  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1288) took 1071 ms. Trace:
I20260709 12:27:08.794461  6020 rpcz_store.cc:276] 0709 12:27:07.722452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.722690 (+   238us) service_pool.cc:224] Handling call
0709 12:27:08.793776 (+1071086us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b5400363dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:08.793912 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2096590592}
I20260709 12:27:08.769301 11156 raft_consensus.cc:697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 12 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:08.800007  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:08.763316 11211 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:08.796550  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:08.804076 11220 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:08.802790  5870 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:08.805269  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:08.807482  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:08.800644  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:08.812572  5873 raft_consensus.cc:2410] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:27:08.774870 11386 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:27:08.820833  5871 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:08.825500  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:08.826258  5944 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:27:08.766569 11361 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:08.825145  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:08.796321 11410 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:08.836436  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 53857.6us
I20260709 12:27:08.832888  5867 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:27:08.839641  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:08.834333  5865 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:08.838215  5913 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:08.820333 11227 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:08.852837  5946 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:27:08.853384  5946 pending_rounds.cc:85] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 8
I20260709 12:27:08.847179  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:08.845939  5941 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:08.855432  5946 pending_rounds.cc:107] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:27:08.855849  5946 raft_consensus.cc:2889] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:08.799885 11195 consensus_queue.cc:1059] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:27:08.856791  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:08.854534  5948 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
W20260709 12:27:08.857934  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 83867.2us
I20260709 12:27:08.841019 11414 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:08.864910  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:08.868019  5867 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:08.845605 11346 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:08.866881  5838 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:08.879097  5838 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:08.872098  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:08.872959  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:08.880359  6773 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 21: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:08.856590 11368 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.884157  5872 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:08.843353 11338 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:27:08.867295 11069 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:08.864337 11262 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.903906  6756 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:08.878021 11414 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:08.922237  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 165762us
I20260709 12:27:08.895103 11061 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:08.884565 11361 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:08.846020 11410 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:08.930331  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:08.895016 11246 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:08.932188  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:08.902048 11220 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:08.948900 11220 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.949389 11220 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:08.950457 11423 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:08.951932 11427 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:08.952270 11426 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:08.952564 11427 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.953104 11427 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:08.953860 11200 raft_consensus.cc:2764] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:27:08.954856  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:08.955945  5946 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:08.956122  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:08.935786  5840 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:08.946123  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:08.956996  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:08.962432  5946 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:27:08.963730  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:08.932667 11061 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.972884  5944 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:08.932363 11414 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:08.977931  5947 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:08.939889 11411 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.993686  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:08.923183 11195 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
W20260709 12:27:08.971329 11406 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 12:27:08.992372  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:09.011361  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:08.936267 11234 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:27:08.985039 11061 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:08.929388 11368 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:08.917567 11156 consensus_queue.cc:237] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:08.938491 11262 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:09.021474 11408 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.225s	user 0.002s	sys 0.000s
I20260709 12:27:09.014395 11227 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.039727 11227 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.016592  5945 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 21: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:09.029819 11211 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:09.034037  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 102080us
I20260709 12:27:09.037716  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:09.018270  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:09.034824  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 61285.7us
I20260709 12:27:08.999454 11411 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:09.021693 11417 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.051987 11417 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.052528 11341 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:09.053014 11341 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:09.036358 11234 consensus_queue.cc:1243] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:09.065402  5944 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
W20260709 12:27:09.067884  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 134746us
W20260709 12:27:09.073151  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:09.027582 11422 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:27:09.068886  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:09.010069 11356 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:09.026603 11412 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:27:09.092008  5940 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:27:09.093060  6757 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:09.094172  6757 raft_consensus.cc:606] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:09.094687  6757 raft_consensus.cc:614] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:09.005879 11421 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.113s
I20260709 12:27:09.105942  6020 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79"
dest_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:34888
I20260709 12:27:09.106523  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:09.106876  6020 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:09.111078  6020 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:09.114068 11275 raft_consensus.cc:3055] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:09.114568 11227 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:09.115645 11428 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.115832  5948 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:09.116074  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:09.117345 11418 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:27:09.118506  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 126200us
I20260709 12:27:09.120406  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:09.106503  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:09.114471 11356 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.122911  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:09.127691  6021 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 21: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:09.055660 11420 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:09.131258 11434 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:09.131438 11368 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:09.131860  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:09.132464  6778 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:09.133236 11436 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:09.123972 11275 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:09.116220 11428 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.135854 11262 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:09.136451  5990 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:09.095716 11211 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.067500 11338 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:09.143316 11377 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.824s	user 0.013s	sys 0.003s
I20260709 12:27:09.153090  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:09.182096  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:09.183413  5940 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:09.141964 11420 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.189829  5946 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:09.194394  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:09.142899 11434 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:27:09.199432  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 76666.5us
I20260709 12:27:09.200134  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:09.201432  5947 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
W20260709 12:27:09.202036  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 49159.7us
I20260709 12:27:09.195529  5865 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:09.203517  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:09.214566  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:09.215145  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 78794.3us
I20260709 12:27:09.217129  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:09.200318 11343 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.500s	user 0.014s	sys 0.000s
I20260709 12:27:09.197463 11420 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:09.239562  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 170757us
W20260709 12:27:09.220669 11199 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.741s	user 0.016s	sys 0.001s
I20260709 12:27:09.196174 11417 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:09.244637  5940 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:09.213115 11368 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:09.246282  5840 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:09.246580  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:09.219992 11415 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:09.218684 11377 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:09.203644 11434 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:09.243515  6778 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "428a50885681465a98089cb853b233b9"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:09.179677 11211 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:09.231572 11436 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:09.145339 11341 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:09.187748 11338 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:09.224725 11262 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:09.271906 11429 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.272512 11429 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:09.274892 11429 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:09.275910  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.263908 11436 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.278932 11341 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:09.276324  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:09.281447  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1450) took 1046 ms. Trace:
I20260709 12:27:09.281991  6768 rpcz_store.cc:276] 0709 12:27:08.235209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.388718 (+153509us) service_pool.cc:224] Handling call
0709 12:27:08.389145 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:08.416750 (+ 27605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:08.416786 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:08.469249 (+ 52463us) log.cc:844] Serialized 26 byte log entry
0709 12:27:08.792896 (+323647us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:08.792967 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.793066 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:09.281119 (+488053us) raft_consensus.cc:1685] finished
0709 12:27:09.281148 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.281361 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":323,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":26470,"thread_start_us":292966,"threads_started":1,"wal-append.queue_time_us":426941}
I20260709 12:27:09.283671 11338 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:09.284024 11410 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:09.284250 11436 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:09.284590  6020 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:09.287123  6020 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:09.287506 11262 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.287914 11386 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:09.288298 11262 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:09.270890 11417 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:09.247073 11275 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:09.290995 11275 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:09.291167 11211 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:09.294116 11432 consensus_queue.cc:1175] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" responder_term: 14 status { last_received { term: 13 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:27:09.295418 11227 consensus_queue.cc:1175] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1fc57407d56241b38a94d543e073f8af" responder_term: 14 status { last_received { term: 13 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:27:09.297649 11178 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:09.302665 11220 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:09.243681 11349 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:09.262051  6778 raft_consensus.cc:606] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:09.264583  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.245442 10671 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:09.322458 10671 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.312379 11308 raft_consensus.cc:3055] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:09.322995 10671 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:09.323318 11308 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:09.324599 11308 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:09.308209 11428 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:09.325158  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:09.326951 11368 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:09.327524 11368 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:09.327935 11368 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:09.330024  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:09.330790  5871 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:09.252785 11415 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.331907 11415 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:09.331921 11436 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:09.327035 11308 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:09.305953 11220 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:09.336295 11436 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:09.312953  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 76996.9us
I20260709 12:27:09.341583 11415 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.342067  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:09.344082  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:09.237574 11343 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:09.345788  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:09.346483  6773 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:09.346812  6773 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:09.347234 11343 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:09.347899  6773 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:09.348909  6773 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:09.310161 11417 raft_consensus.cc:697] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:27:09.312206 11358 consensus_meta.cc:287] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.833s	user 0.015s	sys 0.000s
I20260709 12:27:09.349926 11358 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:09.352039  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.352226 11358 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:09.310308  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.353267  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:09.354858 11195 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:09.356364 11417 consensus_queue.cc:237] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:09.357061 11437 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:27:09.357297  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:09.358090  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:09.358767  6768 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:09.301198 11414 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.359557  5946 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 15. Ops: []
I20260709 12:27:09.358120  6785 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:09.363200 11414 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:09.366663 11411 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:09.336033 11356 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:09.368139  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:09.368916  5868 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:09.369805 11156 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:09.371451 11442 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.093s	user 0.001s	sys 0.001s
I20260709 12:27:09.374228  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:09.374800 11438 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:27:09.375710  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:09.375815  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:09.376116 11407 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:09.376477  5872 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:09.376881  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.377374 11411 raft_consensus.cc:3055] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:09.377737  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:09.377923 11411 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:09.379626  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:09.379792 11265 raft_consensus.cc:3055] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:09.380277 11265 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:09.380560  5913 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:09.381026 11426 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:09.381505 11426 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.381940 11426 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:09.281436  5940 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:09.345870 11410 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:09.382668  6021 pending_rounds.cc:85] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 9
I20260709 12:27:09.383136  6021 pending_rounds.cc:107] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:27:09.383428  6021 raft_consensus.cc:2889] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:09.386436 11370 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:09.387090 11370 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:09.314358  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.385392  5940 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:09.270934  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:09.334931 11262 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:09.400256  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56279.7us
I20260709 12:27:09.404242  6785 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:09.313207  6778 raft_consensus.cc:614] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:09.407554  6020 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:09.412647  6768 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
W20260709 12:27:09.396131 11440 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.089s	user 0.004s	sys 0.000s
I20260709 12:27:09.376899 11349 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:09.432590  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:09.433746  5940 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:09.427313  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:09.422354 11436 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.428139  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:09.437798  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 189212us
I20260709 12:27:09.250963 11199 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:09.408080 11370 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:09.410429 11246 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:09.369102 10671 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.381220 11265 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:09.485293  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:09.344923 11415 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:09.439371  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:09.478359  5868 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
W20260709 12:27:09.490065  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 50773.6us
I20260709 12:27:09.452733  6773 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:09.491086  5990 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:09.477727  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 100108us
I20260709 12:27:09.476456  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:09.492964  6785 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:09.305585 11427 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.420379 11234 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:09.487267  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.490924  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:09.438797 11195 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.438429 11343 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.496256  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:09.500955  5940 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
W20260709 12:27:09.499805  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.478174 11414 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:09.504477  5945 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:27:09.429188 11220 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:27:09.475695 11431 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:09.428987 11428 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 12:27:09.439128 11410 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:27:09.517994  5947 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 13: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:09.516144  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:09.525539  5949 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:09.500514 11433 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:09.521162  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:09.508803 11377 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:09.509375 11421 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:09.456100 11061 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.457056 11436 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:09.531174  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 77459.7us
I20260709 12:27:09.531036 11069 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:27:09.541074  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:09.513295 11195 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:09.520452 11200 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:09.545683 11199 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:09.546294 11200 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:09.547260 11425 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:09.547885 11425 raft_consensus.cc:697] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:09.549868 11195 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:09.558569 11265 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:09.561693 11131 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:09.562278 11131 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.563517 11415 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:09.566342 11415 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:09.570569 11262 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:09.495149 11370 consensus_queue.cc:1059] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:27:09.571970  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:09.562408 11200 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:09.539091 11361 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:09.574947 11421 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.521448 11414 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:09.514940 11343 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:09.527267 11420 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:09.544209 11377 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:09.537735  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:09.539371  6609 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:09.518581 11431 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.535006  6785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:09.523442 11358 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:09.548957 11425 consensus_queue.cc:237] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:09.593955  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 53034.5us
W20260709 12:27:09.565531 11446 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:27:09.577898  5692 catalog_manager.cc:5697] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:09.596913  5940 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 19: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:09.605911  5872 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:09.594954  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:09.701692  5946 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:09.564352 11061 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:09.590476 11424 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:09.577572 11421 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:09.630693  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 91422us
I20260709 12:27:09.600090  6785 raft_consensus.cc:606] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:09.678730  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:09.676403 11370 raft_consensus.cc:3055] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:09.539952 11433 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:09.705243  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:09.707693  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:09.720993  5948 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:09.707952  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:09.562711 11131 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:09.678020  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:09.724690  5945 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:09.647709 11156 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:09.726171  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:09.709285  5913 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:09.590204 11436 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:09.588120  5691 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40812 (request call id 143) took 32189 ms (client timeout 15000 ms). Trace:
I20260709 12:27:09.589219 11420 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:09.738145  6609 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:09.742280  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:09.744299  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:09.745190  5940 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:27:09.746912 11199 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:09.747736 11199 raft_consensus.cc:697] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:09.750026  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:09.750970  6020 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:09.607784 11431 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:09.752993  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:09.581682 11415 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:09.586262 11343 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:09.766018  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 61849.5us
I20260709 12:27:09.578709 10671 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:09.591455 11427 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:09.719373 11370 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:09.730154  5949 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:09.720378  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:09.730744  6773 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:09.706482 11195 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:09.750092  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:09.774506  5692 catalog_manager.cc:5697] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:09.760504  5946 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:27:09.762274  5948 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:09.720211  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:09.781267  6768 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:09.771596  5838 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:09.579579 11414 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:09.713331 11358 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:09.590794 11377 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:09.771225  5945 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:09.784339 11234 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:09.753860 11436 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.784569 11276 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:09.785101 11449 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:27:09.787511 11370 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:09.789026 11246 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:09.733042 11426 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:09.790333  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 89220.9us
I20260709 12:27:09.758893 11454 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:09.794374  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44438.1us
I20260709 12:27:09.794744  5947 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:09.794973 11410 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:09.772538 11156 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:09.776058  6021 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:27:09.837071  5915 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:09.749001 11199 consensus_queue.cc:237] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.802862  5838 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:09.811445  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 58530.4us
I20260709 12:27:09.804649  5692 catalog_manager.cc:5697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 17 to 18, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 18 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:09.814579  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.782819 11178 raft_consensus.cc:2764] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 12:27:09.852473  6756 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:09.852922 11349 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:09.852659  5692 catalog_manager.cc:5697] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:09.854542  5692 catalog_manager.cc:5697] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 13 to 16, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 16 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:27:09.855929  5692 catalog_manager.cc:5697] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:09.819242 11436 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.785125 11195 raft_consensus.cc:697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:09.803859 11415 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:09.791141 11411 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.804041 11234 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.862455 11234 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:09.862841 11411 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:09.768745 11368 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.866453 11338 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.867269 11338 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.867869 11368 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:09.846662  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:09.822417 11276 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.870391 11338 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:09.871922 11276 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:09.844192 11211 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:09.873153  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:09.873100 11211 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:09.875989 11211 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:09.876786 11211 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:09.850060 11421 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:09.878057 11421 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:09.851917  5943 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:09.851436 11410 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:09.882169  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 48701.4us
I20260709 12:27:09.880430 11425 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:09.880524  5945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533"
dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:48330
I20260709 12:27:09.908777  5945 raft_consensus.cc:606] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:27:09.909186  5945 raft_consensus.cc:614] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:09.836150 11361 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:09.880971  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:09.915407  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:09.917176  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:09.887694  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:09.918748  5949 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 13: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:09.888783 11436 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:09.890699 11459 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.877410 11211 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:09.878837 11370 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:09.895910 11234 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:09.899076  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:09.901044  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:09.836714 11433 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:09.860409 11415 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:09.906147 11386 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:09.907984  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:09.911015  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:09.936128  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:09.942860  5867 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:09.920543  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:09.922169  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:09.924667  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:09.916078  6611 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:09.886449 11426 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:09.927680  5915 leader_election.cc:400] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21.
I20260709 12:27:09.944204  5873 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:27:09.947638 11234 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:09.946130  5871 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:09.930598 11425 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:09.935843 11459 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.946599 11069 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:09.935770 11421 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:09.947901  5915 leader_election.cc:403] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:09.861125 11061 consensus_queue.cc:1059] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:27:09.894184  5692 catalog_manager.cc:5697] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 16 to 19, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 19 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
W20260709 12:27:09.952780  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.934909 11211 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:09.936870 11386 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:09.969988 11386 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:09.967665  5871 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 12:27:09.931134 11448 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:27:09.966302 11156 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:09.927112 11462 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:09.967355 11459 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:09.938501 11434 consensus_meta.cc:287] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.617s	user 0.012s	sys 0.000s
I20260709 12:27:09.982049  5692 catalog_manager.cc:5697] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:09.982586 11434 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:09.983783  5692 catalog_manager.cc:5697] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 9 to 16, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 16 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:09.960597  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:09.930982 11454 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:09.961508 11436 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:09.979961  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:09.985932  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 44906.6us
I20260709 12:27:09.985762  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:09.990576  5941 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:09.991034  5872 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:09.933815 11349 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:09.993490 11349 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.993902 11349 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:09.995162 11425 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:09.936394 11415 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:09.987673 11454 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.997020 11361 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:09.998359 11454 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:09.993016 11462 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:10.000499 11462 raft_consensus.cc:697] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:10.001598 11462 consensus_queue.cc:237] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:10.002328 11466 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21.
I20260709 12:27:09.990128  5870 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
W20260709 12:27:10.007704  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:10.010099  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.009894 11436 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:10.011477 11436 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:10.011581 11410 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:10.017874 11195 consensus_queue.cc:237] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:09.974727 11200 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:10.028312  5692 catalog_manager.cc:5697] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 12 to 13, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 13 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:27:10.030844  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:09.984686 11434 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:10.030998  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:10.027518  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.035812  5870 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:10.038028  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:10.025097  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:09.920377 11358 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:10.043781  5945 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:27:10.040861 11246 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:10.050405  5873 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:10.025985 11417 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:09.971179 11386 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:10.027441 11200 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.047880  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:10.045756  5692 catalog_manager.cc:5697] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 12 to 16, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 16 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:27:10.053129  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:10.058351  5865 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:10.021375 11349 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:10.059453  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.060497  5867 raft_consensus.cc:2393] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:10.060655 11349 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:10.054991  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 171429us
W20260709 12:27:10.055903  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 139916us
I20260709 12:27:10.063185  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.063068  5692 catalog_manager.cc:5697] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 13 to 16, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 16 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:27:10.063843  5872 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:10.065699  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:10.069891  5871 raft_consensus.cc:2393] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:10.071509  5692 catalog_manager.cc:5697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:10.075229  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:10.078321  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:10.081004  5873 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:27:10.091794  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:10.092651  6773 raft_consensus.cc:606] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:10.093117  6773 raft_consensus.cc:614] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:10.096119  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.099732  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.062217 11200 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:10.102756  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.066017 11433 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:10.064623 11246 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:10.119374  5871 raft_consensus.cc:2410] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:27:10.129684  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.103231 11386 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:10.134482  5873 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:10.135185  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:10.137064  5947 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:10.062719 11417 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:10.139779 11462 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.140873  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.179065  6021 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:27:10.145102 11377 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.145651 11453 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.146204  5946 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:27:10.122386 11433 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.148384 11415 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:10.122968 11131 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:10.150276  5947 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:10.149995  5692 catalog_manager.cc:5697] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 16 to 17, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 17 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:27:10.151441  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:27:10.152720 11454 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:10.155550  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:10.170200  5940 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:27:10.176673  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.182514  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.189471  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:10.191624  5692 catalog_manager.cc:5697] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:27:10.192869  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:10.202795  6017 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:10.196170  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.192469  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:10.204838  5865 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:10.212920  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.220614  5871 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:27:10.227494  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:10.233976  6022 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:10.234206  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:10.190869 11433 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:10.238744  5692 catalog_manager.cc:5697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:10.241009  5873 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:10.199033 11234 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.241246  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:10.242911  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.254942  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.255674  5870 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:27:10.189270 11377 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:10.258665  5941 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:27:10.187386 11462 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:10.244663  5915 connection.cc:256] RPC call timeout handler was delayed by -2.2475486879998243s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2215s for a response
W20260709 12:27:10.263775  5915 connection.cc:256] RPC call timeout handler was delayed by -1.8252313790003427s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1754s for a response
W20260709 12:27:10.264151  5915 connection.cc:256] RPC call timeout handler was delayed by -1.8009313790003034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1781s for a response
W20260709 12:27:10.264443  5915 connection.cc:256] RPC call timeout handler was delayed by -1.6785313790001055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
I20260709 12:27:10.264816  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:10.265554  5870 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:10.266355  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:10.269904  5692 catalog_manager.cc:5697] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
W20260709 12:27:10.304754  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.320176  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:10.191651 11453 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:10.321758  5873 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:27:10.253665 11265 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.329078  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:10.329759 11417 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:10.330561 11471 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:27:10.338423 11454 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:10.340585  5692 catalog_manager.cc:5697] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:10.330019 11475 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:27:10.353078  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:10.346504  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.349287  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:10.248811 11234 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:10.357992  5873 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:10.253502 11361 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 12:27:10.242507 11431 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:10.388094  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:10.390681  5692 catalog_manager.cc:5697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 12 to 14, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 14 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:27:10.363535  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:10.269279 11246 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:10.372253  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:10.374219  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.405689  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:10.374743 11458 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.514s	user 0.002s	sys 0.001s
W20260709 12:27:09.732546  5691 rpcz_store.cc:269] 0709 12:26:37.413343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.413733 (+   390us) service_pool.cc:224] Handling call
0709 12:27:09.585571 (+32171838us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:03.871909 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:03.872634 (+   725us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:03.872689 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:27:03.872709 (+    20us) tablet.cc:662] Decoding operations
0709 12:27:03.874265 (+  1556us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:03.874366 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:03.874399 (+    33us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:27:04.022458 (+148059us) tablet.cc:701] Row locks acquired
0709 12:27:04.022514 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:27:04.022716 (+   202us) write_op.cc:270] Start()
0709 12:27:04.022885 (+   169us) write_op.cc:276] Timestamp: P: 1783600024022636 usec, L: 0
0709 12:27:04.022929 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 12:27:04.023744 (+   815us) log.cc:844] Serialized 6265 byte log entry
0709 12:27:04.930553 (+906809us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:05.559066 (+628513us) op_driver.cc:464] REPLICATION: finished
0709 12:27:05.754395 (+195329us) write_op.cc:301] APPLY: starting
0709 12:27:05.762600 (+  8205us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:06.980982 (+1218382us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:06.997338 (+ 16356us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:07.603250 (+605912us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:07.649978 (+ 46728us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 12:27:07.650119 (+   141us) write_op.cc:312] APPLY: finished
0709 12:27:07.651227 (+  1108us) log.cc:844] Serialized 147 byte log entry
0709 12:27:07.677909 (+ 26682us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:07.678701 (+   792us) write_op.cc:454] Released schema lock
0709 12:27:07.678860 (+   159us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:07.712533 (+ 33673us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:07.712566 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1745821,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":164639,"apply.run_cpu_time_us":59249,"apply.run_wall_time_us":1980080,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71071,"num_ops":16,"prepare.queue_time_us":404290,"prepare.run_cpu_time_us":24762,"prepare.run_wall_time_us":552049,"raft.queue_time_us":420215,"raft.run_cpu_time_us":40980,"raft.run_wall_time_us":121497,"replication_time_us":1535964,"spinlock_wait_cycles":5201792,"thread_start_us":1132019,"threads_started":5,"wal-append.queue_time_us":544670,"wal-append.run_cpu_time_us":44298,"wal-append.run_wall_time_us":3054814}]]}
W20260709 12:27:10.383186  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.322132 11069 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.420702  5941 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:27:10.358017 11454 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:10.358319 11460 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.408963  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.439203  5692 catalog_manager.cc:5697] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 12 to 13, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 13 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:10.440059  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.410267  5874 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:10.356290 11370 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.426628  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.443733  6017 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:10.427455  5949 raft_consensus.cc:2368] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17.
W20260709 12:27:10.404642  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51656.5us
I20260709 12:27:10.447398  6019 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:10.401343  5865 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:10.433691  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:10.433786  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:10.434173 11431 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:10.427611  5871 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:10.350306 11265 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:10.470233  5692 catalog_manager.cc:5697] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:27:10.475793  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:10.450003  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.429490  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.481407  5865 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:10.481408  5866 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:27:10.487205  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.492499 11308 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:10.492666  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 70008.6us
I20260709 12:27:10.496100  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:10.497854  5873 raft_consensus.cc:2393] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:10.501645  5692 catalog_manager.cc:5697] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 8 to 14, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 14 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:10.514567 11195 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:10.514737  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.515446  5874 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 13.
I20260709 12:27:10.520073  5692 catalog_manager.cc:5697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 13 to 17, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 17 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:27:10.515880  5870 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:10.523242  6022 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:27:10.523823  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:10.524770  5865 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:27:10.473049 11454 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:10.425058 11481 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:10.525007  5866 raft_consensus.cc:2376] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14. Re-sending same reply.
W20260709 12:27:10.534968 11451 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:27:10.540333 11200 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:10.542233  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 46268.1us
I20260709 12:27:10.543224 11200 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:10.543668  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:10.544937 11200 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:10.549890 11447 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:27:10.556236  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:10.569885  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:10.572890  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:10.444178 11069 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:10.511183 11432 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.458809 11460 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:10.586985  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.590358  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.575165  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:10.577360 11200 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:10.474769 11370 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:10.599037  6021 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:10.599262 11200 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:10.599884 11200 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:10.600535 11370 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:10.600811 11200 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:10.608964  6021 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:10.578553 11308 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:10.583671 11195 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:10.586354 11131 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:10.629029 11480 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:10.630920 11069 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:10.631218  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.634104 11481 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:10.638110  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.641808 11479 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:10.667920  5866 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:10.644752  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.645620  6757 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:10.646111  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.646221  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:10.671856  6762 raft_consensus.cc:2393] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 18: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:10.645694  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:10.645707  6019 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 12:27:10.637724  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 111061us
I20260709 12:27:10.584798 11415 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:10.648859  5990 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:10.677197  6022 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:10.648989  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:10.677886  5990 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:10.647264  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.652570  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.665251  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:10.670459  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:10.662890 11451 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:27:10.627332 11377 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:10.672425  6775 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:27:10.686008  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:10.686715  6775 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:10.678406  5870 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:10.680785  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:10.688380  6757 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:10.682623  6793 raft_consensus.cc:2368] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 14. Current term is 15.
I20260709 12:27:10.679487  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:10.668525 11480 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:10.690219 11464 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:10.690636  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:10.691411  6762 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 10: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:10.691511 11464 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:10.591627 11432 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:10.692521 11480 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:10.692281 11464 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:10.684151 11131 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:10.693660 11131 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.693787 11432 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:10.694173 11131 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:10.694656 11370 consensus_queue.cc:1059] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:10.696233 11459 raft_consensus.cc:3055] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:10.696771 11459 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:10.672384 11481 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:10.697790 11459 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:10.680035  6778 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:10.699101  5915 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:10.699916 11481 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:10.676411 11454 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:10.683509  6020 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:10.705857  6766 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:10.708030  6024 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:10.708386  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:10.708963  6785 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:10.709386  5947 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:10.708115 11407 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:10.710234 11407 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:10.693822 11368 raft_consensus.cc:3055] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:10.710614 11407 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:10.710811 11368 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:10.711045  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:10.711895 11368 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:10.712433  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.713308  6017 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:10.709384  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:10.713228 11368 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:10.715517  5840 connection.cc:256] RPC call timeout handler was delayed by -2.0158754210001462s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1748s for a response
I20260709 12:27:10.670234 11479 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:10.726356  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:10.727380  6020 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:27:10.730113  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.732075  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:10.733840 11479 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:10.681947 11453 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:10.734790  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:10.692340 11361 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:10.731276  6024 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:27:10.748890  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:10.750252  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:10.751760  6756 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 10: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:10.751791  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1478) took 1201 ms. Trace:
W20260709 12:27:10.764190  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.752348  5940 rpcz_store.cc:276] 0709 12:27:09.550673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.765829 (+215156us) service_pool.cc:224] Handling call
0709 12:27:10.169782 (+403953us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:10.197796 (+ 28014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:10.751606 (+553810us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b54006761d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:10.751734 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":918232832}
I20260709 12:27:10.733494  6785 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:10.699137 11459 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:10.775031  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:10.785360  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:10.790357  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 74761.6us
I20260709 12:27:10.791385  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:10.794353  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.795275  6023 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:27:10.796489  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.798162  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:10.799232  5947 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 19: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:10.753696 11361 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.800516  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.801858  6023 raft_consensus.cc:2410] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:27:10.802922  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:10.803635  6023 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:10.815104  6609 leader_election.cc:400] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17.
I20260709 12:27:10.773710 11211 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:10.815574  6609 leader_election.cc:403] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:27:10.779822 11417 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:10.825778  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 133884us
I20260709 12:27:10.821417  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:10.817140  6020 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:10.826645  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:10.780215 11265 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:10.789754 11481 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:10.835605  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44359.5us
I20260709 12:27:10.836570  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:10.815191 11276 raft_consensus.cc:3055] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:10.844424  6023 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:10.815932 11361 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:10.846418  6611 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:10.846669  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.847534  6023 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:10.823163 11211 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:10.856141  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:10.814651 11473 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:27:10.853984  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:10.850775  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:10.839751 11431 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:10.868549  6611 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:10.845983 11276 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:10.852700 11358 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:10.860914 11211 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:10.881167  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:10.882542 11246 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:10.882402 11178 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:10.883236 11200 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:10.884606 11488 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
W20260709 12:27:10.903683  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 77128us
I20260709 12:27:10.836645 11417 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:10.870059 11069 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:10.864151 11492 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:10.866585 11265 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.845049 11481 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:11.137012 11481 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:10.920584  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:10.926481 11480 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:10.906535 11246 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.906589 11460 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:10.912231 11349 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:11.091982 11386 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.916s	user 0.010s	sys 0.007s
I20260709 12:27:11.092934 11500 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.111478 11433 consensus_meta.cc:287] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.860s	user 0.013s	sys 0.001s
I20260709 12:27:11.093143 11131 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:11.094363 11234 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:11.096131 11417 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:11.096645 11464 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:11.099498  6020 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:11.104426  6019 raft_consensus.cc:2410] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 12:27:11.007139 11496 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:10.971911 11276 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:11.111164  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 723091us
W20260709 12:27:11.112534  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.113978 11220 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.114842  5840 connection.cc:256] RPC call timeout handler was delayed by -1.6789987860001929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
W20260709 12:27:10.996035 11494 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:27:11.197160  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:11.202239 11492 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17.
I20260709 12:27:11.115110 11156 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.115939  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 265302us
I20260709 12:27:11.122990 11432 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:27:11.127532  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.130760  5942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1477) took 1714 ms (client timeout 2055 ms). Trace:
I20260709 12:27:10.873710 11360 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:11.161890 11433 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:11.143918 11480 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:11.144724 11462 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:11.145584  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 225091us
I20260709 12:27:11.116917 11361 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:11.147795 11460 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:11.148660  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1118) took 1027 ms. Trace:
I20260709 12:27:11.156934  6016 raft_consensus.cc:2410] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:27:11.219887  5945 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 10.
I20260709 12:27:11.209314 11432 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:11.221402 11432 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:11.222980 11432 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:11.219373  5867 rpcz_store.cc:276] 0709 12:27:10.121013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.122164 (+  1151us) service_pool.cc:224] Handling call
0709 12:27:10.137932 (+ 15768us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:10.189405 (+ 51473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:10.189454 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:10.223039 (+ 33585us) log.cc:844] Serialized 52 byte log entry
0709 12:27:10.446441 (+223402us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:11.147688 (+701247us) spinlock_profiling.cc:243] Waited 6.43 ms on lock 0x7b18000fbc80. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39d282b4bc 00007f39cfb7948b 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d667b3c7 00007f39d2c8d9a7 00007f39d2c9d0bf
0709 12:27:11.147887 (+   199us) log.cc:844] Serialized 14 byte log entry
0709 12:27:11.148164 (+   277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:11.148249 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:11.148294 (+    45us) raft_consensus.cc:1685] finished
0709 12:27:11.148402 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:11.148575 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10776,"prepare.queue_time_us":256305,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":18147,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":4551,"spinlock_wait_cycles":14800256,"thread_start_us":372106,"threads_started":2,"wal-append.queue_time_us":345355}
I20260709 12:27:11.167397  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:11.228520  6756 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:27:11.228968 11276 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:11.168458  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:11.234429 11489 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:27:11.234515  6768 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:11.149350 11265 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:11.236704  6017 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:27:11.242447 11200 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:11.242789 11361 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:11.171358 11505 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.077s	user 0.002s	sys 0.000s
I20260709 12:27:11.173563 11464 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:11.246018  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:11.178738  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:11.191387  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.246970  6768 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:11.248768  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:11.195211  5840 connection.cc:256] RPC call timeout handler was delayed by -1.4918083659999866s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
W20260709 12:27:11.249809  5840 connection.cc:256] RPC call timeout handler was delayed by -1.4440987860002679s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
W20260709 12:27:11.250119  5840 connection.cc:256] RPC call timeout handler was delayed by -1.400898786000198s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19330000000000003s for a response
I20260709 12:27:11.146204 11246 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:11.206231 11220 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:11.214586 11195 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.003s
I20260709 12:27:11.215477 11480 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:11.209141  5915 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:11.268281  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:11.209348  5872 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:27:11.209789  5942 rpcz_store.cc:269] 0709 12:27:09.416652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.571501 (+154849us) service_pool.cc:224] Handling call
0709 12:27:11.130470 (+1558969us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400672d54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:11.130635 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2711579904}
I20260709 12:27:11.212488  6024 raft_consensus.cc:2368] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14.
I20260709 12:27:11.210999  5869 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:11.214380  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:11.213219 11131 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:11.249413  5865 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:11.272897  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:11.250401  5870 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:11.273566  5867 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:27:11.247012  5871 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:11.250402  5840 connection.cc:256] RPC call timeout handler was delayed by -1.3868517370001427s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
I20260709 12:27:11.253777  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:11.275358  5873 raft_consensus.cc:2410] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:27:11.263250  5947 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:11.263520  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 1362) took 1072 ms. Trace:
I20260709 12:27:11.208274 11156 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:11.282898 11065 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:11.282761 11427 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:11.283480 11065 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:11.283954 11065 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:11.137034 11479 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:11.284310 11156 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:11.285337  6773 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
W20260709 12:27:11.264251  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.255158  5838 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:11.291185  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.265424  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:11.162531 11069 raft_consensus.cc:697] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:11.267066  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:11.298383  6766 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:11.269066  5868 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:27:11.277058  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:11.246881 11407 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.507s	user 0.016s	sys 0.000s
I20260709 12:27:11.278721 11220 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:11.280057  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 65735.7us
I20260709 12:27:11.301199  6793 raft_consensus.cc:2393] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 17: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:27:11.302034  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 92961.7us
I20260709 12:27:11.303385  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:11.304349  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:11.161216 11500 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:11.280881  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:11.282295 11464 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:11.292207  6756 raft_consensus.cc:2410] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 12:27:11.293292  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.307981  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.297067  5865 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:11.298507  6778 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:11.275902  6016 rpcz_store.cc:276] 0709 12:27:10.191174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.192599 (+  1425us) service_pool.cc:224] Handling call
0709 12:27:11.263245 (+1070646us) spinlock_profiling.cc:243] Waited 950 ms on lock 0x7b540026fa54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:11.263441 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2185524992}
I20260709 12:27:11.309067  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:11.350309 11492 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:11.351519 11341 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:11.351327 11492 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:11.310338  5866 raft_consensus.cc:2393] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 15: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:11.306325 11069 consensus_queue.cc:237] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:11.310770  5915 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:11.205583 11211 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:11.275141 11195 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:11.310999  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:11.295779  6768 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:11.360066  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:11.360697  6768 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:11.314044  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:11.361306  6773 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:11.314388  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:11.305400 11156 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:11.322414  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:11.323318  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:11.365685  5871 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:11.370534  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:11.301266 11433 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:11.375304  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:11.375106  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:11.376327  5872 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:27:11.376538  6766 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:11.371551  5943 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:27:11.367218  5913 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:11.314277 11464 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:11.326244  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:11.327420  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:11.328157  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:11.330302  5990 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:11.312165 11361 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:11.303722 11460 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:11.306465 11265 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:11.403010  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 72772.1us
W20260709 12:27:11.313088 11498 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.225s	user 0.002s	sys 0.003s
I20260709 12:27:11.335345  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:11.339326  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 225401us
I20260709 12:27:11.306737 11407 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:11.347335  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:11.408064  5947 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 10.
I20260709 12:27:11.351220  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:27:11.388258  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.389266  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:11.389197  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:11.392874  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 81930.6us
I20260709 12:27:11.364037  5942 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:11.409956  5940 raft_consensus.cc:2393] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:11.336642 11131 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:27:11.335532 11434 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:27:11.393357  6768 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:11.420480  6768 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:27:11.434559  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:11.434860  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:11.435119  6793 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:11.342048 11368 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:27:11.391736  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:11.414646  6773 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:11.414347  5838 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:11.415866  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:11.416467  6024 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:11.416575  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:11.396174  5868 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:27:11.335058  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:11.421761  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:11.396148  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:11.422438  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:11.476814  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 85198.8us
I20260709 12:27:11.478435  6756 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:27:11.479080  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 64832.4us
I20260709 12:27:11.478579  6609 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:11.398397 11195 raft_consensus.cc:993] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: : Instructing follower 7a0204b931394fc48ffc503716e0d111 to start an election
I20260709 12:27:11.479597  6023 raft_consensus.cc:2393] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:11.394246 11361 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.426455  5990 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:11.479246  6778 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
W20260709 12:27:11.476123  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 172986us
W20260709 12:27:11.486490  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 60142.9us
I20260709 12:27:11.488138  5915 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:11.480682  6019 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:27:11.489024  6019 pending_rounds.cc:85] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 7
I20260709 12:27:11.489290  6019 pending_rounds.cc:107] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:27:11.489511  6019 raft_consensus.cc:2889] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:11.419387 11308 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:11.421408 11178 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:27:11.494513  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 58182.1us
I20260709 12:27:11.500638  6611 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:11.475246 11341 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:11.410895 11417 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:11.418238 11504 raft_consensus.cc:2764] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 12:27:11.506299  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 83908.6us
W20260709 12:27:11.506903  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 155751us
I20260709 12:27:11.491962 11361 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:11.509451  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:11.407084 11156 raft_consensus.cc:697] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:11.493855 11220 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:11.419857 11503 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:11.495154 11460 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:11.514346  5990 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:11.513685  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:11.515537  5942 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:11.513708  5940 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:11.521559  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:11.522018 11464 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:11.522228 11511 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:11.523134 11511 raft_consensus.cc:1081] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Signalling peer 7a0204b931394fc48ffc503716e0d111 to start an election
I20260709 12:27:11.524109  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:11.524825  5945 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:11.421928 11386 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 12:27:11.517493 11503 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.535583 11377 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:11.508239 11360 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:27:11.514238 11417 raft_consensus.cc:697] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:11.522710 11464 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.538244 11464 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:11.538462 11417 consensus_queue.cc:237] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:11.505656 11433 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:11.527511 11490 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.505990 11308 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:11.540544 11490 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:11.531599 11460 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:11.535941 11503 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:11.503687 11414 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:11.543758  5943 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:27:11.398095 11481 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.108s
I20260709 12:27:11.544842  5945 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:27:11.552932  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:11.515439  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:11.497802 11509 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:11.539582  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:11.554275  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:11.561786 11200 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:11.562693 11262 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:11.570021 11407 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:11.530522 11473 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:27:11.531315 11349 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:27:11.561834  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:11.582324  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:11.584129  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:11.514287 11220 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.512593 11479 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:11.543295 11341 raft_consensus.cc:697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:11.598610  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.530184 11462 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:11.593258 11377 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:11.615181  5942 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:11.618868  6766 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
W20260709 12:27:11.620182  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 110866us
I20260709 12:27:11.506635 11178 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:11.576743 11386 raft_consensus.cc:3055] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:11.622884 11434 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.610389 11220 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:11.621291  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:11.623272  5990 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:11.557502 11509 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.624946  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:11.614293  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:11.643002  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:11.593458 11360 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:11.578542 11511 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:11.612567 11432 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:27:11.609894 11492 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:11.623333 11459 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:11.631776 11500 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:11.645272  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 116577us
I20260709 12:27:11.646328  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:11.626835 11386 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:11.628371 11462 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:11.593238 11308 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:11.614930 11065 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:11.558311  6609 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:11.681262  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 122982us
I20260709 12:27:11.641965 11509 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:11.652259 11246 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:11.667961 11459 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:27:11.682767 11481 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:11.683243 11481 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.683607 11481 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:11.684415 11246 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:11.685127 11246 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:11.646759 11276 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:11.678887  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:11.632586 11434 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:11.695842  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:11.651086 11436 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:11.576007 11460 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:11.669595  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 181489us
I20260709 12:27:11.726048  5915 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:11.649448 11511 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.654165 11131 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:11.713191  6609 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:11.656343 11368 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:11.738698  5942 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:11.743556 11178 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:11.747352  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:11.748365  5873 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:11.694926 11481 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:11.770646 11436 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:11.771765 11481 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:11.774380  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 48428.5us
W20260709 12:27:11.774752  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.775436  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:11.759037 11276 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:11.778554  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:11.715754 11490 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:11.679047 11479 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
W20260709 12:27:11.790788  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 78580.5us
I20260709 12:27:11.728031 11511 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:11.645577 11156 consensus_queue.cc:237] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:11.812426 11065 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:11.789649 11508 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 12:27:11.815994  6785 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:11.816352  6778 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:11.817118  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:11.818629  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:11.819401 11368 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:11.821144  6757 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:11.807906 11492 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:11.823523  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:11.826254  6766 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
W20260709 12:27:11.825387  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 177158us
I20260709 12:27:11.843456  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:11.844249  5947 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:11.844602  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:11.826181  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 205014us
I20260709 12:27:11.718398 11341 consensus_queue.cc:237] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:11.844892  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:11.787159 11265 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.793923 11504 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:11.790035 11427 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:11.846765  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:11.852874  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:11.773907 11262 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:27:11.858095  6793 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:11.859702  5945 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:27:11.861438  5943 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:11.862324 11433 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:11.862874 11433 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:11.862944  5942 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:11.863513  5942 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:11.863953 11433 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:11.864573  5942 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:11.865252  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:11.865875  5872 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:11.866146  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:11.866362  5871 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:54876
I20260709 12:27:11.866741  5869 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:11.868361 11480 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:11.869127  5865 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:27:11.870054 11480 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.870671 11480 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:11.874007  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:11.874735  5869 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:11.875101  6609 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:11.876291 11490 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:11.876729 11490 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:11.877118 11490 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:11.887557  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:11.887624  6020 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:11.890081  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:11.895146  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:11.896085  5943 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:11.937700  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:11.942993  5872 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:11.855507 11211 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:11.946985  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:11.945353  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:11.899575  5943 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:11.952732  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:11.771780 11436 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:11.955057  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 108360us
I20260709 12:27:11.886026 11410 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:11.977604  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:11.942495 11500 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:11.934612 11432 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.853964 11368 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:11.934322 11492 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.992142  5840 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:11.883249 11265 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:11.959177 11234 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:11.901536 11453 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.078s
W20260709 12:27:11.997120  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 188230us
I20260709 12:27:11.998819  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:11.984853 11500 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.013978  5945 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:27:12.016688  5869 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:11.858698 11462 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:12.038957  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:12.040220  5945 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:12.043511  6609 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:11.932515 11276 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:12.006692 11522 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.080029  6609 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:12.080405 11504 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:12.089298  5871 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:12.090080  5871 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:12.088361  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:12.033473 11500 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:12.009299 11234 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:12.106472  5871 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:11.994807 11432 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:12.004558 11492 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:12.145805  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 153775us
I20260709 12:27:12.097868 11454 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:12.105058 11473 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:12.146800  5840 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:12.117167 11195 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:12.036667 11523 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.131s
I20260709 12:27:12.069044 11349 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 12:27:12.075712 11246 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 12:27:12.075361 11433 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:27:12.154218 11454 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:12.155722 11195 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.156230 11262 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:12.105420 11522 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:12.194428  5867 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:27:12.164109 11360 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:12.173652 11516 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.333s	user 0.000s	sys 0.002s
I20260709 12:27:12.176043 11211 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:12.168465 11473 raft_consensus.cc:697] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:12.207927 11510 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:27:12.204562 11360 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:12.197371 11262 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:12.212462 11504 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:12.200981 11549 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.195428 11454 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:12.273240  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:12.274412  6778 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
W20260709 12:27:12.219573 11220 consensus_meta.cc:287] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.000s
I20260709 12:27:12.220922 11432 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:12.221328 11386 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:12.223515 11368 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:12.218611 11529 log.cc:927] Time spent T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Append to log took a long time: real 0.085s	user 0.002s	sys 0.000s
I20260709 12:27:12.197654 11195 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:12.287026  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:12.271721  6611 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:12.315403  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:12.316385  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44831.1us
W20260709 12:27:12.292448  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46329.7us
W20260709 12:27:12.275913 11503 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.703s	user 0.013s	sys 0.000s
W20260709 12:27:12.253921 11527 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.139s	user 0.000s	sys 0.003s
W20260709 12:27:12.254117 11524 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.092s	user 0.001s	sys 0.001s
W20260709 12:27:12.343967  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 152880us
I20260709 12:27:12.233592 11360 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:12.239635 11522 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:12.235698 11262 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:12.215548 11211 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:27:12.257853 11520 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.292s	user 0.001s	sys 0.003s
I20260709 12:27:12.346582  5990 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:12.234855 11265 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:12.283929 11539 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:27:12.275614 11069 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:12.317802 11361 consensus_meta.cc:287] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.776s	user 0.009s	sys 0.006s
W20260709 12:27:12.320473 11377 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.695s	user 0.015s	sys 0.000s
W20260709 12:27:12.351195  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:12.350962 11511 consensus_meta.cc:287] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.001s
I20260709 12:27:12.341874 11200 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:12.352280 11538 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:12.374452  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:12.263370 11549 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:12.249761 11234 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:12.343743 11433 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:27:12.381868 11535 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.385111 11480 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:12.390048 11522 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:27:12.391541  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.389348  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:12.400490  5987 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 2.020s (SENT)
I20260709 12:27:12.429155  5987 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:12.430631  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:12.432046  5840 leader_election.cc:400] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 14. Current term is 15.
I20260709 12:27:12.432412  5840 leader_election.cc:403] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:27:12.404188  5871 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:12.366766 11523 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.411511 11410 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:12.289603 11220 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:12.349263 11432 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:27:12.367504 11211 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:12.368695 11360 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:12.346814 11503 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:12.416528 11377 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:12.422116 11534 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.483471  5873 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:12.442831 11410 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:12.378556 11361 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:12.478180  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 77390us
I20260709 12:27:12.379335 11069 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:12.447574 11545 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:12.455294 11360 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.418305 11511 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:12.370353 11473 consensus_queue.cc:237] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:12.539503  5987 leader_election.cc:336] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 2.129s (SENT)
I20260709 12:27:12.540417  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:12.484639 11541 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:12.484838 11555 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:12.451035 11523 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:12.504321 11545 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.542629 11360 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:12.441462 11535 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:12.543476 11368 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 14. Current term is 15.
I20260709 12:27:12.543921 11377 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:12.564167 11555 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:12.577617 11464 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.039s	user 0.006s	sys 0.010s
I20260709 12:27:12.489862 11534 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:12.630964 11558 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:27:12.632105 11557 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 12:27:12.632920  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 201036us
I20260709 12:27:12.577744 11545 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:12.608937 11432 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:12.606173 11462 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.626922 11555 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:12.661942 11462 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:12.670923 11377 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:12.674288 11504 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.676690 11504 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:12.677335 11503 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:12.656064 11511 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:12.623962 11464 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:12.706604  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 167114us
W20260709 12:27:12.708796  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 62238.4us
I20260709 12:27:12.711853  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:12.714498  5987 leader_election.cc:336] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 2.139s (SENT)
I20260709 12:27:12.715296  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:12.717550  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:12.718358  6793 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:12.719426  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:12.721218  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:12.687635 11519 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.668s	user 0.000s	sys 0.001s
I20260709 12:27:12.681241 11200 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:12.710789 11377 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:12.735670  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:12.733281  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:12.759078  6773 raft_consensus.cc:3055] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:12.759626  6773 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:12.734230  6785 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: [16.10-18.11]
I20260709 12:27:12.680701 11368 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:27:12.640286 11410 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:12.723940 11460 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:12.759079  6762 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:12.727098 11503 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:12.758287 11500 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.612s	user 0.005s	sys 0.017s
W20260709 12:27:12.762527 11509 consensus_meta.cc:287] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.080s	user 0.020s	sys 0.008s
I20260709 12:27:12.788731  6773 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:12.792771  6773 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:12.794265 11500 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:12.757084 11436 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:12.797465 11500 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:12.798688 11436 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:12.809425 11436 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:12.790307 11427 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.018s	sys 0.027s
W20260709 12:27:12.779958 11490 consensus_meta.cc:287] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.902s	user 0.017s	sys 0.009s
I20260709 12:27:12.771871 11156 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:12.771783 11511 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:12.791900 11410 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:12.832420 11427 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.019s	sys 0.028s
I20260709 12:27:12.763140 11200 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:12.771669 11540 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:27:12.830065 11473 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.896953  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:12.894055  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:12.886451  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:12.903535  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:12.878957  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1493) took 1338 ms. Trace:
I20260709 12:27:12.896479  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:12.903923  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:12.905313  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:12.887214  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:12.903829  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:12.907037  5871 raft_consensus.cc:2376] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16. Re-sending same reply.
I20260709 12:27:12.907388  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:12.907575  5869 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:12.907513  5872 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:12.826826 11504 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:12.793888 11503 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:12.922307  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:12.915048  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:12.917892  5840 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:12.904558  6756 rpcz_store.cc:276] 0709 12:27:11.540476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.540763 (+   287us) service_pool.cc:224] Handling call
0709 12:27:11.541115 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:11.541599 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:11.541767 (+   168us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:11.552154 (+ 10387us) log.cc:844] Serialized 26 byte log entry
0709 12:27:11.776030 (+223876us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:11.776091 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:11.776182 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:12.672776 (+896594us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:12.863551 (+190775us) raft_consensus.cc:1685] finished
0709 12:27:12.863579 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:12.878857 (+ 15278us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1011,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":7687,"spinlock_wait_cycles":58031488,"thread_start_us":213984,"threads_started":1,"wal-append.queue_time_us":397251}
I20260709 12:27:12.920521  6773 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:12.923120  5866 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:12.867697 11560 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 12:27:12.868597 11570 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:27:12.925995  5874 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:12.845317 11549 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:12.876850 11578 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.827600 11509 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:12.930078  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:12.864261 11427 raft_consensus.cc:697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:12.906272 11540 consensus_queue.cc:1243] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:12.876513 11534 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:12.846012 11462 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:12.870494 11464 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:12.932425  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:12.839092 11490 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:12.875439 11523 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:12.937671  5869 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:12.913888 11535 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:12.946995  5868 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:12.947827  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:12.955904  5874 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:12.933990  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:12.845721 11511 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:12.953073  5869 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:12.933037 11368 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:12.960647  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:12.962005  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.956s	user 0.007s	sys 0.013s
W20260709 12:27:12.962976  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.007s	sys 0.013s
I20260709 12:27:12.963245 11574 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
W20260709 12:27:12.963985  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 965165us
I20260709 12:27:12.961913 11368 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:12.916451 11473 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:12.967290  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:12.976933  5874 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:12.973392  6766 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:12.956631 11200 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:12.981268  5840 leader_election.cc:400] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14.
I20260709 12:27:12.962302 11566 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:12.983533  5840 leader_election.cc:403] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:27:12.985103  5915 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:12.987783  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:12.963106 11576 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:12.990926  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:12.996941  6773 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:12.991591  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:12.993517  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:12.986330 11566 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.999457 11532 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:12.999562 11566 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:12.983532 11574 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:12.976809 11368 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:13.001055  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:12.958264 11069 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:13.002940  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:13.004154  6756 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:13.004956  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:13.008944  5940 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:13.005620  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:12.938764 11578 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:12.989563 11534 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:13.017009  5868 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:27:13.012722  5947 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:13.023059  6756 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:13.032886  6773 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:13.001204 11504 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:13.035836  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:13.007625 11464 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:13.007969 11532 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:13.020270 11436 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:13.070845  6773 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:13.027470 11584 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 12:27:13.078030 11574 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:13.078733 11586 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:13.078676 11574 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:13.083797  5940 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:13.011291 11360 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:13.094354  6020 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:27:13.100934  5840 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:13.027460 11576 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:27:13.056430 11535 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.020083 11156 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:13.103016  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:13.104980  6021 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:27:13.052340 11565 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:13.092702  5838 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:13.110735  5947 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:27:13.058661 11534 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:13.080850 11504 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:27:13.125490  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.079517 11586 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:13.128433  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 125600us
I20260709 12:27:13.109493 11579 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:13.131033  6022 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:13.076047 11540 consensus_queue.cc:1059] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:27:13.025815 11427 consensus_queue.cc:237] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:13.132911  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:13.140323  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:13.113842 11473 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:13.118687 11565 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.147804  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:13.129683 11572 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.220s	user 0.002s	sys 0.000s
I20260709 12:27:13.143662  5915 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:13.145591  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:13.151163  6021 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:13.148638  6022 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:13.111292 11584 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:27:13.139226 11579 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.157773 11585 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 13. Current term is 14.
W20260709 12:27:13.158633 11581 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 12:27:13.161034  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.146620 11532 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:27:13.171607  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 79046.5us
I20260709 12:27:13.120694 11535 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:13.137562 11503 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:13.135401 11504 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:13.175680 11589 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:13.178668  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:13.179069 11592 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:27:13.180446 11511 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:13.180681 11525 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.986s	user 0.002s	sys 0.001s
I20260709 12:27:13.150472 11565 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:13.186038 11594 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:27:13.189122 11555 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.526s	user 0.016s	sys 0.002s
W20260709 12:27:13.191282 11545 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.010s
I20260709 12:27:13.155846 11509 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:13.162958 11576 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:13.248533  5874 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:13.250479  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:13.198915 11156 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:13.193285 11246 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:27:13.255666  5871 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
W20260709 12:27:13.171684  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68784.4us
I20260709 12:27:13.169198 11579 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:13.197394  6611 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:13.196568 11580 log.cc:927] Time spent T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 12:27:13.181962 11584 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:13.195986 11434 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:13.216166 11574 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:13.202239 11549 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:27:13.214859 11555 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:13.222188 11590 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:27:13.223901 11503 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:13.224264 11561 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.564s	user 0.000s	sys 0.002s
W20260709 12:27:13.230422 11432 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.575s	user 0.010s	sys 0.013s
I20260709 12:27:13.233182 11386 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:13.237121 11578 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:13.200449 11234 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:13.239154 11540 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:13.233209 11535 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:13.242357 11360 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:13.314400 11540 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:13.317112 11540 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:13.317811 11195 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.972s	user 0.012s	sys 0.012s
W20260709 12:27:13.318670 11262 consensus_meta.cc:287] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.947s	user 0.008s	sys 0.015s
I20260709 12:27:13.318426 11195 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:13.319180 11262 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:13.319782 11490 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:13.304207 11434 raft_consensus.cc:3055] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:13.320588 11434 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:13.243016  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:13.244671  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:13.247144 11504 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:13.269161  5866 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
W20260709 12:27:13.269829  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 72625.3us
W20260709 12:27:13.250589 11211 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.761s	user 0.009s	sys 0.006s
W20260709 12:27:13.229516 11541 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.666s	user 0.013s	sys 0.000s
I20260709 12:27:13.325610  5865 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:27:13.351590  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:13.351423  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:13.349213  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:13.356081  5940 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:13.350713  6022 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:13.350394  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:13.278713  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 123738us
I20260709 12:27:13.279572 11597 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:13.201323 11462 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:27:13.260519 11532 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.284693  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:13.296797  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:13.248620 11565 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:13.298280 11584 raft_consensus.cc:697] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:13.318744 11454 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.002s	user 0.013s	sys 0.011s
I20260709 12:27:13.301299 11432 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:13.414183  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1499) took 1171 ms. Trace:
I20260709 12:27:13.414942 11590 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.415746 11590 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:13.311574 11436 raft_consensus.cc:3055] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:13.417869 11436 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:13.418954 11436 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:13.330508  6019 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:13.334451  6756 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:27:13.340708  5869 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:13.351188  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:13.422330  6762 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:13.281283 11473 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:13.292953 11576 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:13.414662  6766 rpcz_store.cc:276] 0709 12:27:12.242864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.258037 (+ 15173us) service_pool.cc:224] Handling call
0709 12:27:12.258497 (+   460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:12.979024 (+720527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.413872 (+434848us) spinlock_profiling.cc:243] Waited 715 ms on lock 0x7b54002a0dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:13.414097 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1643498880}
I20260709 12:27:13.369375  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:13.444499  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:13.445375  6768 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:13.250013 11545 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:13.307355 11593 consensus_queue.cc:1059] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:27:13.321343 11587 raft_consensus.cc:3055] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:13.402745  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:13.403352 11534 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:13.404184 11504 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:13.322193 11234 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:13.321458 11434 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:13.406451 11462 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:13.407227 11565 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:13.408327 11220 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:13.408202 11584 consensus_queue.cc:237] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:13.466216 11234 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:13.466616  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:13.467329 11434 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:13.468319 11579 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:13.471467 11566 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:13.474407 11462 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:13.412185 11361 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:13.298476 11503 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:13.477202  5870 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:13.478963 11503 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:13.321406 11262 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:13.321995 11578 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 12:27:13.423930  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:13.482137  6019 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 14: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:13.425006  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:13.432281  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:13.483124  6015 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:13.483471  6773 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:13.323091 11555 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:13.376274  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 91671.7us
I20260709 12:27:13.446395  5915 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:13.446550  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:13.489470  6768 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:13.489760  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:13.441471  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:13.447567  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:13.447763  6766 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:27:13.491885  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:13.492884  5946 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:13.493520  6766 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:13.494122  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:13.444960  6793 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:13.449906  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:13.442441  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:13.495294  5946 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:13.495393  5866 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:13.495978  6756 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:13.451862 11492 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:13.366891 11541 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:13.329706 11195 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:13.401226 11532 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:13.475461 11462 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:13.481359 11578 consensus_queue.cc:1243] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:27:13.490942  5945 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
W20260709 12:27:13.499495  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 197726us
I20260709 12:27:13.325423 11596 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.148s
I20260709 12:27:13.491708  6778 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:27:13.543605  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:13.545269  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:13.372645 11540 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:13.443125 11454 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:13.500828 11511 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:13.440966 11414 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.518219  5946 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:13.518901  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:13.454972 11601 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 12:27:13.519481 11590 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:13.531026  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:13.562199  5868 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:13.552332 11540 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:13.542945  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:13.506987  6773 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
W20260709 12:27:13.545951  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 48667.2us
I20260709 12:27:13.565217  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:13.546851  6020 raft_consensus.cc:2410] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:27:13.547540  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:13.566608  5945 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:13.567382  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:13.567628  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:13.568241  5945 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:13.569077  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:13.569690  5945 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:13.547892  5866 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:13.548722  5947 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:13.440582 11473 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.555860  5869 raft_consensus.cc:2393] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:13.371235 11211 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:13.453477 11587 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:13.575387  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 129653us
I20260709 12:27:13.554337 11462 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:13.578933  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:13.588562  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:13.594969  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:13.579124  5990 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:13.590298  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:13.595219  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:13.599340  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:13.600073  5866 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:13.600421  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:13.562352 11427 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:13.606020  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1342) took 1899 ms. Trace:
I20260709 12:27:13.596694  5990 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:13.608718  6773 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 12:27:13.576416 11368 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.556s	user 0.012s	sys 0.001s
I20260709 12:27:13.618054  5949 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:13.570350 11509 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.588915  6017 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:13.570165 11566 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:13.607007  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:13.550439 11276 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:27:13.580161 11602 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:13.615711  5945 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:13.729748  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:13.687901  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:13.555591 11541 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:27:13.574975 11417 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:13.582650 11473 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:13.703107  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:13.579591 11593 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:13.580808 11565 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.125s
I20260709 12:27:13.708746 11492 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:13.604653 11386 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:13.557529 11511 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:13.687570  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:13.730552  5945 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:13.599360 11156 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:13.567714 11414 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:13.704504 11608 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:13.742257  5868 raft_consensus.cc:2393] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:13.803766  6015 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
W20260709 12:27:13.828651  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 261319us
I20260709 12:27:13.617394 11549 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.978026 11549 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:13.980834 11549 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:13.606531  6023 rpcz_store.cc:276] 0709 12:27:11.694256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.694440 (+   184us) service_pool.cc:224] Handling call
0709 12:27:11.694879 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:11.820339 (+125460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:11.820376 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:11.820971 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 12:27:12.321361 (+500390us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:12.321409 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.578941 (+1257532us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b540041d554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:13.578998 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:13.579051 (+    53us) raft_consensus.cc:1685] finished
0709 12:27:13.579065 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:13.593573 (+ 14508us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1218,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":4140,"spinlock_wait_cycles":287462656,"thread_start_us":499913,"threads_started":1,"wal-append.queue_time_us":634511}
I20260709 12:27:13.984706  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:13.986649 11200 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:13.987324 11200 raft_consensus.cc:697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:13.988451 11200 consensus_queue.cc:237] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:13.690709 11410 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.994714 11410 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:13.693907 11523 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.919222  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:14.000439  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:14.001895  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:13.726918 11566 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:13.817988  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 211020us
I20260709 12:27:14.003513  5946 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:14.004058 11566 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:14.007723 11566 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:14.010775  6762 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.570s	user 0.013s	sys 0.000s
I20260709 12:27:14.012390  6762 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:13.695583 11368 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:13.897116 11464 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:13.703420 11575 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.276s
I20260709 12:27:13.923727 11276 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:13.719832  5866 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
W20260709 12:27:13.966683  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 370031us
I20260709 12:27:13.741853 11434 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:13.845167  5947 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:27:14.049312  5990 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:14.019101  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.057736  6609 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:13.576128 11576 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:14.073959 11603 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.502s	user 0.000s	sys 0.001s
I20260709 12:27:13.926445 11386 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.705577 11509 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:13.921084 11579 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:13.942049  5945 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:13.887017 11593 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:14.079244  5872 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:14.082672  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:14.020092 11504 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:14.092937  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:14.097470  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:14.102901  5945 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:13.848315 11589 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:13.851049 11417 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:14.042174 11464 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.962041 11156 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:14.048620 11276 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:14.091042 11575 consensus_queue.cc:1243] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 16.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:27:14.036099 11601 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:27:14.088505 11492 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:14.090773 11545 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:14.113201  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:14.095049 11386 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:13.820502 11602 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:14.074932 11523 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:13.849851 11587 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:14.050493 11607 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 12:27:14.104269 11504 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:14.107815 11589 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:14.103709 11541 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:14.111018 11464 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:14.141431 11532 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:14.140547 11565 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:27:14.148319  6756 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.652s	user 0.015s	sys 0.000s
I20260709 12:27:14.199342  6756 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:14.155093  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:14.201892  6022 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:14.147200  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 125654us
I20260709 12:27:14.208498  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:14.209380  5869 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:14.213152  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:14.216032  5868 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:14.216533  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:14.155776 11589 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:14.159487 11587 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:14.212025 11565 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:27:14.238023 11605 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:27:14.238520 11511 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:14.239095  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1770) took 1228 ms. Trace:
W20260709 12:27:14.247085 11606 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:27:14.160251 11414 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:14.148062 11454 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:14.153735 11410 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:14.155756 11211 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:14.147418 11549 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:14.252395  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:14.311012  5869 raft_consensus.cc:2410] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 15 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 12:27:14.158854 11541 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:14.184752 11540 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:14.278389  5867 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:14.295022  6022 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:14.290830  5949 rpcz_store.cc:276] 0709 12:27:13.011113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.011242 (+   129us) service_pool.cc:224] Handling call
0709 12:27:13.011562 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:13.618596 (+607034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.238735 (+620139us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b540043f8d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:14.238993 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1394245376}
I20260709 12:27:14.302702 11613 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:14.313333  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:14.303704 11565 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:14.317453  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:14.318445  6018 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:14.315605  5949 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:14.273980 11618 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:14.321425  6611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.023s	sys 0.010s
W20260709 12:27:14.324863  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 116725us
W20260709 12:27:14.326577  6611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.024s	sys 0.010s
I20260709 12:27:14.324743  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:14.168829 11276 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:14.328145  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:14.328637  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:14.328713  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 871214us
I20260709 12:27:14.336591  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:14.337447  5868 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:14.337762  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:14.314502 11611 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:14.347476  6762 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: [16.10-18.11]
I20260709 12:27:14.347743  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:14.349951  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:14.350646  6778 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:14.348131  5944 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.003s	sys 0.011s
I20260709 12:27:14.348692  6775 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
W20260709 12:27:14.351446  5944 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.003s	sys 0.011s
I20260709 12:27:14.351888  6775 raft_consensus.cc:606] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:14.352397  6775 raft_consensus.cc:614] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:14.327682 11540 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.353402  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:14.327975 11368 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:14.318694 11414 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
W20260709 12:27:14.352005 11246 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.000s	sys 0.011s
I20260709 12:27:14.359396  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:14.328236 11625 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.357044  6756 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:14.346680 11509 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:14.373857 11246 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.001s	sys 0.012s
I20260709 12:27:14.354455 11511 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:14.360662 11276 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:14.377290  5868 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:14.380992  5868 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:14.381273  5949 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:14.362661 11541 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:14.394177  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:14.354139 11410 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:27:14.396149  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:14.382193  5868 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:14.399530  5868 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:14.371914 11417 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:14.402376  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:14.374297 11156 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:14.407296  5947 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 14: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:27:14.408138  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 202558us
I20260709 12:27:14.332996 11618 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:14.363587 11540 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:14.412024  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:14.423913  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:14.424690  5947 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:14.424450  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:14.425536  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:14.426820 11619 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.398211 11454 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.397061 11434 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:14.386003 11511 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:14.356657 11611 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:14.420301 11541 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:14.452801  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:14.458966  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:14.461752  5868 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:27:14.462522  5949 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:14.374665 11625 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:14.466102  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:14.463157  5868 pending_rounds.cc:85] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 10
I20260709 12:27:14.471138  5868 pending_rounds.cc:107] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:27:14.471659  5868 raft_consensus.cc:2889] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:14.412978 11464 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:14.480638  5947 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
W20260709 12:27:14.481776 11612 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.267s	user 0.002s	sys 0.000s
I20260709 12:27:14.500946 11386 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:14.483244 11627 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:27:14.480273  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:14.490097  6778 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
W20260709 12:27:14.482483 11626 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 12:27:14.529559  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 49459.4us
I20260709 12:27:14.509223  6018 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:14.442867 11454 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:14.530483  6022 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:27:14.444247 11434 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:14.530817  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1361) took 1072 ms. Trace:
I20260709 12:27:14.530507  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:14.532508  6018 rpcz_store.cc:276] 0709 12:27:13.458359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.317076 (+858717us) service_pool.cc:224] Handling call
0709 12:27:14.530748 (+213672us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:14.427480 11619 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:14.510727 11462 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.909s	user 0.006s	sys 0.007s
I20260709 12:27:14.539814  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:14.540866  6018 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:27:14.542101  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:14.542825  6018 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:14.544015  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:14.544943  6018 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 17: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:14.531522 11414 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:14.555967  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:14.535405 11631 raft_consensus.cc:2764] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 12:27:14.560878 11473 consensus_meta.cc:287] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.664s	user 0.012s	sys 0.002s
W20260709 12:27:14.561450  6611 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.196:34835 timed out after 2.196s (SENT) [suppressed 26 similar messages]
I20260709 12:27:14.568329 11454 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:14.554672 11602 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:14.573482  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:14.579581  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:14.580545  6018 raft_consensus.cc:2368] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19.
I20260709 12:27:14.582695 11611 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:14.562292 11636 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:27:14.589913 11386 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:14.591775 11625 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:14.593214  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1148) took 1158 ms. Trace:
I20260709 12:27:14.594177 11511 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:14.593812  5874 rpcz_store.cc:276] 0709 12:27:13.434401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.434577 (+   176us) service_pool.cc:224] Handling call
0709 12:27:13.459175 (+ 24598us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:13.492344 (+ 33169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:13.492380 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:13.500045 (+  7665us) log.cc:844] Serialized 52 byte log entry
0709 12:27:13.603087 (+103042us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:14.155767 (+552680us) log.cc:844] Serialized 14 byte log entry
0709 12:27:14.156157 (+   390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.156283 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:14.592783 (+436500us) raft_consensus.cc:1685] finished
0709 12:27:14.592830 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:14.593085 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32566,"prepare.queue_time_us":647995,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":39807,"raft.run_cpu_time_us":421,"raft.run_wall_time_us":23909,"thread_start_us":516352,"threads_started":2,"wal-append.queue_time_us":407219}
W20260709 12:27:14.604954 11629 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:27:14.619037  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:14.620779  6018 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:14.622723  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:14.569362 11473 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:14.635426  6785 raft_consensus.cc:2393] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 19: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:14.636868 11473 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:14.565122 11589 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:14.644426 11636 consensus_queue.cc:1243] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:27:14.679796 11607 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:14.637244 11464 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:14.655191 11540 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:14.656963 11619 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:14.727730  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 166271us
I20260709 12:27:14.721118 11613 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:14.714118 11607 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:14.835364 11611 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.835712 11621 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:14.846248 11640 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
W20260709 12:27:14.869741 11641 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
W20260709 12:27:14.890383 11642 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
W20260709 12:27:14.890647 11639 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.310s	user 0.000s	sys 0.001s
I20260709 12:27:14.896909  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1505) took 1408 ms. Trace:
I20260709 12:27:14.897379  5944 rpcz_store.cc:276] 0709 12:27:13.488436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.491054 (+  2618us) service_pool.cc:224] Handling call
0709 12:27:13.502616 (+ 11562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:14.351966 (+849350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:14.352004 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:14.369807 (+ 17803us) log.cc:844] Serialized 26 byte log entry
0709 12:27:14.459469 (+ 89662us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:14.459504 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.459573 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:14.895936 (+436363us) raft_consensus.cc:1685] finished
0709 12:27:14.895960 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:14.895124 (+  -836us) spinlock_profiling.cc:243] Waited 6.92 ms on lock 0x7b48001bc258. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d7bacaa1 00007f39d7ba3b37 00007f39d7ba39ca 00007f39d7c9b26f 00007f39d7c9af37 00007f39d7caf55c 00007f39d7caf508 00007f39d7caf488 00007f39d7caf443 00007f39d7cae6ec 00007f39d6bd8a96
0709 12:27:14.896827 (+  1703us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":74089,"raft.run_cpu_time_us":22640,"raft.run_wall_time_us":1317329,"spinlock_wait_cycles":15905024,"thread_start_us":907693,"threads_started":2,"wal-append.queue_time_us":198693}
I20260709 12:27:14.879544 11611 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:14.882864 11621 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:14.920606  5869 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:14.920750  5868 pending_rounds.cc:85] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 6
I20260709 12:27:14.921131  5868 pending_rounds.cc:107] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:27:14.921388  5868 raft_consensus.cc:2889] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:14.922053  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:14.922724  5869 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:14.924921  5874 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:14.932504  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:14.935328  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:14.937572 11602 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:14.936707 11613 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.936956 11585 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 12:27:14.945212  5869 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:14.947053  6778 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:14.947916  6775 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:27:14.934581 11589 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:14.949347  6756 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:14.941306 11613 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:14.953437  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:14.954304  5874 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:14.955590  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:14.956389  6775 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:14.958057  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:14.958819  6756 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:14.940495 11602 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:14.959535  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:14.958762  6778 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:27:14.966085 11649 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:14.971387  5874 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:27:14.975631  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:14.976436  6756 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
W20260709 12:27:14.967751 11648 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:27:14.983146  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:14.974255 11649 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:14.985442 11602 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:14.991945 11613 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:14.993439 11611 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:14.995957 11504 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.829s	user 0.014s	sys 0.002s
I20260709 12:27:14.993263 11434 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:14.996788 11504 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:14.975315 11621 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:14.998164  5870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1413) took 2098 ms (client timeout 1968 ms). Trace:
W20260709 12:27:14.998847  5870 rpcz_store.cc:269] 0709 12:27:12.899513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.933715 (+ 34202us) service_pool.cc:224] Handling call
0709 12:27:14.997875 (+2064160us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b5400653e54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:14.998032 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1190075136}
W20260709 12:27:15.009460 11647 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:27:15.013646 11635 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:15.014190 11635 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:15.015177  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1366) took 1590 ms. Trace:
I20260709 12:27:15.015242 11635 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:15.016484 11635 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:15.012804  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1849) took 1765 ms. Trace:
I20260709 12:27:15.017797  5872 rpcz_store.cc:276] 0709 12:27:13.246958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.249458 (+  2500us) service_pool.cc:224] Handling call
0709 12:27:13.252422 (+  2964us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:14.100504 (+848082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.012504 (+912000us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b540069ee54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:15.012732 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1900603648}
I20260709 12:27:15.011071 11368 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:27:15.018525 11276 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.626s	user 0.010s	sys 0.003s
I20260709 12:27:15.018893 11368 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:15.019909 11368 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:15.020917  5942 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:15.020931 11368 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:15.015717  6018 rpcz_store.cc:276] 0709 12:27:13.424385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.634804 (+1210419us) service_pool.cc:224] Handling call
0709 12:27:14.635251 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:14.635724 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:14.635766 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:14.636413 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 12:27:14.895668 (+259255us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:14.895713 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.895790 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.014935 (+119145us) raft_consensus.cc:1685] finished
0709 12:27:15.014960 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.015104 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":40676,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":16548,"thread_start_us":258832,"threads_started":1,"wal-append.queue_time_us":288120}
I20260709 12:27:15.028151  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:15.028810  6018 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:15.029953 11534 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:15.030737 11534 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:15.033984  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:15.036223  6775 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:15.036728  6773 raft_consensus.cc:606] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:27:15.037106  6773 raft_consensus.cc:614] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:15.039485  5942 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 12:27:15.040130  5942 pending_rounds.cc:85] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 7
I20260709 12:27:15.041203  5942 pending_rounds.cc:107] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:27:15.041579  5942 raft_consensus.cc:2889] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:15.042799 11622 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.043468  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1057) took 3184 ms. Trace:
I20260709 12:27:15.043596 11622 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:15.044021  5942 rpcz_store.cc:276] 0709 12:27:11.858538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.862468 (+  3930us) service_pool.cc:224] Handling call
0709 12:27:11.862801 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:13.605164 (+1742363us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b18003b3700. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2cbe646 00007f39d2bf858e 00007f39d2c79f9e 00007f39d2c7b7de 00007f39d2c85b49 00007f39d2c86416 00007f39d2c82d8f 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4
0709 12:27:15.020371 (+1415207us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b580057e6d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bdf0aa 00007f39d2c61447 00007f39d2c79fe7 00007f39d2c7b7de 00007f39d2c85b49 00007f39d2c86416 00007f39d2c82d8f 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4
0709 12:27:15.042509 (+ 22138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.043356 (+   847us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4515622400}
I20260709 12:27:15.046238 11534 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:15.046937 11649 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:15.049443  6018 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
W20260709 12:27:15.056033  6017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1338) took 3377 ms (client timeout 2196 ms). Trace:
I20260709 12:27:15.056318  6021 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:15.056527  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:15.058094  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.058957  6024 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:15.060148  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:15.060878  6024 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:15.061704  6017 rpcz_store.cc:269] 0709 12:27:11.678321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.678543 (+   222us) service_pool.cc:224] Handling call
0709 12:27:15.055698 (+3377155us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b54001711d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:15.055886 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4190068480}
I20260709 12:27:15.063680 11574 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.064492  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.065371  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.065574  6017 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:15.067520  6018 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:15.068400  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:15.069483  6017 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 14: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:15.069190  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.070784  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:15.071260  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:15.072371  6018 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:15.071558  6017 raft_consensus.cc:2376] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16. Re-sending same reply.
I20260709 12:27:15.083319 11538 raft_consensus.cc:3055] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:15.083917 11538 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:15.089299  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1068) took 1660 ms. Trace:
I20260709 12:27:15.089860  5940 rpcz_store.cc:276] 0709 12:27:13.429196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.429437 (+   241us) service_pool.cc:224] Handling call
0709 12:27:13.429985 (+   548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:13.517295 (+ 87310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:13.517516 (+   221us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:13.545030 (+ 27514us) log.cc:844] Serialized 26 byte log entry
0709 12:27:14.015699 (+470669us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:14.015743 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.088843 (+1073100us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b5400407c54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:15.088902 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.088958 (+    56us) raft_consensus.cc:1685] finished
0709 12:27:15.088974 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.089226 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96890,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":10443,"spinlock_wait_cycles":61505280,"thread_start_us":488139,"threads_started":2,"wal-append.queue_time_us":561764}
I20260709 12:27:15.104225  6018 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:15.064410 11574 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:15.107149  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1504) took 1619 ms. Trace:
I20260709 12:27:15.110751  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.084931 11538 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:15.109743  5948 rpcz_store.cc:276] 0709 12:27:13.488101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.490364 (+  2263us) service_pool.cc:224] Handling call
0709 12:27:13.490773 (+   409us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:14.358630 (+867857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:14.358681 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:14.361376 (+  2695us) log.cc:844] Serialized 52 byte log entry
0709 12:27:14.518656 (+157280us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:14.533617 (+ 14961us) log.cc:844] Serialized 14 byte log entry
0709 12:27:14.561567 (+ 27950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.093371 (+531804us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b54008aff54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:15.093433 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.093489 (+    56us) raft_consensus.cc:1685] finished
0709 12:27:15.093608 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.107050 (+ 13442us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29577,"prepare.queue_time_us":14973,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":101647,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":9497,"spinlock_wait_cycles":1261823488,"thread_start_us":390635,"threads_started":3,"wal-append.queue_time_us":222691}
I20260709 12:27:15.111616 11211 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:15.111851 11617 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.866s	user 0.009s	sys 0.007s
I20260709 12:27:15.109400  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.121659  6017 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:15.123557  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1368) took 1667 ms. Trace:
I20260709 12:27:15.125020  6017 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:15.126271  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1376) took 1652 ms. Trace:
I20260709 12:27:15.126777  6017 rpcz_store.cc:276] 0709 12:27:13.474235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.122575 (+1648340us) service_pool.cc:224] Handling call
0709 12:27:15.122899 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.125993 (+  3094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.126204 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44288}
I20260709 12:27:15.126283  6018 rpcz_store.cc:276] 0709 12:27:13.455788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.114943 (+1659155us) service_pool.cc:224] Handling call
0709 12:27:15.122914 (+  7971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.123140 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:15.123178 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:15.123238 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:15.123283 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.123355 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.123492 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38656}
I20260709 12:27:15.138521 11593 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:15.138909 11574 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:15.142257 11504 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:15.143293 11586 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:15.121517 11538 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:15.146118  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:15.152453  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1500) took 1665 ms. Trace:
W20260709 12:27:15.155391  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.156584  5948 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:15.158051  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:15.158370  6024 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:15.158804  5940 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:15.159672  6024 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:27:15.159750  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:15.160521  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1371) took 1684 ms. Trace:
I20260709 12:27:15.161087  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:15.160862  6024 rpcz_store.cc:276] 0709 12:27:13.475880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.159076 (+1683196us) service_pool.cc:224] Handling call
0709 12:27:15.159428 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:15.160240 (+   812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.160464 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:15.164050  6024 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:15.164741  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1362) took 1688 ms. Trace:
I20260709 12:27:15.165117  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1370) took 1689 ms. Trace:
I20260709 12:27:15.165114  6024 rpcz_store.cc:276] 0709 12:27:13.476749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.163546 (+1686797us) service_pool.cc:224] Handling call
0709 12:27:15.163864 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.164494 (+   630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.164688 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:15.165771  6018 rpcz_store.cc:276] 0709 12:27:13.475599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.150387 (+1674788us) service_pool.cc:224] Handling call
0709 12:27:15.164548 (+ 14161us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.164741 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:15.164770 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:15.164816 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:15.164898 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.164952 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.165055 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:15.166350  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:15.167676  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.152980  5943 rpcz_store.cc:276] 0709 12:27:13.486756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.510291 (+ 23535us) service_pool.cc:224] Handling call
0709 12:27:13.510857 (+   566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:13.690497 (+179640us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:13.975828 (+285331us) log.cc:844] Serialized 14 byte log entry
0709 12:27:15.023986 (+1048158us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b18001ea9c0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:27:15.107220 (+ 83234us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:15.110509 (+  3289us) log.cc:844] Serialized 26 byte log entry
0709 12:27:15.110689 (+   180us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:15.110725 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.110822 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.147543 (+ 36721us) raft_consensus.cc:1685] finished
0709 12:27:15.152186 (+  4643us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.152383 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97810,"prepare.queue_time_us":325690,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":67891,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":19298,"spinlock_wait_cycles":4907392,"thread_start_us":261936,"threads_started":2,"wal-append.queue_time_us":1084589}
I20260709 12:27:15.173075 11593 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:15.176537  6024 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:27:15.177158  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 110582us
I20260709 12:27:15.178591  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1363) took 1701 ms. Trace:
I20260709 12:27:15.179071  6024 rpcz_store.cc:276] 0709 12:27:13.477122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.166081 (+1688959us) service_pool.cc:224] Handling call
0709 12:27:15.176085 (+ 10004us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.178272 (+  2187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.178527 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:15.176923 11593 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:15.181105  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1368) took 1615 ms. Trace:
I20260709 12:27:15.181546  6024 rpcz_store.cc:276] 0709 12:27:13.566033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.179948 (+1613915us) service_pool.cc:224] Handling call
0709 12:27:15.180504 (+   556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.180735 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:15.180760 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:15.180806 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:15.180845 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.180904 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.181040 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:15.182359 11593 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:15.183230  6024 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:15.173447 11262 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:15.184051  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1379) took 1575 ms. Trace:
I20260709 12:27:15.184448 11262 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:15.184484  6024 rpcz_store.cc:276] 0709 12:27:13.608350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.182622 (+1574272us) service_pool.cc:224] Handling call
0709 12:27:15.183006 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.183752 (+   746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.183989 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:15.183092 11587 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:15.188392 11504 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.189185 11504 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:15.189731 11262 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:15.178335  5838 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:15.192677 11262 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:15.194108  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:15.193287 11262 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:15.195963 11262 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:15.200431  6022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1364) took 1723 ms (client timeout 2184 ms). Trace:
I20260709 12:27:15.202204 11504 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:15.203961  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:15.204814 11622 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:15.200984  6022 rpcz_store.cc:269] 0709 12:27:13.477395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.328316 (+850921us) service_pool.cc:224] Handling call
0709 12:27:15.200233 (+871917us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54001725d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:15.200368 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456912640}
I20260709 12:27:15.174913 11586 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:15.207926 11604 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 1.469s	user 0.008s	sys 0.005s
I20260709 12:27:15.208115  6022 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:27:15.208271 11555 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:27:15.212076  6022 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:27:15.175367 11368 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:15.214080 11368 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:15.215941  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1361) took 1842 ms. Trace:
I20260709 12:27:15.216531  6021 rpcz_store.cc:276] 0709 12:27:13.373321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.373520 (+   199us) service_pool.cc:224] Handling call
0709 12:27:15.055915 (+1682395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.056900 (+   985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.215676 (+158776us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54001711d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:15.215870 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3713988992}
I20260709 12:27:15.238050  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:15.239413  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:15.245505  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:15.248327  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:15.275096  5945 raft_consensus.cc:2410] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:27:15.251098  5948 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:27:15.254643  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.255610  5874 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:15.230038 11622 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.258347  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:15.277989  6609 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:15.260530  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:15.262607 11211 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:15.265834  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:15.282455  5871 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:15.286574  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.266670 11368 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:15.267480  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:15.269285  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:15.251965 11234 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:15.295392  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:15.302017  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1369) took 1826 ms. Trace:
I20260709 12:27:15.350224  5870 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:27:15.350914  5870 pending_rounds.cc:85] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 10
W20260709 12:27:15.350740  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 55456.6us
I20260709 12:27:15.351217  5870 pending_rounds.cc:107] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:27:15.351678  5870 raft_consensus.cc:2889] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:15.351938  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:15.356244  5871 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:15.281533 11613 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:15.358547  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:15.354139  6017 rpcz_store.cc:276] 0709 12:27:13.475305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.127782 (+1652477us) service_pool.cc:224] Handling call
0709 12:27:15.129691 (+  1909us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.129936 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:15.229176 (+ 99240us) log.cc:844] Serialized 14 byte log entry
0709 12:27:15.301352 (+ 72176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:15.301444 (+    92us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:15.301521 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.301652 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.301917 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":40090,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":837,"raft.queue_time_us":1104,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":8128,"thread_start_us":169402,"threads_started":2}
I20260709 12:27:15.345502 11601 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:15.363057 11601 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:15.353895 11195 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:15.366202  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1501) took 2035 ms. Trace:
I20260709 12:27:15.366664 11601 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:15.366925 11600 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.797s	user 0.000s	sys 0.021s
I20260709 12:27:15.345479 11555 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.358807  5947 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 21: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:15.354666 11556 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:15.370649  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:15.366740  6757 rpcz_store.cc:276] 0709 12:27:13.330714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.349613 (+ 18899us) service_pool.cc:224] Handling call
0709 12:27:13.350012 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:13.476510 (+126498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:13.476541 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:13.476594 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:13.476631 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.365907 (+1889276us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54002a21d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:15.365958 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.366133 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":290332032}
I20260709 12:27:15.370836  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:15.355829 11131 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:15.372970  6773 raft_consensus.cc:606] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:27:15.281271 11473 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.374841 11131 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:15.379961 11131 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:15.377166 11503 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:15.374989 11433 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:15.384531  5943 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757"
dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:48330
I20260709 12:27:15.385288  5943 raft_consensus.cc:606] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:15.385109 11503 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:15.385897  5943 raft_consensus.cc:614] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:15.386672 11503 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:15.387394 11523 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:15.390421  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:15.391151  6773 raft_consensus.cc:606] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:15.391521  6773 raft_consensus.cc:614] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:15.365034 11262 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:15.385183 11433 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:15.354805 11625 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:27:15.353547 11234 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:15.361027 11571 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:27:15.360647 11613 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:15.293226 11622 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:15.399910  5874 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:15.395884 11473 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:15.372324 11555 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:15.406662  5865 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1410) took 2504 ms (client timeout 1644 ms). Trace:
W20260709 12:27:15.407847  5865 rpcz_store.cc:269] 0709 12:27:12.898655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.905040 (+  6385us) service_pool.cc:224] Handling call
0709 12:27:15.403276 (+2498236us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b5400210a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:15.403413 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":964220800}
I20260709 12:27:15.409727 11368 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.410293 11368 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:15.410940  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:15.411959 11574 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.412472 11368 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:15.413117 11368 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.412611 11574 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:15.413769 11587 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:15.414688 11587 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:15.415076 11574 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:15.415614 11587 raft_consensus.cc:3055] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:15.416095 11587 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:15.415838 11368 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:15.412657  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:15.423022  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:15.423743  6773 raft_consensus.cc:606] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:27:15.425160  5944 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:15.423604  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:15.423473  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1515) took 1518 ms. Trace:
I20260709 12:27:15.425833  5947 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:15.425992  5946 rpcz_store.cc:276] 0709 12:27:13.905363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.004307 (+ 98944us) service_pool.cc:224] Handling call
0709 12:27:14.004588 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:14.016149 (+ 11561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:14.016189 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:14.016252 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:14.016288 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.423244 (+1406956us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b54006770d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:15.423289 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.423418 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25913728}
I20260709 12:27:15.429993 11234 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:15.430193 11262 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:15.417157 11587 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:15.432296 11555 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:15.432430 11587 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:15.433179 11586 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:15.435007 11659 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
W20260709 12:27:15.438057  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.440179  6773 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:15.440716  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:15.441448  5947 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:15.442494  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.445029  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:27:15.446131 11433 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:15.447921  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.453585  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:15.454519  5946 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:15.454957  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.455730  6773 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:15.457607  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1381) took 1130 ms. Trace:
I20260709 12:27:15.456668  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.464067  6757 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:27:15.458340  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.458941  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.460310  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.461189  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.461647  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.462432  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.462709  5943 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:27:15.469182  5947 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:15.463795  6024 rpcz_store.cc:276] 0709 12:27:14.327701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.185813 (+858112us) service_pool.cc:224] Handling call
0709 12:27:15.186279 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:15.203390 (+ 17111us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:15.203433 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:15.204080 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 12:27:15.359611 (+155531us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:15.373004 (+ 13393us) log.cc:844] Serialized 14 byte log entry
0709 12:27:15.373278 (+   274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.373349 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.457360 (+ 84011us) raft_consensus.cc:1685] finished
0709 12:27:15.457381 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.457538 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15306,"prepare.queue_time_us":13868,"raft.queue_time_us":23210,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":3361,"spinlock_wait_cycles":2645248,"thread_start_us":155029,"threads_started":1,"wal-append.queue_time_us":194482}
I20260709 12:27:15.471868  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.463313 11584 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.466796  6785 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:15.466574 11613 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:15.478243 11622 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:15.484472  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.484136  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.485530  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1776) took 2109 ms. Trace:
I20260709 12:27:15.485864  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.489966  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.490747  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.490958  6017 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:15.491720  6775 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 11: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:15.492559 11584 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:15.479205 11607 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:15.497452  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:15.483390  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.501128 11618 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:15.483543  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.502485  6778 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:27:15.502862  6021 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 22: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:15.503886  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.485919  5941 rpcz_store.cc:276] 0709 12:27:13.376332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.376545 (+   213us) service_pool.cc:224] Handling call
0709 12:27:13.377010 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:13.571460 (+194450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:13.571516 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:13.588248 (+ 16732us) log.cc:844] Serialized 26 byte log entry
0709 12:27:14.098342 (+510094us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:14.098457 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.101147 (+1002690us) spinlock_profiling.cc:243] Waited 6.13 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:15.485210 (+384063us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b540058f554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:15.485262 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.485309 (+    47us) raft_consensus.cc:1685] finished
0709 12:27:15.485321 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.485471 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":167057,"raft.queue_time_us":42903,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":82042,"spinlock_wait_cycles":426026496,"thread_start_us":376986,"threads_started":1,"wal-append.queue_time_us":565554}
W20260709 12:27:15.503536  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.506588 11607 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.507102 11607 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:15.468708  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:15.505314  6020 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:15.510675  5838 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:15.511540 11587 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:15.511974 11574 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:15.512487  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1365) took 2034 ms. Trace:
I20260709 12:27:15.512702 11574 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.509413 11607 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:15.512045 11587 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.513712  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:15.514204 11587 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:15.514612 11490 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.515368 11490 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:15.516731 11622 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.517380 11211 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:15.517333 11622 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:15.518010 11262 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:15.518108 11211 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:15.520666 11211 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:15.513027  6018 rpcz_store.cc:276] 0709 12:27:13.477698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.166832 (+1689134us) service_pool.cc:224] Handling call
0709 12:27:15.178075 (+ 11243us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:15.178523 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:15.178553 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:15.179431 (+   878us) log.cc:844] Serialized 26 byte log entry
0709 12:27:15.419449 (+240018us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:15.470382 (+ 50933us) log.cc:844] Serialized 14 byte log entry
0709 12:27:15.470631 (+   249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.470695 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.512251 (+ 41556us) raft_consensus.cc:1685] finished
0709 12:27:15.512277 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.512402 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3767,"prepare.queue_time_us":48668,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":20061,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":16681,"spinlock_wait_cycles":6372480,"thread_start_us":183144,"threads_started":1,"wal-append.queue_time_us":246124}
I20260709 12:27:15.514501  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.525056  5872 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 17. Ops: []
I20260709 12:27:15.418159 11368 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:15.527266  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:15.527611  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.543473  6768 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:15.512027  5838 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:15.528748  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.547895  6793 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:15.513393 11574 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:15.525147  6024 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:15.550141  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:15.520696 11622 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:15.532461  6775 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:15.533077  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:15.533406  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.534145  5867 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:27:15.561753  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:15.563436  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:15.564450  5867 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:15.535266 11490 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:15.536264  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:15.566444  5870 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:15.537248 11584 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:15.538420  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.542151  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.570729  6019 raft_consensus.cc:2393] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:27:15.546896  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 176311us
W20260709 12:27:15.551210  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.571913  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:15.572275  5915 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:15.555678  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:15.575449  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:15.575482  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:15.576921 11360 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:15.577518 11360 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:15.577747  5867 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:15.581048  6611 leader_election.cc:400] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19.
I20260709 12:27:15.581379  6611 leader_election.cc:403] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:15.582286  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:15.582824  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.578464 11360 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:15.597544  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.596024  6611 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:15.585561  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:15.599916  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.600440  6611 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:15.601024  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.591917  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.601281  6611 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:15.593272  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.542044 11211 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:27:15.594161 11631 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:15.603538 11510 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:15.604179 11510 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.604641 11510 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:15.607764 11473 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:15.608597 11473 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:15.609304 11473 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:15.611579  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.610613 11473 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:15.630429  5870 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:27:15.622216  5915 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:15.601819  6793 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:15.628386  6768 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:15.579391 11522 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:15.597201 11490 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 19
W20260709 12:27:15.631778  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 168062us
I20260709 12:27:15.634378  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:15.638412  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:15.645680  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.597143 11504 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:15.646636  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.647336  6762 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
W20260709 12:27:15.653499  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.649047 11522 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:15.659116  6775 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:15.636097 11607 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:15.642455 11635 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:15.599073 11178 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:27:15.645283 11211 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:15.647022 11504 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.668159  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.631872 11587 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:15.678426 11601 consensus_queue.cc:1059] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:27:15.682605  6775 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:15.683179 11601 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:15.664585 11607 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:15.666635 11667 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:15.666853 11635 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:15.677446 11504 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:15.652863 11622 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:15.672955  6768 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
W20260709 12:27:15.643303  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 365454us
I20260709 12:27:15.696512  6609 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:15.683552 11178 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:15.608583  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:15.692936  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:15.698033  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 89550.4us
I20260709 12:27:15.690644 11607 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:15.700481 11178 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:15.700801 11433 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:15.705377 11433 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:15.705812 11433 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:15.710796 11178 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:15.676880 11211 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:15.734869  5874 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 12:27:15.735365  5874 pending_rounds.cc:85] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 8
I20260709 12:27:15.735613  5874 pending_rounds.cc:107] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:27:15.735467  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.735858  5874 raft_consensus.cc:2889] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:15.671554  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.740657  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:15.741926  5868 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:15.741969  5840 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:15.685665 11131 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:15.748202  5840 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:15.758493  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 49264.6us
W20260709 12:27:15.759469  5987 connection.cc:256] RPC call timeout handler was delayed by -1.1926127229999111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1968s for a response
I20260709 12:27:15.774672  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:15.779826  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.728652 11622 raft_consensus.cc:3055] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:15.694851 11667 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:15.750993 11586 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.773006 11262 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:15.795665 11586 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:15.722283 11522 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:15.798959 11586 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:15.785248 11635 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:15.799427 11667 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:15.746176 11433 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:15.791795 11587 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:15.784411 11622 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:15.794255 11131 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:15.782718 11432 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.805490  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:15.803108  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:15.833896  5944 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:15.849255  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:15.850160  6778 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:15.839496 11668 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:15.854437 11668 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.854846 11668 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:15.854586 11622 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:15.856292 11622 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:15.834991 11432 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:15.858496 11211 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:15.859858 11432 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:15.867447  6762 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15. Ops: []
W20260709 12:27:15.870580  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 223567us
I20260709 12:27:15.815234 11574 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:15.899032  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 124475us
I20260709 12:27:15.900518  5840 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:15.904055  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:15.929360  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:15.930352  6768 raft_consensus.cc:2376] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16. Re-sending same reply.
I20260709 12:27:15.875020  6778 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
W20260709 12:27:15.877861  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:15.935452  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 239052us
W20260709 12:27:15.877179  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 242907us
W20260709 12:27:15.883973  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 282711us
I20260709 12:27:15.889014  5944 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:15.940397  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:15.956820  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:15.984372  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:16.000326  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:15.934849 11262 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:15.896605 11211 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 12:27:15.947863 11432 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.012656  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:15.949265 11674 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:16.018779  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:16.034926  6762 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:27:16.040470  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 154938us
I20260709 12:27:16.005658 11667 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:16.006385 11131 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:16.047617  5915 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:15.930186 11631 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:16.054705  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:16.010982 11522 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:16.015084 11574 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:15.999861 11625 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:27:16.062410 11368 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:16.011360 11503 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
W20260709 12:27:16.118542  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 181900us
I20260709 12:27:16.044132 11262 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:16.060911 11667 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:16.016729 11584 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:16.036669 11432 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:16.136351  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:16.138983  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 238597us
I20260709 12:27:16.152810  6778 raft_consensus.cc:2376] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20. Re-sending same reply.
I20260709 12:27:16.115610 11522 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:16.154698  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:16.121811 11460 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:27:16.157195  5840 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:16.123400 11500 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:27:16.173947  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:16.159354 11490 consensus_meta.cc:287] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.005s
I20260709 12:27:16.148303 11667 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:16.151413 11584 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:16.164317 11685 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:27:16.163942 11433 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:16.176338 11211 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:16.174692 11586 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:16.159986 11587 consensus_queue.cc:1059] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:27:16.179209 11679 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:27:16.221912  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 216938us
W20260709 12:27:16.217922 11673 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.283s	user 0.002s	sys 0.000s
W20260709 12:27:16.233517 11672 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.305s	user 0.001s	sys 0.001s
I20260709 12:27:16.179133  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:16.221043 11694 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.188115 11490 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19.
I20260709 12:27:16.251269  5942 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 12:27:16.250907  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 71912.3us
W20260709 12:27:16.218776  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 278508us
I20260709 12:27:16.253309  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:16.255069  5944 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:16.219641  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 268513us
W20260709 12:27:16.233436 11688 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:27:16.217011 11677 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.221390 11211 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:16.261682  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:16.249534  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:16.250241  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:16.223186 11586 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:16.253787  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:16.241658 11675 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.235s	user 0.001s	sys 0.001s
I20260709 12:27:16.273654 11490 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:16.274411 11490 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:16.259534  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:16.187821 11584 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:16.274880 11490 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:16.265501 11211 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:16.245323 11686 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:27:16.242203 11680 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.268900  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:16.271034  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:16.271291  5949 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:16.282164  6019 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:16.273414  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:16.227422  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 179830us
I20260709 12:27:16.177868 11631 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:16.280885  6773 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:16.304946  5946 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:16.288846  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 237186us
I20260709 12:27:16.246966 11681 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:16.282716 11695 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:16.379824 11695 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:16.380889 11695 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:16.269730 11677 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:16.387871 11607 consensus_meta.cc:287] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.655s	user 0.011s	sys 0.003s
I20260709 12:27:16.350199 11587 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:27:16.388460 11607 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:16.322830 11680 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:16.391974 11680 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:16.392072 11689 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:27:16.299453 11674 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:16.393977 11684 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:27:16.363076 11699 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.393107 11718 raft_consensus.cc:3055] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:16.395119 11718 raft_consensus.cc:740] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:16.394712 11699 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:16.396100 11718 consensus_queue.cc:260] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:16.397495 11718 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:16.398643  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.399621 11705 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:16.400200 11705 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:16.404381 11432 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:16.405547 11667 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:16.407186 11706 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:16.407814 11706 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:16.414409 11368 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:16.414970 11368 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:16.415351 11667 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:16.416455 11504 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.715s	user 0.012s	sys 0.000s
I20260709 12:27:16.416970 11504 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:16.231449 11262 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:16.418099 11510 consensus_meta.cc:287] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.813s	user 0.006s	sys 0.010s
I20260709 12:27:16.418517 11510 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:16.313289 11436 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:16.314386  5990 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:16.316476  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 138149us
I20260709 12:27:16.424499  5840 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:16.308715  5944 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:16.231324 11679 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:16.435879  5944 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:27:16.323009  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:16.249549 11710 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:16.285851 11509 raft_consensus.cc:993] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: : Instructing follower 6add331326ca47038308a447bf798fa7 to start an election
I20260709 12:27:16.306062  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:16.356824  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 45586.6us
I20260709 12:27:16.260453 11696 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 12:27:16.260291 11694 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:16.235117 11522 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:16.361931 11709 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:16.281512 11586 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:16.425769  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:16.465032  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:16.440763  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 126611us
W20260709 12:27:16.447710  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.450984  5946 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:16.397320 11699 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:16.420619 11683 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.304s	user 0.001s	sys 0.001s
W20260709 12:27:16.455127  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 149125us
I20260709 12:27:16.455936  5913 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:16.466598  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:16.468765  5990 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:16.482051  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:16.479172  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:16.470458  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 204434us
I20260709 12:27:16.486479  5990 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:16.481803  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.453269 11703 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:16.506989  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51277.2us
I20260709 12:27:16.454033 11710 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:16.454172 11707 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:16.466557 11510 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:16.465447 11713 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:16.498502 11571 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:16.498235 11625 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:16.496116 11714 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:16.505322 11677 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:16.499194 11178 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:16.506529 11703 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:16.517547 11678 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:16.561334  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:16.517719 11710 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:16.562078  6022 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:16.562822  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 74192.8us
I20260709 12:27:16.563237  6785 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:16.554665  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:16.556914  6611 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:16.557323  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:16.532753 11713 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.558393  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:16.558775  6020 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:16.575070  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:16.575924  6778 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:16.577150  6775 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:16.549328 11714 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:16.582721  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:16.557587 11262 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:16.513227 11433 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:27:16.557018  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:16.594498  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:16.562419 11625 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:16.572937 11713 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:16.540970 11234 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:27:16.565552 11703 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:16.567847  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:16.531953 11361 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.002s	sys 0.015s
I20260709 12:27:16.568001 11706 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:16.571094 11574 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:16.566680 11584 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:16.571295 11571 raft_consensus.cc:697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:16.579249 11697 raft_consensus.cc:1081] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Signalling peer 6add331326ca47038308a447bf798fa7 to start an election
W20260709 12:27:16.618417 11361 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.002s	sys 0.015s
I20260709 12:27:16.642582 11361 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:16.642992 11361 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:16.643824 11361 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:16.592439 11668 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.737s	user 0.015s	sys 0.000s
W20260709 12:27:16.659380 11682 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.543s	user 0.006s	sys 0.000s
I20260709 12:27:16.647215 11700 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:16.661152 11700 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.661605 11700 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:16.663971 11509 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.664633 11509 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:16.666932 11509 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:16.669938 11694 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:16.671406  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 204897us
W20260709 12:27:16.680579  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 97957.3us
I20260709 12:27:16.681950  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:16.604652  6022 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:16.584400 11705 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:16.532186 11674 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:16.584349 11504 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:16.583959 11714 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:16.626765 11622 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.770s	user 0.016s	sys 0.000s
I20260709 12:27:16.627754 11625 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:16.630573 11717 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:27:16.565483 11178 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:16.586903 11613 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:27:16.638208 11574 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.638799 11584 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:16.750213 11262 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:16.751250 11262 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:16.751961 11584 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:16.640295 11571 consensus_queue.cc:237] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:16.685740  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:16.753274  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.754143 11676 raft_consensus.cc:993] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: : Instructing follower 6add331326ca47038308a447bf798fa7 to start an election
I20260709 12:27:16.754318  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:16.755177 11676 raft_consensus.cc:1081] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Signalling peer 6add331326ca47038308a447bf798fa7 to start an election
I20260709 12:27:16.760630 11703 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:16.763628 11703 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:16.765832  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:16.766848  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 1319) took 1169 ms. Trace:
I20260709 12:27:16.767230  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:16.767452  6773 rpcz_store.cc:276] 0709 12:27:15.597104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.597290 (+   186us) service_pool.cc:224] Handling call
0709 12:27:16.766606 (+1169316us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b5400005354. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:16.766755 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1569429376}
I20260709 12:27:16.692828 11195 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:16.646224 11668 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:16.645598 11361 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:16.703408  5840 connection.cc:256] RPC call timeout handler was delayed by -1.2144676010002513s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2141s for a response
I20260709 12:27:16.652884 11555 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:16.706475  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 103053us
I20260709 12:27:16.653218 11607 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:16.733180  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 51386.7us
W20260709 12:27:16.785662  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.786245  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:16.788337  6019 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:16.788658  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:16.647007 11712 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.086s
W20260709 12:27:16.781082  5840 connection.cc:256] RPC call timeout handler was delayed by -1.101067601000068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2267s for a response
I20260709 12:27:16.736539  5947 raft_consensus.cc:2410] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:27:16.690429 11276 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 12:27:16.790428  6017 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:16.795778  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:16.796924  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:16.739228 11701 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:16.800673  6019 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19"
dest_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:34888
I20260709 12:27:16.789817  6021 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32"
dest_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:34888
I20260709 12:27:16.742744 11705 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:16.774761 11574 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:16.815919  6021 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:16.823693  6021 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:16.823807  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:16.799300  6756 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:16.816932  6611 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:16.735246 11697 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 12:27:16.781679 11703 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.829903  6773 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:16.800526 11613 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:16.836061  6773 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:16.835587  6019 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:27:16.824786 11211 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.004s
I20260709 12:27:16.840440  6019 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:16.839361  6773 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:16.813162 11679 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:16.825031 11584 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.847576  6021 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:16.817683 11473 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:16.826572 11538 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.848308  6773 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:16.831182 11680 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:16.800707 11555 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:16.883322  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 63670.5us
I20260709 12:27:16.884639  6609 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:16.887818  6021 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:16.833860 11705 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:16.900876 11586 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:16.889994 11680 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:16.906270  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 105912us
I20260709 12:27:16.898195 11368 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:16.905694  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:16.848721 11703 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:16.902551 11698 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.665s	user 0.003s	sys 0.000s
I20260709 12:27:16.853899 11211 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:16.915726  6019 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:16.922492  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:16.858654 11701 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:27:16.923069  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:16.930039  5870 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:16.932164  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:16.933028  5867 raft_consensus.cc:2368] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15.
I20260709 12:27:16.923929  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:16.926229  5874 raft_consensus.cc:2410] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:27:16.923543  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:16.902568 11696 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:16.871552 11538 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:16.941507  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:16.926535  6773 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:16.865123 11584 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:16.949057  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:16.952950  5872 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:16.953418  6609 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:16.954128  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:16.954630 11704 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:16.951016  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:16.951864  5870 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:27:16.959064  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:16.960732  5870 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:27:16.971331  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:16.982501  5940 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:16.985371  5872 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:16.987920  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:16.923509 11522 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:16.994218  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:16.997650  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:16.999784  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:17.002513  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:16.941912 11613 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:17.009721  5840 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:17.011663  5867 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:27:17.014900  5867 pending_rounds.cc:85] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 8
I20260709 12:27:17.015280  5867 pending_rounds.cc:107] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:27:17.015343  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1880) took 1075 ms. Trace:
I20260709 12:27:17.015550  5867 raft_consensus.cc:2889] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:17.004832 11178 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:16.982156 11704 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.015916  5868 rpcz_store.cc:276] 0709 12:27:15.939837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.941554 (+  1717us) service_pool.cc:224] Handling call
0709 12:27:15.942112 (+   558us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:15.948931 (+  6819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:15.948966 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:16.013983 (+ 65017us) log.cc:844] Serialized 26 byte log entry
0709 12:27:16.158631 (+144648us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:16.158703 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.158807 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:16.901924 (+743117us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:17.007667 (+105743us) raft_consensus.cc:1685] finished
0709 12:27:17.007695 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:17.015247 (+  7552us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6221,"raft.queue_time_us":6645,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":24957,"spinlock_wait_cycles":5451520,"thread_start_us":95147,"threads_started":1,"wal-append.queue_time_us":153514}
I20260709 12:27:16.931273 11473 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.070s
W20260709 12:27:17.017264 11490 consensus_meta.cc:287] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.742s	user 0.014s	sys 0.002s
W20260709 12:27:17.029572  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.012642 11695 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:17.022651  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:17.032296  5868 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:17.038313  5868 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:17.038971 11719 raft_consensus.cc:2764] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:27:17.039283 11728 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.039513 11724 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:17.001650 11522 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:17.067893 11635 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:17.033349 11510 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:17.035908 11695 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:17.090754  5874 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:17.096284 11724 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 14 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:17.097661 11724 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:17.078423 11727 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:27:17.062094 11668 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:17.044102 11490 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:17.044965 11676 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 12:27:17.087486 11703 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:17.093402 11695 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:17.022032 11680 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:17.071296 11613 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:27:17.034196 11704 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:17.114108 11262 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:17.140928 11681 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.148553 11713 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:17.152398 11704 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:17.154714 11490 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:17.164294  5873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 1874) took 1655 ms (client timeout 1804 ms). Trace:
I20260709 12:27:17.079344 11728 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:17.168836  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 660573us
I20260709 12:27:17.124572 11555 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:17.175767  5873 rpcz_store.cc:269] 0709 12:27:15.508705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.532739 (+ 24034us) service_pool.cc:224] Handling call
0709 12:27:17.163991 (+1631252us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b540083f254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:17.164205 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1008608000}
W20260709 12:27:17.170713 11736 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
W20260709 12:27:17.162262 11700 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.500s	user 0.020s	sys 0.001s
I20260709 12:27:17.169483 11705 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:17.191759 11740 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
W20260709 12:27:17.193082 11744 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:27:17.198336 11734 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:27:17.198724  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:17.328353  5942 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:17.201480 11694 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.202247 11678 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:17.202486 11745 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 12:27:17.203155  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 448901us
I20260709 12:27:17.203649 11607 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:17.214193 11522 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:17.169870 11586 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:17.198385  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:17.215620  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:17.363909  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:17.216661  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:17.368013  5945 raft_consensus.cc:2393] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:17.217667 11707 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.284672  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:17.370563  5944 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
W20260709 12:27:17.285408  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.372387  5946 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:17.340518  5872 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
W20260709 12:27:17.346686  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 337035us
I20260709 12:27:17.178277 11211 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:17.375625  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:17.375998  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:17.366096  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:17.374250  5840 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:17.376538  5944 raft_consensus.cc:2368] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
I20260709 12:27:17.174067 11681 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:17.376538  5941 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:17.377952  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:17.377252  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:17.378598  5945 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:17.378273  5946 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:17.378937  5943 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:17.289098 11538 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:17.352746  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:17.383322  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:17.395084  5943 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 19: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:17.411957  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:17.410028  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:17.416285  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:17.421365 11622 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:17.425990  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:17.426882  5873 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:17.328708 11700 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:17.394418 11674 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:17.459976  5915 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:17.462901  5943 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:17.398540 11631 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:17.470546  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:27:17.414968 11436 consensus_meta.cc:287] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.961s	user 0.015s	sys 0.002s
I20260709 12:27:17.351612 11678 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:17.476122  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:17.349776 11694 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:17.478976  5873 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
W20260709 12:27:17.479279  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 84939.9us
I20260709 12:27:17.486781  6016 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:27:17.487232  6016 pending_rounds.cc:85] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 8
I20260709 12:27:17.487437  6016 pending_rounds.cc:107] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:27:17.487645  6016 raft_consensus.cc:2889] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:17.470660 11574 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.633s	user 0.014s	sys 0.000s
I20260709 12:27:17.462208 11211 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.490087  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:17.381773 11707 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:17.494509  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:17.489583  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:17.496248  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.497231  5942 raft_consensus.cc:2410] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:27:17.460893 11674 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:17.468755 11622 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:17.499323  5867 raft_consensus.cc:2410] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:27:17.489787 11618 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.482643 11709 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:17.513005  5915 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:17.518893  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:17.522675  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:17.500186 11699 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:17.510958 11695 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:17.538404 11754 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:27:17.539511 11755 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:27:17.539613 11622 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:17.540098 11622 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:17.540445 11622 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:17.540900 11195 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
W20260709 12:27:17.545433  6762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1528) took 1500 ms (client timeout 1766 ms). Trace:
W20260709 12:27:17.546260  6762 rpcz_store.cc:269] 0709 12:27:16.045614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.054441 (+  8827us) service_pool.cc:224] Handling call
0709 12:27:17.545083 (+1490642us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002a0dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:17.545328 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2458893056}
I20260709 12:27:17.523211 11601 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:17.554473  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.556274 11716 raft_consensus.cc:3055] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:17.556874 11716 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:17.520367 11361 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.737s	user 0.011s	sys 0.004s
I20260709 12:27:17.558090 11716 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:17.559662 11716 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:17.564208  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 190018us
I20260709 12:27:17.565096  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:17.565618 11586 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:17.566105 11586 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:17.566481 11586 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:17.567250 11752 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:17.567706 11752 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:17.568059 11752 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:17.570341 11746 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:27:17.572999 11681 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:17.574091 11713 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:17.575737 11700 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:17.575944  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:17.576957  6768 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:17.577370  6768 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:17.577803 11678 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:17.578415  6768 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:17.579577 11694 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:17.579762  6768 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:17.523483 11709 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:27:17.580899 11742 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.462s	user 0.009s	sys 0.004s
I20260709 12:27:17.581385 11674 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:17.583063 11699 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:17.584146 11699 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:17.590633 11556 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.591300 11556 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:17.592804  6793 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:17.595585  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1535) took 1322 ms. Trace:
I20260709 12:27:17.596101  5949 rpcz_store.cc:276] 0709 12:27:16.272784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.303877 (+ 31093us) service_pool.cc:224] Handling call
0709 12:27:16.304288 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:16.494265 (+189977us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:16.517697 (+ 23432us) log.cc:844] Serialized 14 byte log entry
0709 12:27:17.203025 (+685328us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:17.203105 (+    80us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:17.203164 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.595323 (+392159us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54004678d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:17.595364 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:17.595507 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":434758,"prepare.queue_time_us":26305,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":428842,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"spinlock_wait_cycles":435989888,"thread_start_us":256691,"threads_started":1,"wal-append.queue_time_us":452175}
I20260709 12:27:17.598397  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:17.580933 11709 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:17.606967 11368 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:17.597594  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:17.610621  5944 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:27:17.521194 11178 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:17.611352 11178 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:17.611765 11178 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:17.612655  6609 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:17.613538 11436 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:17.614174 11436 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:17.615273 11714 consensus_meta.cc:287] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.871s	user 0.016s	sys 0.000s
I20260709 12:27:17.615698 11714 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:17.618571 11712 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:17.618793 11361 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.619352 11361 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:17.619866 11758 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.104s	user 0.001s	sys 0.001s
I20260709 12:27:17.621793 11361 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:17.622387 11361 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:17.501240 11211 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:17.623203 11705 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:17.624475 11625 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:17.625135 11211 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:17.626363  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1806) took 1358 ms. Trace:
I20260709 12:27:17.497406 11574 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:17.526675 11618 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:17.627527  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:17.566372  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:17.627581  5945 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
W20260709 12:27:17.629058  5947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1525) took 1379 ms (client timeout 1630 ms). Trace:
I20260709 12:27:17.663327  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:17.663782  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:17.553385  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.664439  6023 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:17.665659  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:17.666093  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:17.666757  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:17.667693  6017 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:17.668665  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:17.669441  6773 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:17.670392  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:17.670994  6773 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:17.671836  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:17.672380  6773 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:17.673173  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:17.632956  5941 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:17.635901  5987 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:17.614570  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:17.612905  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:17.654367  5940 rpcz_store.cc:276] 0709 12:27:16.268288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.268459 (+   171us) service_pool.cc:224] Handling call
0709 12:27:16.268839 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:16.983312 (+714473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.626041 (+642729us) spinlock_profiling.cc:243] Waited 713 ms on lock 0x7b5400633154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:17.626286 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1640481536}
I20260709 12:27:17.656479  5867 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:17.677287  6773 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:17.666844  6023 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
W20260709 12:27:17.663319  5947 rpcz_store.cc:269] 0709 12:27:16.249411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.249788 (+   377us) service_pool.cc:224] Handling call
0709 12:27:17.628826 (+1379038us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b5400188b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:17.628976 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1000508800}
W20260709 12:27:17.680536  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 114220us
I20260709 12:27:17.684741  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:17.682911  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:17.689251  6756 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:17.681406  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:17.654732 11710 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.076s	user 0.011s	sys 0.006s
W20260709 12:27:17.690801  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 76342.8us
W20260709 12:27:17.702610  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 93099.1us
I20260709 12:27:17.705601  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:17.709308  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:17.709777  5867 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:27:17.710397  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:17.714658  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:17.716164  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:17.713718  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:17.719573  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:17.721159  6024 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:17.721904  5865 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:17.722512  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:17.644470 11625 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:17.708329  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:17.737071  5940 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:27:17.737529  5940 pending_rounds.cc:85] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 8
I20260709 12:27:17.737865  5940 pending_rounds.cc:107] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:27:17.738161  5940 raft_consensus.cc:2889] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:17.739079  6017 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:17.741525 11556 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:17.742558 11556 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:17.743091 11556 raft_consensus.cc:697] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:17.744026 11556 consensus_queue.cc:237] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:17.744747  5870 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:17.745132  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1363) took 4366 ms. Trace:
I20260709 12:27:17.721325 11712 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:17.745514  6016 rpcz_store.cc:276] 0709 12:27:13.378729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.378857 (+   128us) service_pool.cc:224] Handling call
0709 12:27:13.379155 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:17.488079 (+4108924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.744841 (+256762us) spinlock_profiling.cc:243] Waited 4.1 s on lock 0x7b5400764154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:17.745030 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9435588480}
I20260709 12:27:17.746250 11681 raft_consensus.cc:2764] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:27:17.746771 11601 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:17.675666 11695 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:17.689327 11714 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:17.698359 11719 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:17.698010 11361 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.813469  6768 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:17.712313 11685 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:17.846927  6793 raft_consensus.cc:2410] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 21 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:27:17.846647  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:17.724963 11696 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:17.725083 11678 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:17.732034 11473 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:17.731827 11713 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:17.814248 11607 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:17.722195 11710 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:17.839509 11601 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:17.732060 11703 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.127s
I20260709 12:27:17.840252 11728 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:17.840770 11724 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:17.851029 11685 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.853874 11473 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:27:17.864779 11733 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.846s	user 0.019s	sys 0.011s
W20260709 12:27:17.888147 11635 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.819s	user 0.020s	sys 0.002s
I20260709 12:27:17.866377 11678 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:17.852142 11361 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:17.901556 11473 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:17.894397 11685 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:17.891463 11601 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:17.916953 11631 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.943696 11710 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:17.945473 11753 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:27:17.948598 11707 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:17.954480 11707 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:17.955192 11707 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:17.961208 11759 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.440s	user 0.000s	sys 0.003s
I20260709 12:27:17.971745 11752 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:17.996595 11752 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:17.988771 11709 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:17.986438 11707 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:18.042835  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1173) took 1074 ms. Trace:
I20260709 12:27:18.044147  5866 rpcz_store.cc:276] 0709 12:27:16.967841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.037267 (+ 69426us) service_pool.cc:224] Handling call
0709 12:27:17.037746 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:17.038159 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:17.038192 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:17.064209 (+ 26017us) log.cc:844] Serialized 26 byte log entry
0709 12:27:17.340075 (+275866us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:17.340124 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.340212 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:17.960388 (+620176us) spinlock_profiling.cc:243] Waited 65.3 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:18.041398 (+ 81010us) raft_consensus.cc:1685] finished
0709 12:27:18.041507 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.042388 (+   881us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":48382,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":12774,"spinlock_wait_cycles":150173440,"thread_start_us":261260,"threads_started":1,"wal-append.queue_time_us":385873}
I20260709 12:27:18.053479  5869 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:17.841202  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 127602us
I20260709 12:27:18.056320  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1861) took 2561 ms. Trace:
I20260709 12:27:18.057200  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1529) took 1806 ms. Trace:
I20260709 12:27:18.059285  5870 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:18.062656  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1424) took 1142 ms. Trace:
I20260709 12:27:18.063339  5865 rpcz_store.cc:276] 0709 12:27:16.920070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.920241 (+   171us) service_pool.cc:224] Handling call
0709 12:27:16.920629 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:17.723997 (+803368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.061584 (+337587us) spinlock_profiling.cc:243] Waited 801 ms on lock 0x7b540018a954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:18.062363 (+   779us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1842187136}
I20260709 12:27:18.066731  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:18.074414  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:18.076093  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.080621  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:18.083025  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 377349us
I20260709 12:27:18.092465  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:18.118144  5872 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:18.093786 11361 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:18.122385 11361 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.123611 11361 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:18.126950 11587 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:18.098562  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:18.142283  5942 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:27:18.098966  6019 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:18.099438 11574 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:18.056816  5869 rpcz_store.cc:276] 0709 12:27:15.494734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.499553 (+  4819us) service_pool.cc:224] Handling call
0709 12:27:16.365876 (+866323us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b58002e2f98. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:27:16.366168 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:18.054588 (+1688420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.055816 (+  1228us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54001be754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:18.056191 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3936427264}
I20260709 12:27:18.107656  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:18.108263  5947 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:18.144781 11574 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.112147  5940 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:18.112630 11510 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:18.145748 11574 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:18.148404 11584 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:18.149363 11574 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:18.150568  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:18.151723  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:34888 (request call id 1399) took 1360 ms. Trace:
I20260709 12:27:18.152305  6019 rpcz_store.cc:276] 0709 12:27:16.791719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.800533 (+  8814us) service_pool.cc:224] Handling call
0709 12:27:18.059216 (+1258683us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b54001716d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c7aa3d 00007f39d7c4acd0 00007f39d11cf254 00007f39d11cf1e2 00007f39d11cf111 00007f39d11cf08c 00007f39d11ce342 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:18.151542 (+ 92326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46116736}
I20260709 12:27:18.153716 11584 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:18.113238 11601 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:18.115571 11607 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:18.157498 11607 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:18.161163 11607 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:18.161394 11510 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:18.162665 11510 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:18.167455 11510 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:18.058509  5948 rpcz_store.cc:276] 0709 12:27:16.250721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.269278 (+ 18557us) service_pool.cc:224] Handling call
0709 12:27:16.269620 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:16.480641 (+211021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:16.480676 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:17.582240 (+1101564us) spinlock_profiling.cc:243] Waited 4.06 ms on lock 0x7b18000a63c0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb642a6 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a
0709 12:27:17.582877 (+   637us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b2c001b1e88. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:27:17.583447 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 12:27:18.055679 (+472232us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:18.055816 (+   137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.056543 (+   727us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b5400467dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:18.056611 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:18.056687 (+    76us) raft_consensus.cc:1685] finished
0709 12:27:18.056701 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.056964 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":664866,"raft.queue_time_us":30369,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":14909,"spinlock_wait_cycles":491919744,"thread_start_us":471395,"threads_started":1,"wal-append.queue_time_us":414242}
I20260709 12:27:18.199146 11601 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:18.130625 11631 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:18.200222 11601 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:18.138437 11436 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:18.169974  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:18.170531  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1392) took 1383 ms. Trace:
I20260709 12:27:18.212299  6015 rpcz_store.cc:276] 0709 12:27:16.786716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.789326 (+  2610us) service_pool.cc:224] Handling call
0709 12:27:16.789821 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:16.931576 (+141755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:16.931609 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:16.931653 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:16.931686 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.169735 (+1238049us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400160b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:18.169863 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.170214 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":325835008}
I20260709 12:27:18.213474  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:18.176623 11674 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:18.127779 11361 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:18.181231 11678 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:18.182535 11276 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:18.218343 11276 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:18.219877 11276 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:18.222069 11276 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:18.213557  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.186383 11584 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.188702  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:18.226588  5865 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:18.228144  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:18.190631  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:18.229652 11601 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:18.230163  5866 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:18.231302  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:18.232187 11211 consensus_queue.cc:1059] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:18.233644  5866 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:18.195108  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:18.229439  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:18.197508  5869 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:17.882716 11625 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:18.238305 11625 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:17.914970 11460 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.220s
I20260709 12:27:18.248725 11685 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:18.251972 11584 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:18.204782 11635 consensus_queue.cc:1059] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:18.208424 11587 raft_consensus.cc:476] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Not starting pre-election -- already a leader
I20260709 12:27:18.209044 11679 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:18.209704 11607 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:18.213766 11510 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:18.254576 11211 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:18.255553 11601 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:27:18.201305  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 81078.6us
W20260709 12:27:18.313091  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 100040us
I20260709 12:27:18.330901  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:18.331626 11635 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:18.332696 11725 raft_consensus.cc:3055] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:18.333715 11694 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:18.343508 11635 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.348368 11725 raft_consensus.cc:740] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:18.361786 11694 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:18.373152 11635 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:18.391848  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 66883.6us
W20260709 12:27:18.400669  6766 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.016s	sys 0.012s
W20260709 12:27:18.401316  6766 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.016s	sys 0.012s
W20260709 12:27:18.403613  5990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.003s	sys 0.015s
W20260709 12:27:18.404181  5990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.003s	sys 0.015s
W20260709 12:27:18.404552  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 720600us
I20260709 12:27:18.405779  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:18.405915 11601 consensus_queue.cc:1059] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:18.407658 11618 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:18.408583 11618 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:18.409030 11618 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.409355 11618 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:18.409916  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:18.418530  5940 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:18.421446  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 49079us
I20260709 12:27:18.432032  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:18.405771  5873 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.030s	sys 0.032s
I20260709 12:27:18.439605  5948 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:18.446643  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:18.458997  5873 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.031s	sys 0.032s
I20260709 12:27:18.418973 11685 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:18.420454 11713 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:18.419581 11709 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:18.448066 11607 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:18.382776 11510 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:18.497609  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:18.477489 11713 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:18.417204 11473 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:18.489794 11607 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.521374  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:18.506649  5838 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:18.526572  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 118078us
W20260709 12:27:18.516727  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 144032us
W20260709 12:27:18.523512 11680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.023s	sys 0.087s
W20260709 12:27:18.526705  5987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.006s	sys 0.013s
W20260709 12:27:18.555128  5987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.007s	sys 0.014s
W20260709 12:27:18.542685  5988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.011s	sys 0.005s
I20260709 12:27:18.554128 11601 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:18.560940 11601 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.531683 11607 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:18.569358  5988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.012s	sys 0.006s
I20260709 12:27:18.561396 11601 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:18.556159 11680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.023s	sys 0.088s
I20260709 12:27:18.526652 11693 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:18.586055  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 442244us
I20260709 12:27:18.596683  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1439) took 1539 ms. Trace:
I20260709 12:27:18.596210 11694 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:18.597792  6785 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:27:18.598133 11694 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:18.600026 11631 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:18.615217  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 209755us
I20260709 12:27:18.590484 11710 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:18.607416  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:18.617197  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:18.625905  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46343.4us
W20260709 12:27:18.617617  6756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.008s	sys 0.007s
I20260709 12:27:18.527659 11725 consensus_queue.cc:260] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:18.619493 11584 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:18.619833 11710 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:18.635708  6756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.009s	sys 0.007s
W20260709 12:27:18.655427  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.597184  5868 rpcz_store.cc:276] 0709 12:27:16.981172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.046667 (+ 65495us) service_pool.cc:224] Handling call
0709 12:27:17.084611 (+ 37944us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:17.084882 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:17.410017 (+325135us) log.cc:844] Serialized 14 byte log entry
0709 12:27:18.052126 (+642109us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b1800126a00. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:27:18.447172 (+395046us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:18.447323 (+   151us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:18.481300 (+ 33977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.489191 (+  7891us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.520311 (+ 31120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18213,"prepare.queue_time_us":327710,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":8117,"raft.queue_time_us":18873,"raft.run_cpu_time_us":446,"raft.run_wall_time_us":58218,"spinlock_wait_cycles":2585856,"thread_start_us":648336,"threads_started":2}
W20260709 12:27:18.658568  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 1.02259e+06us
W20260709 12:27:18.670245  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 94284.5us
I20260709 12:27:18.645978 11710 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:18.676524  5874 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.041s	sys 0.042s
W20260709 12:27:18.639974 11711 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.002s	user 0.050s	sys 0.088s
I20260709 12:27:18.677492 11679 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
W20260709 12:27:18.681465  6762 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.017s	sys 0.036s
I20260709 12:27:18.645747 11716 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:18.691397  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:18.692071  5874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.042s	sys 0.042s
I20260709 12:27:18.697121 11678 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.673488 11601 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:18.707672  6762 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.019s	sys 0.036s
I20260709 12:27:18.660688 11674 consensus_queue.cc:1059] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:27:18.659200 11765 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
W20260709 12:27:18.717222 11178 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.105s	user 0.007s	sys 0.024s
W20260709 12:27:18.701488 11711 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.174s	user 0.051s	sys 0.089s
I20260709 12:27:18.718204 11473 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:18.733675  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 1331) took 1065 ms. Trace:
I20260709 12:27:18.706826 11211 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:27:18.722330 11693 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:18.740370  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:18.702680 11678 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:18.762943  6793 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:18.758997 11699 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:18.748747 11693 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:18.764830  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51210.5us
I20260709 12:27:18.764703 11699 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:18.735816  6756 rpcz_store.cc:276] 0709 12:27:17.668180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.690893 (+ 22713us) service_pool.cc:224] Handling call
0709 12:27:17.691251 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:17.691465 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:17.748217 (+ 56752us) log.cc:844] Serialized 14 byte log entry
0709 12:27:18.686659 (+938442us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:18.706624 (+ 19965us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:18.718560 (+ 11936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.726776 (+  8216us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.733547 (+  6771us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31493,"prepare.queue_time_us":121630,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":5590,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":25625,"thread_start_us":824136,"threads_started":1,"wal-append.queue_time_us":781691}
W20260709 12:27:18.769419  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 49729.4us
I20260709 12:27:18.727560 11178 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:18.773519 11765 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:18.789525  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:18.793831 11262 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:18.794597 11699 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:18.789526 11772 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.771311 11693 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:18.742275 11716 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:18.807529  6018 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:18.796890  5941 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:18.809571  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:18.810151 11716 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:18.810945  5990 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:18.811529  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:18.799753 11276 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:27:18.803678 11681 raft_consensus.cc:3055] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:18.804747  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:18.807042 11556 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:18.797379 11765 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.819875  5838 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:18.812455  6024 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:18.812494 11769 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.147s	user 0.002s	sys 0.000s
I20260709 12:27:18.830588 11556 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:18.812556 11460 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:18.838232  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:18.838732  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:18.839708  6021 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:27:18.840715 11773 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.206s	user 0.005s	sys 0.000s
I20260709 12:27:18.842454 11178 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:18.870261  6016 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 19
W20260709 12:27:18.843945 11625 consensus_meta.cc:287] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.605s	user 0.018s	sys 0.000s
I20260709 12:27:18.871093  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:18.872143  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 98984.7us
I20260709 12:27:18.846545 11601 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:18.848043  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.882650 11587 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:18.847652 11713 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:18.848788  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:18.850013  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:18.851007  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:18.835009  5913 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:18.856377  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1178) took 1844 ms. Trace:
I20260709 12:27:18.822643 11681 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:18.829330 11765 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:18.858814 11622 consensus_meta.cc:287] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.318s	user 0.009s	sys 0.018s
I20260709 12:27:18.858271 11772 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:18.861848 11779 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 12:27:18.847697 11586 consensus_meta.cc:287] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.281s	user 0.012s	sys 0.014s
I20260709 12:27:18.862593 11699 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:18.864935  5941 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:27:18.866600  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:18.861135 11678 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:18.936085  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:18.936685 11556 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:18.884107  6023 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:18.941073  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.886852  6017 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:18.887959  5874 rpcz_store.cc:276] 0709 12:27:17.011906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.564314 (+552408us) service_pool.cc:224] Handling call
0709 12:27:17.564829 (+   515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:17.565323 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:17.565363 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:17.629863 (+ 64500us) log.cc:844] Serialized 26 byte log entry
0709 12:27:18.758503 (+1128640us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:18.772341 (+ 13838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.777120 (+  4779us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:18.855978 (+ 78858us) raft_consensus.cc:1685] finished
0709 12:27:18.855997 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.856286 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1702,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":127,"thread_start_us":964642,"threads_started":1,"wal-append.queue_time_us":1077440}
W20260709 12:27:18.894174  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 104809us
I20260709 12:27:18.892270 11716 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.894924  6022 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:18.895767 11780 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.177s	user 0.001s	sys 0.001s
I20260709 12:27:18.897887 11707 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:27:18.898521  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 95287.7us
I20260709 12:27:18.875680 11584 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:18.931110 11622 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:18.933051 11772 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:18.883458 11587 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:18.935477 11586 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:18.937595  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:18.890965  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:18.946959 11783 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.306s	user 0.003s	sys 0.000s
I20260709 12:27:18.948230 11680 raft_consensus.cc:3055] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:18.948683 11702 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.949698  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:18.950671  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:18.929534 11601 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:27:18.934032 11460 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:18.948315 11704 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:18.933267 11607 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:18.977900  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 167061us
I20260709 12:27:19.006974  5872 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:18.981161  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:18.958634 11716 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:18.928072 11473 consensus_queue.cc:1059] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:27:18.941342 11178 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:18.992511  6756 raft_consensus.cc:2368] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 22. Current term is 23.
I20260709 12:27:18.992882  6793 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:18.999009  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:18.977572 11584 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.982303 11713 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:19.082226  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:19.083292  5943 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:19.084206  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:18.991660 11702 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:19.086174  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:19.087595 11702 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:19.023105 11704 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:19.023481  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:19.088706  6017 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:19.090641  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:19.090719  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:19.091575 11704 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:19.091713  6017 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:19.091979  6768 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:19.092944  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:19.026341 11635 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.568s	user 0.015s	sys 0.002s
I20260709 12:27:19.093231 11694 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:19.093941 11635 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:19.095201 11694 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:19.095737 11694 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:19.095920  6017 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:19.096575 11635 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:19.027523 11766 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:27:19.097786 11584 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:19.097328 11694 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:19.098934  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1441) took 2108 ms. Trace:
I20260709 12:27:19.027946  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:19.099540  5873 rpcz_store.cc:276] 0709 12:27:16.993140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.492670 (+499530us) service_pool.cc:224] Handling call
0709 12:27:17.493453 (+   783us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:17.569879 (+ 76426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:17.569913 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:17.570459 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 12:27:18.590713 (+1020254us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:18.613994 (+ 23281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.623176 (+  9182us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:19.098623 (+475447us) raft_consensus.cc:1685] finished
0709 12:27:19.098643 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:19.098857 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75525,"raft.queue_time_us":76480,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":133,"thread_start_us":897704,"threads_started":1,"wal-append.queue_time_us":1250779}
I20260709 12:27:19.028606 11460 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:19.030071  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:19.031773  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:18.942391 11681 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:18.990922 11685 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:19.039827 11678 raft_consensus.cc:3055] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:18.989295 11680 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:19.044090  6024 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:27:19.113965  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1179) took 1975 ms. Trace:
I20260709 12:27:19.047695 11607 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:19.049533  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:19.052502 11587 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:19.056254 11693 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:19.057849  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 49972us
W20260709 12:27:19.063378 11786 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.393s	user 0.001s	sys 0.001s
I20260709 12:27:19.065407 11510 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:19.071424 11716 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:19.071883 11725 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:19.153666  6021 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:19.071507  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:19.063390 11586 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:19.079037  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.166009  5948 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:19.079289 11622 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:19.079870  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 62592.5us
W20260709 12:27:19.091176 11768 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.215s	user 0.002s	sys 0.000s
I20260709 12:27:19.170043  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:19.025818  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1537) took 1451 ms. Trace:
I20260709 12:27:19.122267 11178 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:27:19.171432  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.172358 11556 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:19.172693  5987 leader_election.cc:400] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
I20260709 12:27:19.172998  5987 leader_election.cc:403] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:27:19.170830  6762 rpcz_store.cc:276] 0709 12:27:17.574049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.631831 (+ 57782us) service_pool.cc:224] Handling call
0709 12:27:17.632323 (+   492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:17.632825 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:17.632867 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:17.666432 (+ 33565us) log.cc:844] Serialized 26 byte log entry
0709 12:27:18.773250 (+1106818us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:18.781854 (+  8604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:19.025083 (+243229us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b5400466ca0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bfadc0 00007f39d2c86a02 00007f39d2c83dde 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:27:19.025171 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:19.025225 (+    54us) raft_consensus.cc:1685] finished
0709 12:27:19.025246 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:19.025653 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":775,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":14604,"spinlock_wait_cycles":80246144,"thread_start_us":961185,"threads_started":1,"wal-append.queue_time_us":1010552}
I20260709 12:27:19.177125  6773 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:19.177822  6793 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:19.178431  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:19.179518 11510 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:19.180192 11765 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:19.180682 11702 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:19.180390 11510 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:19.181239 11702 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:19.182545 11765 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:19.183426 11702 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:19.161509  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:19.187245 11693 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:19.121153 11618 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.677s	user 0.016s	sys 0.000s
I20260709 12:27:19.192323 11618 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:19.195777 11618 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:19.177361  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:19.164788  5940 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:19.105264  6016 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:19.161054  6756 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:19.103814  6023 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:19.101368  6020 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:27:19.109565  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 179697us
I20260709 12:27:19.113921  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.116665  6022 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:19.099592 11694 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:19.132371  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:19.213480  6768 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
W20260709 12:27:19.133610  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.151443  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:19.113965 11678 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:19.216619  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:19.217916  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 45497.4us
I20260709 12:27:19.219393  6778 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:19.219940  6778 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:19.220850  6778 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:19.159698  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 129787us
I20260709 12:27:19.115305  5871 rpcz_store.cc:276] 0709 12:27:17.019429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.572174 (+552745us) service_pool.cc:224] Handling call
0709 12:27:17.610426 (+ 38252us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:17.610827 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:17.610854 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:18.411077 (+800223us) spinlock_profiling.cc:243] Waited 26.8 ms on lock 0x7b2c00198418. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:27:18.411932 (+   855us) log.cc:844] Serialized 26 byte log entry
0709 12:27:18.625846 (+213914us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:18.782252 (+156406us) log.cc:844] Serialized 14 byte log entry
0709 12:27:18.796148 (+ 13896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.803322 (+  7174us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:18.817825 (+ 14503us) raft_consensus.cc:1685] finished
0709 12:27:18.826052 (+  8227us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.988266 (+162214us) spinlock_profiling.cc:243] Waited 37.8 ms on lock 0x7b54006f2050. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:18.995161 (+  6895us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10603,"prepare.queue_time_us":205929,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":59,"raft.queue_time_us":28929,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":14780,"spinlock_wait_cycles":148665984,"thread_start_us":304523,"threads_started":2,"wal-append.queue_time_us":314776}
I20260709 12:27:19.159744  6024 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:19.161041 11680 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:19.161463  6757 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:27:19.219592  5941 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 22: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:27:19.152297  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 174570us
I20260709 12:27:19.239285  6778 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:19.239336  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:19.259586  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 275585us
W20260709 12:27:19.215852 11710 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.515s	user 0.012s	sys 0.004s
W20260709 12:27:19.262799  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.280953  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.282586  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.224866 11625 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
I20260709 12:27:19.286873  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:19.287277  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:19.287863  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.300654  5869 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:19.301967  6022 raft_consensus.cc:2376] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18. Re-sending same reply.
I20260709 12:27:19.303785  5869 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:19.306110  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:19.311071  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:19.311259  5869 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:19.312584 11702 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:19.306020 11704 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:19.307221  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.311091 11473 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:19.316767 11473 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:19.221988 11685 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:19.318809  5941 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:19.328619  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1887) took 2063 ms. Trace:
I20260709 12:27:19.329200  5867 rpcz_store.cc:276] 0709 12:27:17.265229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.724359 (+459130us) service_pool.cc:224] Handling call
0709 12:27:17.725132 (+   773us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:17.727260 (+  2128us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:27:17.727294 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:17.728160 (+   866us) log.cc:844] Serialized 26 byte log entry
0709 12:27:18.126284 (+398124us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b18001b6b80. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:27:18.530345 (+404061us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:18.565174 (+ 34829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.582920 (+ 17746us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:18.945878 (+362958us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:19.309561 (+363683us) raft_consensus.cc:1685] finished
0709 12:27:19.309593 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:19.328416 (+ 18823us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1591,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":17061,"spinlock_wait_cycles":834910848,"thread_start_us":327691,"threads_started":1,"wal-append.queue_time_us":905675}
I20260709 12:27:19.275878 11710 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:19.225322 11586 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 12:27:19.334326  5868 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:27:19.313282 11702 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:19.349514  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:19.350239  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:19.350883  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:19.351322  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:19.353137  6022 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:27:19.353881  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:19.354744  6021 raft_consensus.cc:2393] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 19: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:19.354789  6022 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:27:19.355173  6022 pending_rounds.cc:85] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 7
I20260709 12:27:19.355369  6022 pending_rounds.cc:107] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:27:19.355635  6022 raft_consensus.cc:2889] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:19.355913  6021 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:27:19.357084  5947 raft_consensus.cc:2410] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:27:19.358032  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:19.358284  5942 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:19.358820  5874 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:19.359046  6023 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:19.359592 11622 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:19.360549  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:19.362295  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 83965us
I20260709 12:27:19.364892 11678 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:19.366401 11678 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:19.351989  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:19.335781  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:19.342334  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:19.266562 11765 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:27:19.342543  5869 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:19.346803  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:19.337340  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:19.341794 11704 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:19.342775 11586 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:19.337193 11473 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:19.335755 11710 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:19.358039 11607 consensus_queue.cc:1059] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:27:19.379721  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:19.383049 11702 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:19.388401  5943 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:19.357403  6021 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:19.346357 11584 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:19.417017  6785 raft_consensus.cc:2368] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19.
W20260709 12:27:19.416491  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:19.419543  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 55305.7us
W20260709 12:27:19.422396  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 149656us
I20260709 12:27:19.377386 11510 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:19.344569 11685 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:19.431592  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:19.433384  6785 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:19.391382 11586 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:19.437889  5696 catalog_manager.cc:5697] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 8 to 14, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 14 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:19.439688  6024 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:19.433710  5990 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:19.442842  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 56402us
I20260709 12:27:19.450464  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:19.451689  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:19.453312 11607 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:19.425165 11556 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:19.387777 11674 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
W20260709 12:27:19.466660  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 87103.7us
I20260709 12:27:19.458216  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:19.474640  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:19.443392 11775 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:19.483994  6778 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:19.485865  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:19.487254  5869 raft_consensus.cc:2393] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:19.424322 11625 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:27:19.489039  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:19.453964 11607 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:19.457136 11710 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.460711 11693 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:19.443202 11794 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.118s
I20260709 12:27:19.496948  5873 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:19.465672 11276 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:19.500752 11584 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:19.502454  5990 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:19.503156 11704 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:19.504626 11802 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:19.505733 11772 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:19.506450 11772 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.506892 11772 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:19.513888  5696 catalog_manager.cc:5697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 7 to 10, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 10 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:27:19.514954 11587 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:19.515753  5696 catalog_manager.cc:5697] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:27:19.519470 11725 raft_consensus.cc:3055] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:19.519951 11725 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:19.526364  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.537731 11797 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:19.538589 11797 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:19.542649  6785 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:19.491360 11473 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:19.546823  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.550570 11791 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:19.550905  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 93435.4us
I20260709 12:27:19.523368 11276 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.520936 11725 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:19.566910  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.560386  5869 raft_consensus.cc:3055] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:19.567602  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.568459  5947 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:19.560449 11791 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.573036  5940 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:27:19.524926 11710 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:19.530809 11711 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.496765 11586 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:19.567891  5869 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:19.560057 11607 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:19.583251  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 80953us
W20260709 12:27:19.585201  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 135094us
I20260709 12:27:19.585908  6611 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:19.586580  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:19.580448 11674 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:19.554903 11765 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:19.557451 11276 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:19.572106  5987 connection.cc:256] RPC call timeout handler was delayed by -1.9623758160000762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
W20260709 12:27:19.589582  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 99406.4us
I20260709 12:27:19.591894  5871 raft_consensus.cc:2410] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:27:19.585256  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.596655  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.599999  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:19.601594  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:19.655233  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:19.602452  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:19.648872  5990 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:19.603605  5945 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:19.589551 11791 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:19.668567  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.668838 11711 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:19.669775  5946 raft_consensus.cc:2410] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:27:19.670001 11791 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:19.671243 11711 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:19.672466 11674 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:19.670912  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.668643 11586 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:19.675372  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:19.675982  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.662312 11704 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:19.556154 11473 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:19.677949 11704 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:19.679004 11704 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:19.679618 11473 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:19.682078 11710 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:19.655153  5987 connection.cc:256] RPC call timeout handler was delayed by -1.8463184080001156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1644s for a response
I20260709 12:27:19.659185  5868 raft_consensus.cc:2368] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23.
I20260709 12:27:19.689365 11797 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:19.685017  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:19.659924  5870 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 19: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:19.661340 11680 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:19.662223  5949 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:19.647192 11725 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:19.665969  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.666530  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:19.666690  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:19.667234  5942 raft_consensus.cc:2368] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19.
I20260709 12:27:19.658424 11765 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:19.695158  5869 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:19.706876  5869 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:19.707679 11765 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:19.667785  5867 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:19.709173 11799 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:27:19.711067  5946 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 12:27:19.712486  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 126013us
I20260709 12:27:19.713167 11681 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:19.713478  5915 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:19.695025  5987 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.575s (SENT) [suppressed 9 similar messages]
I20260709 12:27:19.715322  5990 leader_election.cc:400] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19.
I20260709 12:27:19.705323  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:19.703653  5947 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:19.715813  5990 leader_election.cc:403] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:19.716369  5941 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:19.699000  6609 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:19.705433  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:19.723335  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:19.724005  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:19.695794 11510 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:19.696710 11794 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:19.727476  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:19.727643  5866 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:19.728156  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:19.734405  5940 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 18: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:27:19.719913 11803 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:27:19.740490  5838 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:19.733970  5915 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:19.700266 11473 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:19.735546  5871 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:19.737110  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:19.737906  5947 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:27:19.738792  5946 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:19.738934  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:19.745843  5942 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:19.700760 11693 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:19.745544  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46599us
I20260709 12:27:19.747774  5990 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:19.723884 11797 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:19.692888 11556 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:19.752923  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 184820us
I20260709 12:27:19.732772 11713 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:19.752796 11796 consensus_queue.cc:1175] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1fc57407d56241b38a94d543e073f8af" responder_term: 16 status { last_received { term: 14 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:27:19.754071 11713 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:19.756397 11713 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:19.757015 11804 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:19.703562 11775 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:27:19.758512 11775 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:19.758912 11775 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:19.759820 11775 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:19.761123 11775 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:19.761888  5949 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:19.732650 11510 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.761950  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.771097  5945 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:19.763751  5941 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:19.764499  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:19.744998 11473 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:19.733541 11794 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.738057 11806 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:19.702543 11674 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:19.741961 11791 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:19.741849 11635 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:19.783825  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:19.784644  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:19.746589 11460 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19.
I20260709 12:27:19.783938  5866 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:19.777010  6017 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:27:19.786551  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 91595us
I20260709 12:27:19.787854  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:19.788560  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:19.788820  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.826772  5945 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:19.789671  6020 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:19.784539 11473 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:19.788570 11771 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 12:27:19.794270  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:19.727859 11772 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:19.749123 11693 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:19.763728 11797 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.803853  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:19.805815 11804 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:19.806389  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.812134  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:19.811939  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.773995 11510 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:19.821874  5871 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:27:19.823482  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.852813  5941 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:19.853291  5871 pending_rounds.cc:85] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 8
I20260709 12:27:19.853703  5871 pending_rounds.cc:107] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:27:19.854130  5871 raft_consensus.cc:2889] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:19.815063  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 67385.5us
I20260709 12:27:19.855674  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:19.778672 11711 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:19.825263  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:19.833199  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 57628.9us
I20260709 12:27:19.794324 11794 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:19.801802 11710 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:19.838617  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:19.845191  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:19.844434 11811 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:27:19.851934  5868 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 12: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:19.864045  6757 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:19.864511 11460 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:19.933806  5943 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:27:19.834254 11806 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:19.865474 11809 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.201s	user 0.000s	sys 0.004s
I20260709 12:27:19.842792 11625 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:19.933817  5945 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:19.867880  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:19.846413 11797 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:19.841346 11693 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:20.006881  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:19.825881 11791 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:19.945000  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:20.004626  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:19.948249  5872 raft_consensus.cc:2393] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:20.007982  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:20.013715  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:20.040308  5871 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:19.939482  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:20.040047  6019 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:20.043656  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:20.044742  5869 raft_consensus.cc:2393] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:20.013795 11710 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:20.007145 11460 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:20.018826  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:20.020810  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:20.050788  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 184962us
I20260709 12:27:20.031618  5949 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:27:20.037070  5838 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:20.052098  5868 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:20.034615  5865 raft_consensus.cc:2393] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:20.024381  5866 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:19.866292 11804 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:20.057566  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:20.065099  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:20.066313  5869 raft_consensus.cc:2410] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:27:20.029580 11587 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:27:20.067608  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:20.069233 11810 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 12:27:20.056774  5870 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:20.069027  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:20.074242  5866 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:20.079679  5840 connection.cc:256] RPC call timeout handler was delayed by -1.3365940779999619s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
I20260709 12:27:20.032398 11586 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 12:27:20.084353  5840 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Timed out: RequestConsensusVote RPC to 127.2.4.196:34835 timed out after 1.738s (SENT)
I20260709 12:27:20.083455  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:20.066030 11460 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:20.116730  5866 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:20.107774 11804 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:20.110391  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:20.020140 11276 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:20.052940 11765 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:20.122426  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:20.123347  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:20.127637  6785 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:20.133526  6778 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:20.135986  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:20.137987  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:20.127065 11802 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:27:20.151085 11694 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:20.133788 11765 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:20.152185 11765 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:20.155344  6768 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:20.109598 11799 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:20.157532  5696 catalog_manager.cc:5697] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
W20260709 12:27:20.159050 11819 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.406s	user 0.001s	sys 0.001s
W20260709 12:27:20.152462  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 68868.4us
I20260709 12:27:20.163028  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:20.159253  5696 catalog_manager.cc:5697] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:27:20.126276  5942 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
W20260709 12:27:20.151842 11820 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 12:27:20.179586  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 69235.4us
I20260709 12:27:20.182199  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:20.187268  5915 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:20.188043  6778 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:20.142273 11685 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.210580  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:20.212302  6778 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
W20260709 12:27:20.218505  5987 connection.cc:256] RPC call timeout handler was delayed by -1.7365184080003928s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
W20260709 12:27:20.218986  5987 connection.cc:256] RPC call timeout handler was delayed by -1.6888184080003157s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
W20260709 12:27:20.219393  5987 connection.cc:256] RPC call timeout handler was delayed by -1.5843758159999197s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
I20260709 12:27:20.219524  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:20.220163  6778 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:20.219724  5987 connection.cc:256] RPC call timeout handler was delayed by -1.0854714990000502s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3085s for a response
I20260709 12:27:20.203987 11711 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:20.208320 11713 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:20.252859  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.160900 11707 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:27:20.110249 11806 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:20.260660  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:20.208081 11772 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:20.157332 11799 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:20.255249  6785 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:20.264524  5990 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:20.273705  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:20.268064  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1553) took 1094 ms. Trace:
I20260709 12:27:20.276060  6793 rpcz_store.cc:276] 0709 12:27:19.174077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.178730 (+  4653us) service_pool.cc:224] Handling call
0709 12:27:19.179032 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:19.365902 (+186870us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:19.416238 (+ 50336us) log.cc:844] Serialized 14 byte log entry
0709 12:27:19.698073 (+281835us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:19.698134 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:19.698185 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.267783 (+569598us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b5400008f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:20.267833 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:20.267996 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":173541,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":94,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":12357,"spinlock_wait_cycles":429108480,"thread_start_us":331153,"threads_started":2,"wal-append.queue_time_us":319815}
I20260709 12:27:20.252640 11710 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:20.298094 11814 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.638s	user 0.000s	sys 0.004s
I20260709 12:27:20.299505 11678 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.300149 11678 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:20.254949 11711 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.212939 11685 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:20.307874  5987 leader_election.cc:400] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15.
I20260709 12:27:20.316606  5987 leader_election.cc:403] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:20.267326 11694 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:20.264357 11713 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:20.273442 11556 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:27:20.306413 11680 consensus_meta.cc:287] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.603s	user 0.015s	sys 0.000s
I20260709 12:27:20.318572  5696 catalog_manager.cc:5697] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 10 to 13, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 13 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:20.327590  6778 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:20.350716  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:20.350838 11824 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:27:20.350759 11812 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15.
I20260709 12:27:20.354957 11276 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:20.355649  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:20.358625  6778 raft_consensus.cc:2393] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 16: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:20.373513 11797 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:20.374984  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 84038.3us
I20260709 12:27:20.378509  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:20.308985 11808 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:20.319373 11711 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:20.330123 11713 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:20.323832 11694 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:20.377439 11797 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:20.331875 11791 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:20.437928 11817 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.618s	user 0.003s	sys 0.005s
W20260709 12:27:20.439082  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.397141 11556 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:20.397965 11625 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:20.446066  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:20.449520  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:20.450695  6793 raft_consensus.cc:2393] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 19: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:20.451884  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:20.452849  6757 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:20.455713  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:20.456614  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:20.456162  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:20.457573  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:20.458263  6768 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:27:20.458386  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:20.409181  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 300018us
I20260709 12:27:20.457384  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:20.462291  6756 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:20.464144  5838 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:20.424903  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:20.455819  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:20.466100  6778 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 11.
I20260709 12:27:20.467748  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:20.468458  6778 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:20.485527  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 221510us
W20260709 12:27:20.418329  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 302044us
I20260709 12:27:20.426651 11694 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:20.505313  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:20.507007  6768 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:27:20.542485  6785 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 20. Current term is 21. Ops: []
W20260709 12:27:20.533459  5692 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50588 (request call id 58) took 17575 ms (client timeout 15000 ms). Trace:
W20260709 12:27:20.504998 11725 consensus_meta.cc:287] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.785s	user 0.006s	sys 0.007s
I20260709 12:27:20.411118 11808 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:20.521581 11796 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:27:20.554143 11775 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.788s	user 0.017s	sys 0.000s
I20260709 12:27:20.586002  5948 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:20.584980  6778 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 12.
I20260709 12:27:20.546550 11791 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:20.592773  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:20.595093  5696 catalog_manager.cc:5697] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 10 to 14, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 14 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:20.547626 11678 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:20.604365  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 388777us
I20260709 12:27:20.608572  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:20.609279  6768 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:20.619454  5696 catalog_manager.cc:5697] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:20.620756  6773 raft_consensus.cc:2410] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 12:27:20.623930  6757 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
W20260709 12:27:20.598791 11510 consensus_meta.cc:287] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.664s	user 0.011s	sys 0.002s
W20260709 12:27:20.635432  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 50370.3us
I20260709 12:27:20.594551 11587 consensus_queue.cc:1243] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 16.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:27:20.603912 11796 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:20.600297 11725 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:27:20.664786  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:20.657148  5696 catalog_manager.cc:5697] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:20.683858  6768 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:20.634793 11829 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:20.680383  5696 catalog_manager.cc:5697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:20.676405 11794 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.735s	user 0.008s	sys 0.005s
W20260709 12:27:20.708997  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 138155us
I20260709 12:27:20.641121 11510 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:20.726804  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:20.727105  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 261623us
I20260709 12:27:20.728245  5838 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:20.746100  6756 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:20.726847  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:20.759434  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:20.714444 11678 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.659965 11625 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:20.769326  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 94431.6us
I20260709 12:27:20.770304  6762 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:27:20.757069  5872 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:20.729350 11799 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:20.787849  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 407644us
W20260709 12:27:20.789063  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 113744us
I20260709 12:27:20.788537  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1548) took 1207 ms. Trace:
I20260709 12:27:20.790107  5948 rpcz_store.cc:276] 0709 12:27:19.566709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.567184 (+   475us) service_pool.cc:224] Handling call
0709 12:27:20.773826 (+1206642us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b5400833954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:20.774057 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2133605120}
W20260709 12:27:20.753947 11607 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.002s	sys 0.019s
I20260709 12:27:20.665544 11556 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:20.801829  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 73714.4us
I20260709 12:27:20.715670 11794 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:20.822423  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:20.810130  5990 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:20.791244 11791 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:20.794994 11607 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.002s	sys 0.020s
W20260709 12:27:20.831552  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 79370.7us
I20260709 12:27:20.804783  6611 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:20.833914  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 95635.4us
I20260709 12:27:20.836786 11460 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:20.822026 11587 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:20.847952  6018 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:20.829797 11772 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:20.779054 11678 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:20.826449 11839 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:20.860327  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 187916us
I20260709 12:27:20.857847  5696 catalog_manager.cc:5697] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 8 to 10, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 10 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:27:20.870384 11587 raft_consensus.cc:697] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 12 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:20.870666 11791 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:20.871805 11587 consensus_queue.cc:237] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:20.874753 11276 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:20.875638 11849 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:20.876793 11839 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:20.879189 11834 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 12:27:20.879635 11839 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:20.880693  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1422) took 1082 ms. Trace:
W20260709 12:27:20.880939 11831 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 12:27:20.881151  6024 rpcz_store.cc:276] 0709 12:27:19.798482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.798695 (+   213us) service_pool.cc:224] Handling call
0709 12:27:19.810409 (+ 11714us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:19.810691 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:19.810716 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:19.828537 (+ 17821us) log.cc:844] Serialized 26 byte log entry
0709 12:27:20.398822 (+570285us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:20.398871 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.398945 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:20.880394 (+481449us) raft_consensus.cc:1685] finished
0709 12:27:20.880428 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:20.880610 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":387511,"threads_started":1,"wal-append.queue_time_us":800164}
W20260709 12:27:20.546875  5692 rpcz_store.cc:269] 0709 12:27:02.834325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.843508 (+  9183us) service_pool.cc:224] Handling call
0709 12:27:20.398728 (+17555220us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:11.401376 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:11.515710 (+114334us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:11.525774 (+ 10064us) write_op.cc:435] Acquired schema lock
0709 12:27:11.525801 (+    27us) tablet.cc:662] Decoding operations
0709 12:27:11.527869 (+  2068us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:11.528030 (+   161us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:11.528065 (+    35us) tablet.cc:685] Acquiring locks for 24 operations
0709 12:27:11.757736 (+229671us) tablet.cc:701] Row locks acquired
0709 12:27:11.757803 (+    67us) write_op.cc:260] PREPARE: finished
0709 12:27:11.758007 (+   204us) write_op.cc:270] Start()
0709 12:27:11.758189 (+   182us) write_op.cc:276] Timestamp: P: 1783600031757930 usec, L: 0
0709 12:27:11.758214 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:27:11.768966 (+ 10752us) log.cc:844] Serialized 9313 byte log entry
0709 12:27:12.655292 (+886326us) op_driver.cc:464] REPLICATION: finished
0709 12:27:12.829277 (+173985us) write_op.cc:301] APPLY: starting
0709 12:27:12.845601 (+ 16324us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:14.615524 (+1769923us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:14.650311 (+ 34787us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:15.184542 (+534231us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:15.184880 (+   338us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=0,mrs_lookups=0
0709 12:27:15.184927 (+    47us) write_op.cc:312] APPLY: finished
0709 12:27:15.185509 (+   582us) log.cc:844] Serialized 211 byte log entry
0709 12:27:15.262655 (+ 77146us) spinlock_profiling.cc:243] Waited 7.88 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39cfb790ac 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b
0709 12:27:15.419285 (+156630us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:15.433689 (+ 14404us) write_op.cc:454] Released schema lock
0709 12:27:15.433787 (+    98us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:15.437836 (+  4049us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:15.437874 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1062019,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":123962,"apply.run_cpu_time_us":75001,"apply.run_wall_time_us":2622745,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104543,"num_ops":24,"prepare.queue_time_us":402219,"prepare.run_cpu_time_us":28006,"prepare.run_wall_time_us":591060,"raft.queue_time_us":237842,"raft.run_cpu_time_us":10199,"raft.run_wall_time_us":106627,"replication_time_us":885135,"spinlock_wait_cycles":18130304,"thread_start_us":789914,"threads_started":5,"wal-append.queue_time_us":538822,"wal-append.run_cpu_time_us":58375,"wal-append.run_wall_time_us":2496000}]]}
I20260709 12:27:20.890055  5696 catalog_manager.cc:5697] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 10 to 12, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 12 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:20.896580  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 93837.3us
I20260709 12:27:20.863672 11808 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:20.884869 11846 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.940375  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:20.943444  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 144069us
I20260709 12:27:20.946046  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1560) took 1085 ms. Trace:
W20260709 12:27:20.919433 11832 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
W20260709 12:27:20.950603  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 151201us
W20260709 12:27:20.921970 11710 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.011s	sys 0.014s
I20260709 12:27:20.842818 11796 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:20.946543  6766 rpcz_store.cc:276] 0709 12:27:19.860945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.869605 (+  8660us) service_pool.cc:224] Handling call
0709 12:27:20.006411 (+136806us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:20.215115 (+208704us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:20.215152 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:20.250417 (+ 35265us) log.cc:844] Serialized 26 byte log entry
0709 12:27:20.742245 (+491828us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:20.742313 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.742432 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:20.936535 (+194103us) raft_consensus.cc:1685] finished
0709 12:27:20.936562 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:20.945971 (+  9409us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55267,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"thread_start_us":624365,"threads_started":2,"wal-append.queue_time_us":646235}
I20260709 12:27:20.949066 11276 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:20.957763 11710 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.011s	sys 0.014s
I20260709 12:27:20.975104 11276 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:20.978361 11841 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:27:20.979496 11794 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:20.953526 11839 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:20.992043  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:20.942617 11846 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:21.011759 11848 log.cc:927] Time spent T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:27:21.012714 11710 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:21.013300 11710 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.013713 11710 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:21.016579 11693 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.974s	user 0.015s	sys 0.000s
W20260709 12:27:21.017076 11843 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.119s	user 0.001s	sys 0.001s
I20260709 12:27:21.017401 11693 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:20.970623 11775 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:21.029175 11693 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:21.044256 11710 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:20.972484 11510 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:21.057319  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 148754us
I20260709 12:27:21.059533  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:27:21.060572 11765 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.908s	user 0.012s	sys 0.002s
I20260709 12:27:21.049230 11586 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:21.060735 11794 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:21.001453 11276 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:21.033560 11681 consensus_meta.cc:287] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.313s	user 0.010s	sys 0.005s
W20260709 12:27:21.067052 11473 consensus_meta.cc:287] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.225s	user 0.015s	sys 0.007s
I20260709 12:27:21.065632 11586 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:21.068653 11765 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:21.070466  5915 leader_election.cc:400] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 22. Current term is 23.
I20260709 12:27:21.046360 11725 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:21.083065  5915 leader_election.cc:403] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:27:21.071902 11607 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:21.073261  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 100224us
I20260709 12:27:21.066361 11707 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:21.009647 11839 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:21.080890 11694 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.511s	user 0.013s	sys 0.000s
W20260709 12:27:21.100430  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 44131us
I20260709 12:27:21.101651  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:21.104596 11586 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:21.105295 11460 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:21.106081 11460 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:21.106215 11586 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:21.107455 11854 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 22. Current term is 23.
I20260709 12:27:21.107913 11865 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.108306 11854 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:21.108413 11865 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:21.108922 11854 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:21.109958 11839 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:21.111979 11846 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:21.112352 11685 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:21.088276 11725 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:21.086213 11473 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:21.114971 11725 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:21.115322 11473 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:21.115832  6775 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:21.108312  5913 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:21.119374 11855 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:21.119849 11855 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.098040 11694 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:21.120440 11855 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:21.120710  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.096628 11678 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:21.122314 11694 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:21.092218 11607 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:21.124926  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:21.125779  5866 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:21.127092  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:21.127935  5872 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:21.129257 11877 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:21.130224 11765 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:21.131742  5840 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:21.136374 11680 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:21.127173 11607 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:21.141436  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:21.142839  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:21.091704 11707 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:21.146310  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1412) took 1375 ms. Trace:
I20260709 12:27:21.149076  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:21.148248  6018 rpcz_store.cc:276] 0709 12:27:19.764350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.764511 (+   161us) service_pool.cc:224] Handling call
0709 12:27:21.139729 (+1375218us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540036f954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:21.139992 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2347775872}
I20260709 12:27:21.153067  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:21.146925  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:21.154662  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:21.155512  5946 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:21.158766  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:21.144286 11765 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:21.164587  5866 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 13.
I20260709 12:27:21.165409 11857 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.165757 11607 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:21.166152 11607 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.166561 11607 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:21.169970  6785 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:21.170208  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:21.145750 11711 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.711s	user 0.015s	sys 0.000s
I20260709 12:27:21.136710 11854 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:21.193531  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:21.172423  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:21.195263  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:21.196753 11806 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.197367 11806 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:21.173615  6773 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:21.196309  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:21.198602  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 394023us
W20260709 12:27:21.176088  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.178131  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:21.201344  6756 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 12: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:21.178218  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:21.178906  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:21.179108  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:21.179600  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:21.203034  6757 raft_consensus.cc:2368] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20.
I20260709 12:27:21.184015  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:21.204097  6766 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:21.185215  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:21.196285 11711 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:21.175261  5946 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:21.214126  5913 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:21.211156  5838 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:21.154440 11707 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:21.170943 11713 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.690s	user 0.012s	sys 0.004s
I20260709 12:27:21.227808  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:21.223160  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 50505.8us
I20260709 12:27:21.245059  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:21.245290  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:21.245488  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 45258.2us
I20260709 12:27:21.245659  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:21.247464  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 145312us
W20260709 12:27:21.250232  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 386920us
I20260709 12:27:21.252825  6016 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:21.250144  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:21.174850 11725 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:21.246716  5945 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:21.254279  5915 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:21.254678  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:21.224982  6762 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:21.301095  5915 leader_election.cc:400] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20.
I20260709 12:27:21.305869  5915 leader_election.cc:403] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:27:21.256194  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:21.294862  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:21.306555  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 51614.1us
I20260709 12:27:21.256810  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:21.310492  6757 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 20: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:27:21.308470  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 51701.6us
I20260709 12:27:21.255196  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:21.255406  6015 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:21.257325  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:21.214334 11674 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:21.311499  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:21.256361 11725 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.298450  6019 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:21.335935 11681 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.314604  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:21.247835 11678 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:27:21.382057  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 67539.1us
I20260709 12:27:21.166177 11857 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:21.366499  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:21.306908 11846 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:21.351791 11879 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:21.249310 11713 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:21.370069  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 113942us
I20260709 12:27:21.310657  6611 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:21.242252 11835 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:21.243536 11771 raft_consensus.cc:2764] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:27:21.376226  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:21.380352  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68955.5us
I20260709 12:27:21.233495 11765 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:21.366111 11693 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:21.399269  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 88703.7us
I20260709 12:27:21.408991  6609 leader_election.cc:400] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23.
I20260709 12:27:21.375738 11808 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:21.420077  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:21.381817 11725 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:21.425379  6768 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:27:21.426797  6609 leader_election.cc:403] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:21.330031  6020 raft_consensus.cc:2368] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20.
I20260709 12:27:21.376251 11772 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:21.387897 11678 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.431371  5940 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:21.426039  6018 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:27:21.432926  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:21.390784 11850 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:21.386125 11791 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:27:21.443792  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 129367us
W20260709 12:27:21.483989  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 47682.6us
I20260709 12:27:21.393275 11681 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:21.435176 11678 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:21.429864 11710 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:21.439064 11850 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:21.425704 11587 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 12:27:21.428766 11765 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:21.429143 11839 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 12:27:21.508599 11556 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:21.510120 11804 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23.
I20260709 12:27:21.520769 11877 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.527891 11850 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:21.578014 11772 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:21.582587 11881 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:27:21.584524 11880 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:27:21.587805 11865 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:21.588874 11876 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.209s	user 0.000s	sys 0.000s
I20260709 12:27:21.591159 11765 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:21.594794 11713 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:21.596663 11681 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:21.573520 11806 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:21.598416 11857 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:21.599151 11276 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:21.555763 11711 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:21.514062 11771 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:21.549955  6609 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:21.606086  6773 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
W20260709 12:27:21.633296  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 238340us
W20260709 12:27:21.650409  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 217566us
I20260709 12:27:21.567994 11796 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:21.659518  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:21.655205  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 146622us
I20260709 12:27:21.592957 11707 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:21.724565  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 73090.1us
W20260709 12:27:21.722711  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 199570us
I20260709 12:27:21.734072  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:21.728379  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:21.736878  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:21.738830  5866 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:21.740681  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:21.579470 11877 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:21.732542 11838 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:27:21.745581 11877 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:21.748363 11855 consensus_meta.cc:287] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.627s	user 0.012s	sys 0.000s
I20260709 12:27:21.755620 11855 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:21.748890 11607 consensus_meta.cc:287] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.003s
I20260709 12:27:21.759734 11607 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:21.760447  5990 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:21.762030 11607 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:21.749177 11893 raft_consensus.cc:3055] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:21.762990 11893 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:21.763971 11893 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:21.751405 11857 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:27:21.751628 11473 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:21.735620  5872 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:21.768301  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:21.656956 11827 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:21.570824 11556 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:21.775928  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:21.799154 11694 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:21.800012 11694 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:21.800055 11685 consensus_queue.cc:1059] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:27:21.661859 11804 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:21.776544  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:21.806262  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 80427.8us
I20260709 12:27:21.657545 11794 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.810485  5873 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:27:21.810936  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.767438 11893 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:21.813918  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 45751.9us
I20260709 12:27:21.838258  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:21.830217  5840 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:21.832140  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:21.787916 11460 consensus_meta.cc:287] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.681s	user 0.009s	sys 0.008s
I20260709 12:27:21.839236  5872 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:21.842597 11607 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:21.842907  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 66425.5us
I20260709 12:27:21.854800  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:21.856874  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 81105.9us
W20260709 12:27:21.879709  6017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.023s	sys 0.070s
W20260709 12:27:21.882591  6017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.023s	sys 0.070s
W20260709 12:27:21.880378  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 45947.3us
I20260709 12:27:21.657351 11771 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:21.886864  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:21.880084  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:21.889683  6768 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:21.889727  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:21.891330  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:21.893950  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 63846.3us
I20260709 12:27:21.894897  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:21.697727  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:21.896138  6773 raft_consensus.cc:606] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:27:21.896518  6773 raft_consensus.cc:614] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:21.897573 11799 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:27:21.900202 11886 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:27:21.900908 11888 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:21.901449 11888 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:21.901849 11888 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:21.905898 11473 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:21.906455 11473 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:21.906873 11473 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:21.839041 11625 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:21.915359  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:21.917073  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:21.925802 11862 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:21.926498 11862 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.926905 11862 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:21.925915  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:21.930051 11898 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.930828 11898 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:21.886134 11804 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:21.935812  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44578.6us
I20260709 12:27:21.940121  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:21.941360  6793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:21.942075  6793 raft_consensus.cc:606] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:27:21.942507  6793 raft_consensus.cc:614] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:21.934235  5840 connection.cc:256] RPC call timeout handler was delayed by -1.9196096919999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15580000000000002s for a response
W20260709 12:27:21.946051  5840 connection.cc:256] RPC call timeout handler was delayed by -1.91060969199998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
W20260709 12:27:21.948045  5840 connection.cc:256] RPC call timeout handler was delayed by -1.81160969199982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
W20260709 12:27:21.948411  5840 connection.cc:256] RPC call timeout handler was delayed by -1.8089096919998155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16810000000000003s for a response
W20260709 12:27:21.948810  5840 connection.cc:256] RPC call timeout handler was delayed by -1.7837096919997748s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
W20260709 12:27:21.949111  5840 connection.cc:256] RPC call timeout handler was delayed by -1.7522096920001786s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
W20260709 12:27:21.949486  5840 connection.cc:256] RPC call timeout handler was delayed by -1.725209692000135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
W20260709 12:27:21.949899  5840 connection.cc:256] RPC call timeout handler was delayed by -1.7009096920000957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
W20260709 12:27:21.950273  5840 connection.cc:256] RPC call timeout handler was delayed by -1.673909692000052s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
W20260709 12:27:21.950724  5840 connection.cc:256] RPC call timeout handler was delayed by -1.6082096919999458s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1904s for a response
I20260709 12:27:21.911487 11685 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:21.910553 11865 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:21.952472  6773 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:21.910748 11713 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:21.957865  5840 connection.cc:256] RPC call timeout handler was delayed by -1.5839096919999065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
W20260709 12:27:21.961362  5840 connection.cc:256] RPC call timeout handler was delayed by -1.4327096920001168s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
W20260709 12:27:21.962195  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 198224us
W20260709 12:27:21.962852  5840 connection.cc:256] RPC call timeout handler was delayed by -1.3472096919999785s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2194s for a response
I20260709 12:27:21.954216  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:21.946106 11680 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:21.965570  5840 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:21.966217 11711 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:21.879959 11460 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:21.970945  5872 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:21.955818  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:21.959614  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.840958 11794 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:21.954800 11791 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:21.957834 11898 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:21.961354 11685 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:21.921707 11625 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:21.988925 11802 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.002s	sys 0.023s
W20260709 12:27:22.000826 11883 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.005s	sys 0.015s
I20260709 12:27:22.001003 11904 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:27:22.000877  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 106827us
I20260709 12:27:22.001858 11902 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:27:21.988695 11835 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.005s	sys 0.018s
I20260709 12:27:21.969012 11587 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.000675 11556 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:22.005813 11802 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.002s	sys 0.023s
W20260709 12:27:22.010434 11835 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.005s	sys 0.019s
I20260709 12:27:22.026553 11835 raft_consensus.cc:697] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:22.027926 11835 consensus_queue.cc:237] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:22.016952  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 51427.1us
I20260709 12:27:22.007215  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:27:22.008982 11883 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.006s	sys 0.016s
I20260709 12:27:22.005529 11625 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:21.998530 11586 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:22.004204 11685 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:22.042881  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:22.051133  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:22.052327  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 86122.5us
I20260709 12:27:22.008491 11902 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:22.058472  5868 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:22.060776  6768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "428a50885681465a98089cb853b233b9"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:22.062937  6768 raft_consensus.cc:606] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:22.063519  6768 raft_consensus.cc:614] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:22.064260  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:22.060606  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:21.990254 11791 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:22.070093  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:22.071906  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:22.066813  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:22.076263  5873 raft_consensus.cc:2393] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 14: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:22.043612 11707 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:22.090963 11908 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:27:22.091652 11707 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:22.093906 11725 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.002s
I20260709 12:27:22.094597 11725 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:22.097343  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 52082.2us
I20260709 12:27:22.099486  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:22.100673 11855 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:22.053815 11808 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:22.104661 11808 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:22.105118 11808 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:22.089919 11896 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:27:22.014231 11899 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
W20260709 12:27:22.119954 11693 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.686s	user 0.014s	sys 0.000s
I20260709 12:27:22.089906 11901 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:22.144140 11901 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:22.144737 11901 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:22.067557 11556 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:22.084394 11909 consensus_queue.cc:1059] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 12:27:22.149535 11909 raft_consensus.cc:3055] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:22.150233 11909 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:22.151355 11909 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:22.153059 11909 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:22.161116 11794 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:22.161917 11693 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20.
I20260709 12:27:22.165086 11473 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:22.175770 11694 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:22.182050  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.068181 11902 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:22.190557  6020 raft_consensus.cc:3055] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:22.191138  6020 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:22.192235  6020 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:22.193588  6020 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:22.195735  6019 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:22.035070 11587 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:22.198872  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:22.200649 11771 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:22.212489  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:22.216672  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:22.217800  6023 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:22.228621 11855 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:22.238320  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1840) took 1190 ms. Trace:
I20260709 12:27:22.124029 11827 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:22.130077  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:22.117213  5913 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:22.252043  5945 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
W20260709 12:27:22.252074  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 134994us
I20260709 12:27:22.253314 11827 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:22.130854  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:22.253880  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:22.131558  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:22.134822 11797 raft_consensus.cc:3055] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:22.256436 11797 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:22.135288 11854 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.136216  5840 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:22.137018 11896 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:22.258191 11854 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:22.259369 11896 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:22.261474 11864 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:22.264070  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:22.264976  5868 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:22.267370  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:22.268347  6015 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:22.260703 11896 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:22.272452 11855 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:22.140542 11884 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.901s	user 0.003s	sys 0.004s
I20260709 12:27:22.143105 11846 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.239043  5940 rpcz_store.cc:276] 0709 12:27:21.048054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.048313 (+   259us) service_pool.cc:224] Handling call
0709 12:27:21.049359 (+  1046us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:21.432012 (+382653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.237900 (+805888us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54004fced4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:22.238190 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":877875072}
I20260709 12:27:22.256036 11694 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:22.275115 11846 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:22.275772 11694 raft_consensus.cc:697] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:22.276820 11694 consensus_queue.cc:237] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:22.256559  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:22.280232  6766 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:22.281116  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:22.284759  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:22.285861  6762 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:22.288406  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:22.289080  6757 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:22.290424  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:22.291620  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:22.292296  6768 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:22.294003  6757 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:22.294736  6757 raft_consensus.cc:606] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:22.294598  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:22.300825  5868 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:27:22.302053 11725 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:22.295854  5873 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:22.317090  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 180968us
I20260709 12:27:22.318548  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:22.317242  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 118509us
W20260709 12:27:22.320174  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 136992us
I20260709 12:27:22.321424  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:22.324944  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:22.332681  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.285185 11855 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.335201  6018 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:22.332165  6609 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:22.343022  5870 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:27:22.356813  5940 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: [18.10-20.11]
I20260709 12:27:22.343888 11898 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:27:22.355937 11725 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:22.358253 11635 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:22.355824 11894 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:22.352715  5949 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:22.355270  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:22.353526 11556 consensus_queue.cc:1059] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:27:22.361974  5947 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:22.349422 11855 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:22.362252  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:22.362742  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:22.351905  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:22.363997  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:22.364370  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:22.364926  5987 leader_election.cc:400] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19.
I20260709 12:27:22.365396 11879 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:22.362262  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:22.365931 11771 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:22.361400  5943 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:22.366605 11771 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:22.367108 11771 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:22.366129  5941 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:22.353179 11829 consensus_queue.cc:1059] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:27:22.362473 11556 raft_consensus.cc:3055] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:22.369513 11829 raft_consensus.cc:3055] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:22.369786 11556 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:22.369957 11829 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:22.370592 11901 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:22.370819 11556 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:22.335206 11895 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.372282 11556 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:22.372237 11895 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:22.373241 11901 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:22.350952 11846 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:22.384536 11806 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:22.384683  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 52676us
I20260709 12:27:22.350296  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:22.366387  5942 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:27:22.386662  5940 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:27:22.388393  5945 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:22.352943 11473 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:22.374899 11725 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.390735  5940 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:22.390908 11725 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:22.392292  6024 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 12:27:22.392772  6024 pending_rounds.cc:85] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 10
I20260709 12:27:22.393047  6024 pending_rounds.cc:107] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:27:22.392928  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:22.393191 11725 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:22.376408  5840 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:22.377127  5941 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df"
dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:48330
I20260709 12:27:22.377771 11674 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.353900 11808 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:22.379662  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:22.380200  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:22.380964  5947 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:27:22.449507  5949 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:22.366328 11879 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:22.452941  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:22.382016  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:22.376147  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:22.455448  5868 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:22.385677  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:22.388381  5948 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:22.395488 11791 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:22.397259 11855 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:22.377390  5913 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:22.456998  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 71452.7us
I20260709 12:27:22.459834  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:22.339664  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:22.422448  5873 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
W20260709 12:27:22.426216  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 49936.2us
I20260709 12:27:22.462634  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:22.437951  6768 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:22.451547  6020 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 12:27:22.471756  6020 pending_rounds.cc:85] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 6
I20260709 12:27:22.472260  6020 pending_rounds.cc:107] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 12:27:22.472692  6020 raft_consensus.cc:2889] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:22.371007 11829 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:22.393414  6024 raft_consensus.cc:2889] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:22.491717  5944 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 19: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:27:22.493893  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 114864us
I20260709 12:27:22.497095  6023 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:22.405774 11846 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:22.498359  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 121056us
W20260709 12:27:22.560082  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 97540.6us
W20260709 12:27:22.561296  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 221716us
I20260709 12:27:22.563028  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:22.561435  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:22.569308 11799 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.042s	sys 0.041s
W20260709 12:27:22.569825 11704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.020s	sys 0.063s
I20260709 12:27:22.427146 11711 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:22.578593  5990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.114s	sys 0.190s
I20260709 12:27:22.586575  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:22.592310  5990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.116s	sys 0.195s
I20260709 12:27:22.480017 11473 raft_consensus.cc:2764] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 12:27:22.592765  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 751938us
I20260709 12:27:22.466524 11904 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:22.593417  5866 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.012s	sys 0.024s
W20260709 12:27:22.681594  5866 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.013s	sys 0.024s
I20260709 12:27:22.499912 11460 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:22.687668  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:22.455519 11894 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.136s
I20260709 12:27:22.454936 11674 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:22.573724 11835 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.136s
I20260709 12:27:22.452749 11797 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:22.503114 11889 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:27:22.840562  5872 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.041s	sys 0.092s
W20260709 12:27:22.808244 11678 consensus_meta.cc:287] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.271s	user 0.009s	sys 0.005s
I20260709 12:27:22.763909  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:22.712586  6768 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:22.594946 11879 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:22.497717 11838 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:27:22.859876 11888 consensus_meta.cc:287] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.958s	user 0.014s	sys 0.000s
I20260709 12:27:22.493305 11857 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:22.692057  6756 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:22.797943 11895 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:22.693499  5867 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.009s	sys 0.051s
W20260709 12:27:22.809383 11922 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:27:22.681731  5913 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:22.699409 11829 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:22.571791 11846 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:22.924419 11694 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.004s	sys 0.006s
I20260709 12:27:22.503589 11864 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.196s
I20260709 12:27:22.702582  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:22.802850 11808 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:22.730163 11625 consensus_meta.cc:287] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.664s	user 0.008s	sys 0.002s
W20260709 12:27:22.870642  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:22.839603  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:22.782549 11916 log.cc:927] Time spent T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
W20260709 12:27:22.794836 11862 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.867s	user 0.009s	sys 0.005s
I20260709 12:27:22.708900  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:22.777675 11771 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:22.764864  6022 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:22.727595 11713 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:22.589174 11635 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:22.751534  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:22.777091  6015 consensus_meta.cc:287] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.001s
I20260709 12:27:22.768189  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:22.960382  6015 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
W20260709 12:27:22.585752 11704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.020s	sys 0.063s
W20260709 12:27:22.585404 11799 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.043s	sys 0.041s
I20260709 12:27:22.765580  6016 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:22.896323  6766 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.616s	user 0.015s	sys 0.003s
I20260709 12:27:22.967847  6766 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:22.559012 11887 raft_consensus.cc:2764] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:27:22.764256  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:22.709666  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 118018us
W20260709 12:27:22.769380 11850 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.156s	user 0.012s	sys 0.002s
I20260709 12:27:22.963165  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:22.803007  5943 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:22.728703  6023 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
W20260709 12:27:22.966657  5868 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.000s
I20260709 12:27:22.819530  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:22.811450 11855 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:22.940243 11896 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.011s	sys 0.043s
I20260709 12:27:22.943356 11797 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:22.977854  6793 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:22.844980 11711 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:22.944625  5872 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.043s	sys 0.116s
W20260709 12:27:22.955199  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.987371  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:22.979460  6609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.024s	sys 0.020s
I20260709 12:27:22.986280  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:22.983949  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:22.947837  5867 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.010s	sys 0.051s
I20260709 12:27:22.945055 11835 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:22.794248 11587 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:23.009209  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:22.949050  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 267457us
I20260709 12:27:23.035334  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:23.036513  6023 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 21: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:27:22.950532 11694 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.004s	sys 0.006s
I20260709 12:27:22.971594  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:22.948638  6756 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:22.998093  6609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.024s	sys 0.020s
I20260709 12:27:23.046291  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:23.046365  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 576716us
I20260709 12:27:22.995204  5868 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:23.006752  6018 raft_consensus.cc:2368] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16.
I20260709 12:27:22.965893 11889 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:22.714385 11854 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:22.946043 11674 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:22.994204 11896 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.011s	sys 0.044s
I20260709 12:27:23.136215  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:23.040664  6015 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 12:27:22.693156 11681 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.308s
I20260709 12:27:23.041863  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:22.975646  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:23.047765  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:23.008191 11835 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:23.115922  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:22.968632 11888 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:23.123184  6022 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 12:27:23.154197  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 111433us
I20260709 12:27:23.037509  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:22.955883 11862 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:23.128849  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:22.943944 11904 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:22.949220 11920 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:23.122269 11889 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:23.185959  5940 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.020s	sys 0.026s
W20260709 12:27:23.186758  5940 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.020s	sys 0.026s
I20260709 12:27:23.102049 11556 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
W20260709 12:27:23.101414  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 56360.5us
W20260709 12:27:23.147883  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 100227us
W20260709 12:27:23.146170  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 583181us
I20260709 12:27:22.985528 11894 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.136s
I20260709 12:27:22.953820 11808 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.168s
I20260709 12:27:23.150990  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:23.176158 11771 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:22.944779 11678 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:23.152207  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 176636us
I20260709 12:27:22.983074 11850 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:23.162021 11794 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:23.162588 11896 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:23.164613 11276 consensus_queue.cc:1059] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:23.167764 11888 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:23.168267 11775 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:23.169219  5871 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: []
W20260709 12:27:23.171965 11827 consensus_meta.cc:287] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.918s	user 0.011s	sys 0.001s
I20260709 12:27:22.999922 11864 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.139s
I20260709 12:27:23.175051  6021 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:23.154881  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:23.046594 11635 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:23.140009  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:22.954206 11625 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:22.950980 11846 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:23.212700  6017 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.044s	user 0.029s	sys 0.042s
I20260709 12:27:23.214272  5915 leader_election.cc:400] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20.
I20260709 12:27:23.219650  5990 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:23.151360  6018 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:27:23.223735  5947 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:23.180148 11904 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:23.195123 11865 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.082s	user 0.021s	sys 0.025s
W20260709 12:27:23.194795 11685 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.119s	user 0.008s	sys 0.008s
I20260709 12:27:23.152782  5866 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:23.191277 11920 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:23.236444  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:23.189715 11460 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
W20260709 12:27:23.323235  6017 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.154s	user 0.030s	sys 0.047s
I20260709 12:27:23.294904  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:23.323599  5868 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:23.348292  5945 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:23.321524  6024 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.023s	sys 0.037s
I20260709 12:27:23.324868  5865 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
W20260709 12:27:23.325201  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 105693us
W20260709 12:27:23.294914  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:23.190169 11716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.011s	sys 0.054s
W20260709 12:27:23.332273 11865 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.220s	user 0.021s	sys 0.025s
W20260709 12:27:23.324190  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 110157us
W20260709 12:27:23.270291  5914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.014s	sys 0.038s
W20260709 12:27:23.331475  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 91529.3us
W20260709 12:27:23.332351  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 489102us
I20260709 12:27:23.338985  5941 raft_consensus.cc:606] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:23.356561  5914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.014s	sys 0.043s
W20260709 12:27:23.342244  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 105954us
W20260709 12:27:23.343923  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.356905  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:23.344624  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1558) took 1001 ms. Trace:
I20260709 12:27:23.356904  5941 raft_consensus.cc:614] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:23.350879  6020 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:23.211385 11921 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
W20260709 12:27:23.346612  6019 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.035s	sys 0.028s
W20260709 12:27:23.359375  6019 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.035s	sys 0.028s
I20260709 12:27:23.328435 11901 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:23.364085  6024 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.024s	sys 0.037s
I20260709 12:27:23.371528  5915 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:23.379433  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:23.383584  6756 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:27:23.331620 11827 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:23.392344  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:23.349923 11607 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:27:23.363535 11716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.013s	sys 0.056s
W20260709 12:27:23.362247 11929 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:27:23.391396 11846 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:27:23.428181  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:23.333572 11635 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:23.396548 11771 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:27:23.402721 11920 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:23.404255 11794 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.405522 11857 raft_consensus.cc:2764] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:27:23.407521 11827 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:23.410017 11901 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:23.410372 11939 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:27:23.412143 11678 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:23.412953 11888 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:23.414039 11625 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:23.414848 11850 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:23.416301 11879 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:23.428857 11775 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:23.362185 11791 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:23.426427  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:23.540827  5865 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:23.428277  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:23.346577 11829 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:27:23.434417  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:23.343948 11685 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:23.450541  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:23.544340  6773 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:27:23.544643  6768 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:23.450623  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:23.293694 11835 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:23.458173  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:23.479842  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:23.391091 11713 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:27:23.514778  5945 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
W20260709 12:27:23.517539  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 85155.5us
I20260709 12:27:23.520009  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:23.514519 11854 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.440052 11587 consensus_queue.cc:1059] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.110s
I20260709 12:27:23.366019  5868 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
W20260709 12:27:23.514148 11902 consensus_meta.cc:287] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.328s	user 0.017s	sys 0.000s
W20260709 12:27:23.525916 11473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.020s	sys 0.018s
I20260709 12:27:23.532531 11879 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:23.572162 11678 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.572376  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:23.454761 11794 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:23.574554  6757 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:23.624739  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:23.516633 11681 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:23.521039 11556 raft_consensus.cc:3055] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:27:23.526787  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 361234us
I20260709 12:27:23.562487 11862 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:23.585377 11864 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:23.633384  5872 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.012s	sys 0.019s
W20260709 12:27:23.634114  5872 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.013s	sys 0.019s
I20260709 12:27:23.634460  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:23.459859 11771 raft_consensus.cc:3055] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:23.598994  6773 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:23.600696  6768 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:23.397012 11904 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:23.524382  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:23.618427  6762 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:23.621745  6793 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:23.572981  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:23.516098 11901 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:23.587999 11799 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.010s	sys 0.017s
I20260709 12:27:23.625453  6021 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:23.481015 11894 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 12:27:23.543566 11276 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 12:27:23.636953  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:23.621518 11806 consensus_meta.cc:287] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.236s	user 0.013s	sys 0.003s
W20260709 12:27:23.620762 11473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.025s	sys 0.021s
I20260709 12:27:23.546798 11772 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 12:27:23.601683 11838 raft_consensus.cc:2764] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 12:27:23.778748  6766 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:27:23.560479 11775 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:23.792797  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:23.793380  5873 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.015s	sys 0.038s
I20260709 12:27:23.592671 11888 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.185s
W20260709 12:27:23.624082 11804 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.668s	user 0.015s	sys 0.007s
I20260709 12:27:23.554559 11625 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.083s
W20260709 12:27:23.812801  6610 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.022s	sys 0.015s
W20260709 12:27:23.815017  5873 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.015s	sys 0.038s
W20260709 12:27:23.794145  6609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.030s	sys 0.023s
W20260709 12:27:23.834946  6609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.030s	sys 0.023s
I20260709 12:27:23.637895 11771 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:27:23.835299  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 623630us
I20260709 12:27:23.635581 11556 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:23.636953 11835 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:23.781215 11799 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.012s	sys 0.023s
I20260709 12:27:23.827827  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:23.781987 11806 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:23.630505 11678 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:23.817719 11940 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:23.833770 11936 log.cc:927] Time spent T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 12:27:23.622037 11902 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:23.837926  6773 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:23.775288 11681 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:23.602448 11854 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:23.836513  6609 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:23.837258 11804 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:23.826578 11904 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:23.776585 11864 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:23.882886  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:23.837070  6610 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.023s	sys 0.015s
I20260709 12:27:23.868173  6778 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 15. Current term is 16. Ops: []
W20260709 12:27:23.851377 11674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.027s	sys 0.048s
W20260709 12:27:23.886777  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.890311  5942 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:23.893203 11806 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:23.893765 11806 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:23.894199 11806 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:23.894796 11941 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:27:23.893958 11473 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:23.896597  6609 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:23.863654 11799 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:23.909565  5869 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
W20260709 12:27:23.910580  6019 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.058s	sys 0.017s
W20260709 12:27:23.911027  6019 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.058s	sys 0.017s
I20260709 12:27:23.912204  5868 pending_rounds.cc:85] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 8
I20260709 12:27:23.912851  5868 pending_rounds.cc:107] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:27:23.913250  5868 raft_consensus.cc:2889] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:23.859733 11909 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:27:23.835983 11879 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:23.920697 11879 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:23.920981  5944 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 12:27:23.921588  5944 pending_rounds.cc:85] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 9
I20260709 12:27:23.921872  5944 pending_rounds.cc:107] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:27:23.922150  5944 raft_consensus.cc:2889] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:23.923774 11797 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.923s	user 0.011s	sys 0.002s
I20260709 12:27:23.924535 11797 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:23.924707  6785 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:27:23.925064 11797 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:23.925465 11680 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.904626 11804 raft_consensus.cc:2764] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:27:23.926144 11680 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:23.916769 11276 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:23.927407 11276 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:23.903237 11799 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:23.928356 11276 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:23.928630 11799 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:23.917201 11635 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:23.930094 11799 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:23.930392 11775 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:23.931094 11775 raft_consensus.cc:3055] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:23.931516 11775 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:23.932646 11775 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:23.933084  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:23.892437 11896 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:23.933808  5870 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:23.934042 11681 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:23.935781 11901 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:23.936425  5944 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:23.934116 11896 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:23.938362 11854 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:23.938699 11864 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:23.939280  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:23.939983  5869 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:23.895334 11942 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:23.944356  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:23.954286  6775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.000s	sys 0.035s
W20260709 12:27:23.955200  6611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.014s	sys 0.037s
I20260709 12:27:23.866207 11940 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:23.926245 11927 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:27:23.975476  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1430) took 2746 ms. Trace:
I20260709 12:27:23.976078 11855 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:23.976830 11855 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:23.975965  6017 rpcz_store.cc:276] 0709 12:27:21.229411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.229608 (+   197us) service_pool.cc:224] Handling call
0709 12:27:21.230168 (+   560us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:21.884120 (+653952us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:21.884174 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:22.167512 (+283338us) spinlock_profiling.cc:243] Waited 4.16 ms on lock 0x7b2c001ddd28. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:27:22.168141 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 12:27:23.347375 (+1179234us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:23.347421 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.347493 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:23.975084 (+627591us) raft_consensus.cc:1685] finished
0709 12:27:23.975107 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.975361 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124370,"raft.queue_time_us":184942,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":3923,"spinlock_wait_cycles":9575168,"thread_start_us":1696218,"threads_started":2,"wal-append.queue_time_us":1377588}
I20260709 12:27:23.957551  5865 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:23.961505  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:23.962878  6775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.000s	sys 0.036s
W20260709 12:27:23.963251  6611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.014s	sys 0.037s
W20260709 12:27:23.949837 11674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.028s	sys 0.048s
I20260709 12:27:23.996315  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:23.997295  5865 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:23.997519  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1589) took 1241 ms. Trace:
I20260709 12:27:23.996219  5915 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:23.962520  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1444) took 1197 ms. Trace:
W20260709 12:27:24.001519  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 643594us
I20260709 12:27:24.006340  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:27:24.006889  5940 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.812s	user 0.013s	sys 0.026s
W20260709 12:27:24.009459  5940 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.013s	sys 0.026s
W20260709 12:27:23.998411  5913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.005s	sys 0.032s
W20260709 12:27:24.010604  5913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.006s	sys 0.032s
W20260709 12:27:24.011067  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 585550us
I20260709 12:27:23.995092  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:24.012182  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:23.998008  6775 rpcz_store.cc:276] 0709 12:27:22.756157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.756332 (+   175us) service_pool.cc:224] Handling call
0709 12:27:22.756832 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:22.757176 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:22.762939 (+  5763us) log.cc:844] Serialized 14 byte log entry
0709 12:27:23.997105 (+1234166us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:23.997174 (+    69us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:23.997229 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.997305 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.997453 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5514,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":290493,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":7153,"thread_start_us":1233600,"threads_started":2,"wal-append.queue_time_us":810622}
W20260709 12:27:23.996600 11607 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.016s	sys 0.008s
I20260709 12:27:24.045115  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1855) took 1664 ms. Trace:
I20260709 12:27:24.054287  5940 rpcz_store.cc:276] 0709 12:27:22.381016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.391668 (+ 10652us) service_pool.cc:224] Handling call
0709 12:27:22.391969 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:22.392181 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:23.187406 (+795225us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.044339 (+856933us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:24.044426 (+    87us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:24.044488 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.044580 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.045043 (+   463us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":837663,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":99,"thread_start_us":1609440,"threads_started":2}
I20260709 12:27:23.999845  6019 rpcz_store.cc:276] 0709 12:27:22.765140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.765280 (+   140us) service_pool.cc:224] Handling call
0709 12:27:22.765648 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:22.765905 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:23.359836 (+593931us) log.cc:844] Serialized 14 byte log entry
0709 12:27:23.962126 (+602290us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:23.962190 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:23.962236 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.962306 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.962451 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52237,"prepare.queue_time_us":852786,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":50728,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"thread_start_us":1142997,"threads_started":2}
W20260709 12:27:24.047155  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44505.1us
I20260709 12:27:23.999054 11276 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:23.999228 11808 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:23.896435 11771 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:23.993042 11635 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.058869  6611 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:24.047847  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 52829.1us
I20260709 12:27:23.998776 11935 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.957170 11556 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:23.993311 11862 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:24.061072  6024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.004s	sys 0.064s
W20260709 12:27:24.023115 11865 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.000s	sys 0.025s
W20260709 12:27:24.053931 11607 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.016s	sys 0.008s
I20260709 12:27:24.044359 11896 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:24.060550  5869 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:24.045558 11713 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:24.053406 11685 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:24.076545  5870 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:24.063437 11950 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:27:24.068766  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:24.058476 11674 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:24.091907  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:24.099956 11635 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:24.114748  5990 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:24.115860 11625 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:24.116824 11625 raft_consensus.cc:697] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:27:24.100806  6024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.005s	sys 0.064s
I20260709 12:27:24.118494  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1439) took 1386 ms. Trace:
I20260709 12:27:24.118237 11625 consensus_queue.cc:237] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:24.118903  6024 rpcz_store.cc:276] 0709 12:27:22.732206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.732441 (+   235us) service_pool.cc:224] Handling call
0709 12:27:22.734388 (+  1947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:22.734642 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:23.368618 (+633976us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.118094 (+749476us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:24.118154 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:24.118203 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.118294 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.118433 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":839044,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":2089,"thread_start_us":1349999,"threads_started":2}
W20260709 12:27:24.100441  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.113529 11887 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:24.120968 11925 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.136s	user 0.003s	sys 0.002s
W20260709 12:27:24.102634 11865 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.000s	sys 0.026s
I20260709 12:27:24.104858 11607 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:24.123616 11607 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:24.125290  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1440) took 1362 ms. Trace:
I20260709 12:27:24.125978  6020 rpcz_store.cc:276] 0709 12:27:22.763364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.763540 (+   176us) service_pool.cc:224] Handling call
0709 12:27:24.124926 (+1361386us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b540041d554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:24.125187 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":932282752}
I20260709 12:27:24.105154 11862 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:24.128279 11850 raft_consensus.cc:3055] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:24.128993 11850 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:24.101264 11935 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:24.131436  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1853) took 1750 ms. Trace:
I20260709 12:27:24.132025  5942 rpcz_store.cc:276] 0709 12:27:22.380483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.390037 (+  9554us) service_pool.cc:224] Handling call
0709 12:27:22.393780 (+  3743us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b5800030398. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5eeee 000000000036eb39 00007f39d7bd21e0 00007f39d7c9b0c4 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:27:22.393925 (+   145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:23.890967 (+1497042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.131028 (+240061us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54003089d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:24.131338 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3446820480}
I20260709 12:27:24.135058 11887 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:23.358013  5946 rpcz_store.cc:276] 0709 12:27:22.343607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.344269 (+   662us) service_pool.cc:224] Handling call
0709 12:27:22.376164 (+ 31895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:22.376385 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:22.846333 (+469948us) log.cc:844] Serialized 14 byte log entry
0709 12:27:23.338659 (+492326us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:23.338751 (+    92us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:23.338817 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.344341 (+  5524us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.344544 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10400,"prepare.queue_time_us":805837,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":15719,"raft.queue_time_us":7533,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":6670,"thread_start_us":941059,"threads_started":2}
I20260709 12:27:24.138236 11887 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:24.116307 11635 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:24.097796  5988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.016s	sys 0.035s
W20260709 12:27:24.139930  5988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.016s	sys 0.035s
W20260709 12:27:24.109402 11946 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
W20260709 12:27:24.141839 11928 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.550s	user 0.002s	sys 0.000s
I20260709 12:27:24.112190 11896 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:24.106535  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 47809.4us
I20260709 12:27:24.144337 11887 raft_consensus.cc:476] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Not starting pre-election -- already a leader
W20260709 12:27:24.144948  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.093148 11794 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:24.097528 11797 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:24.146880  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48330 (request call id 7) took 1773 ms. Trace:
I20260709 12:27:24.147692  5941 rpcz_store.cc:276] 0709 12:27:22.373381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.376941 (+  3560us) service_pool.cc:224] Handling call
0709 12:27:24.146346 (+1769405us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b5400467dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c7d853 00007f39d7c4b60a 00007f39d11d17d4 00007f39d11d1762 00007f39d11d1691 00007f39d11d160c 00007f39d11d08c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:24.146681 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2140826368}
W20260709 12:27:24.148941  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:24.149459  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.145290 11940 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:24.112634  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 43982us
I20260709 12:27:24.151404  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:24.162523 11678 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:24.164299 11948 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 12:27:24.166540  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1860) took 1212 ms. Trace:
I20260709 12:27:24.167114  5949 rpcz_store.cc:276] 0709 12:27:22.954001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.954184 (+   183us) service_pool.cc:224] Handling call
0709 12:27:22.954697 (+   513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:22.989365 (+ 34668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:22.989432 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:23.033487 (+ 44055us) log.cc:844] Serialized 26 byte log entry
0709 12:27:23.449026 (+415539us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:23.449071 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.449140 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:24.166155 (+717015us) raft_consensus.cc:1685] finished
0709 12:27:24.166196 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.166424 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":729,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":14633,"spinlock_wait_cycles":3968,"thread_start_us":298433,"threads_started":1,"wal-append.queue_time_us":718269}
I20260709 12:27:24.171737  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 1361) took 1884 ms. Trace:
I20260709 12:27:24.172335  6762 rpcz_store.cc:276] 0709 12:27:22.287293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.290136 (+  2843us) service_pool.cc:224] Handling call
0709 12:27:24.171411 (+1881275us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b540052a8d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:24.171588 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3052310912}
I20260709 12:27:24.175748 11460 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.176602 11460 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:23.966710 11942 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:24.191737  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 99976.1us
I20260709 12:27:24.153167 11791 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:27:24.190088  6022 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.008s	sys 0.065s
W20260709 12:27:24.202801  6022 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.009s	sys 0.066s
W20260709 12:27:24.153259 11944 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:27:24.200503  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56365.2us
I20260709 12:27:24.204461  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1421) took 1161 ms. Trace:
I20260709 12:27:24.208215  6022 rpcz_store.cc:276] 0709 12:27:23.042954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.154751 (+111797us) service_pool.cc:224] Handling call
0709 12:27:23.155093 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:23.157900 (+  2807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:23.157948 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:23.158569 (+   621us) log.cc:844] Serialized 26 byte log entry
0709 12:27:23.458996 (+300427us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:24.203407 (+744411us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.203985 (+   578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.204103 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:24.204164 (+    61us) raft_consensus.cc:1685] finished
0709 12:27:24.204184 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.204374 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":319822,"prepare.run_cpu_time_us":172,"prepare.run_wall_time_us":5440,"raft.queue_time_us":8712,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"thread_start_us":975114,"threads_started":2,"wal-append.queue_time_us":692959}
W20260709 12:27:24.216863  5987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.021s	sys 0.038s
W20260709 12:27:24.218590  5987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.021s	sys 0.038s
W20260709 12:27:24.220602  6016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.006s	sys 0.017s
W20260709 12:27:24.221221  6016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.006s	sys 0.017s
W20260709 12:27:24.219310  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 585516us
W20260709 12:27:24.226486  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.195150 11587 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:24.066378 11808 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:24.205425 11791 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:24.190647 11953 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:24.192191 11855 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:24.245710  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:24.203763 11716 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:24.187332  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:24.218787 11556 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:24.254784  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:24.256839  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:24.337653  5941 raft_consensus.cc:2368] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21.
I20260709 12:27:24.231923 11635 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:24.194595 11713 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:24.305418  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1208) took 2261 ms. Trace:
I20260709 12:27:24.240060 11678 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:24.220918 11959 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:24.310971  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 73577.9us
I20260709 12:27:24.130203 11850 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:24.315795  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 66972.3us
W20260709 12:27:24.359926  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.363266  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1586) took 1664 ms. Trace:
I20260709 12:27:24.350365  5867 rpcz_store.cc:276] 0709 12:27:22.043932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.044309 (+   377us) service_pool.cc:224] Handling call
0709 12:27:22.066659 (+ 22350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:22.081596 (+ 14937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:22.082372 (+   776us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:22.103892 (+ 21520us) log.cc:844] Serialized 26 byte log entry
0709 12:27:23.001225 (+897333us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:23.130665 (+129440us) log.cc:844] Serialized 14 byte log entry
0709 12:27:23.130991 (+   326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.131117 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:23.978725 (+847608us) spinlock_profiling.cc:243] Waited 8.39 ms on lock 0x7b180026b7e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:27:24.120061 (+141336us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:24.305053 (+184992us) raft_consensus.cc:1685] finished
0709 12:27:24.305091 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.305320 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140896,"prepare.queue_time_us":134510,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":14740,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":16658,"spinlock_wait_cycles":457527552,"thread_start_us":896893,"threads_started":1,"wal-append.queue_time_us":821289}
I20260709 12:27:24.224855 11950 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.317413  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:24.248476 11808 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:24.424993  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 62138.4us
W20260709 12:27:24.339387  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.341142 11901 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.274658 11902 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:24.335682  6793 consensus_meta.cc:287] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.554s	user 0.014s	sys 0.001s
W20260709 12:27:24.499310  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.307991 11952 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.196s
I20260709 12:27:24.499697  6793 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
W20260709 12:27:24.339957  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 333677us
W20260709 12:27:24.309336 11946 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:27:24.267901 11772 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.224357 11674 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 12:27:24.331806 11473 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:27:24.367293  6017 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:24.424538 11716 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:24.369582  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:24.369974  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:24.404083  5872 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.007s	sys 0.030s
W20260709 12:27:24.514905  5872 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.007s	sys 0.030s
I20260709 12:27:24.363474 11635 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:24.407732  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 395565us
W20260709 12:27:24.420466  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:24.502004 11925 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:27:24.423844 11827 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:24.534987 11827 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:24.537714 11850 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:24.539088 11680 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:24.545686 11942 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:24.532575 11625 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.550113 11625 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:24.502995 11950 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:24.561519  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:24.562608  6020 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:24.563937  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:24.565084  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:24.315820 11887 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:24.526487  5913 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:24.577693  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51373us
I20260709 12:27:24.510332 11901 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:24.587035  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:24.587198  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:24.588503  5866 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:24.308284 11921 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:24.591995 11921 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:24.492353  5947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.050s	sys 0.119s
W20260709 12:27:24.593323  5947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.051s	sys 0.119s
I20260709 12:27:24.335855 11685 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.129s
I20260709 12:27:24.359010 11957 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.594205 11957 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:24.595849 11901 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:24.460688  5870 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:24.309134 11945 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.195s
I20260709 12:27:24.349509 11460 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:24.597476 11957 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:24.509778  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:24.599223  6017 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:27:24.363915  6778 rpcz_store.cc:276] 0709 12:27:22.698601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.699256 (+   655us) service_pool.cc:224] Handling call
0709 12:27:22.699703 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:23.885560 (+1185857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.362979 (+477419us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002a21d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:24.363193 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2686424960}
I20260709 12:27:24.359534 11678 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:24.601383 11678 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:24.362046 11959 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.603005 11959 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:24.465016 11804 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.421176  5948 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:24.361869 11587 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:24.466305 11963 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:24.531329  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:24.534345 11775 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:24.515709  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1202) took 2478 ms. Trace:
I20260709 12:27:24.648232  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:24.571023  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:24.477744  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 290512us
I20260709 12:27:24.302326 11794 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:24.611101  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:24.660665  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.578770  5913 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:24.616257  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:24.661943  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 83295.4us
I20260709 12:27:24.662933  5913 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:24.626773  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:24.639170 11625 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:24.639619  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 1372) took 1135 ms. Trace:
I20260709 12:27:24.641284 11685 raft_consensus.cc:3055] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:24.642127 11945 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:24.532418 11716 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:24.645838  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.537577 11827 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:24.696774  5870 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:24.647537  6022 raft_consensus.cc:2410] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 12:27:24.568929  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:24.655627  5987 leader_election.cc:336] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.575s (SENT)
W20260709 12:27:24.704958  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.704773  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:24.656956  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:24.656996  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:24.654058  5872 rpcz_store.cc:276] 0709 12:27:22.037423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.037664 (+   241us) service_pool.cc:224] Handling call
0709 12:27:22.038017 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:22.038235 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:22.999442 (+961207us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.515291 (+1515849us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:24.515354 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:24.515405 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.515505 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.515648 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":699651,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":211019,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":7891,"thread_start_us":2340203,"threads_started":3,"wal-append.queue_time_us":983582}
W20260709 12:27:24.659425  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 88506.6us
I20260709 12:27:24.632280 11854 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:24.632007  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:24.612995 11895 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.518378 11772 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:24.512156 11920 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.142s
I20260709 12:27:24.693661 11685 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:24.666692 11711 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:24.697329  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:24.822712  5866 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:24.698115  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:24.824014  5867 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:27:24.662078  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.824887  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:24.825335  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.654878 11957 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:24.669997  6757 rpcz_store.cc:276] 0709 12:27:23.504372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.627836 (+123464us) service_pool.cc:224] Handling call
0709 12:27:23.628143 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:23.774295 (+146152us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:24.123857 (+349562us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b480052c158. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d7bacaa1 00007f39d7ba3b37 00007f39d7ba39ca 00007f39d7c9b26f 00007f39d7c9af37 00007f39d7caf55c 00007f39d7caf508 00007f39d7caf488 00007f39d7caf443 00007f39d7cae6ec 00007f39d6bd8a96
0709 12:27:24.201858 (+ 78001us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.639064 (+437206us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:24.639150 (+    86us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:24.639220 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.639324 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.639522 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124122,"prepare.queue_time_us":588423,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":23820,"raft.queue_time_us":202799,"raft.run_cpu_time_us":3011,"raft.run_wall_time_us":343747,"spinlock_wait_cycles":11819904,"thread_start_us":723760,"threads_started":2,"wal-append.queue_time_us":422582}
W20260709 12:27:24.709278  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 52410.6us
I20260709 12:27:24.835695  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:24.707260  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:24.712070  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:24.556249 11635 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:24.837226  5943 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:24.713809  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:24.615691 11887 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:24.663522 11680 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:24.701853 11945 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:24.839083 11680 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:24.818509 11854 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.733438  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1209) took 2400 ms. Trace:
I20260709 12:27:24.733359  5872 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
W20260709 12:27:24.840107 11954 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.379s	user 0.001s	sys 0.000s
I20260709 12:27:24.840444  5872 pending_rounds.cc:85] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 7
I20260709 12:27:24.677870  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:24.666230 11846 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:24.841408  5941 raft_consensus.cc:2410] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:27:24.840639 11854 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:24.841989 11951 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.410s	user 0.001s	sys 0.001s
I20260709 12:27:24.842734 11846 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:24.666891 11902 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:24.843284 11970 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:27:24.843629 11902 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:24.844863  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1944) took 1806 ms. Trace:
W20260709 12:27:24.717469  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 60552.8us
I20260709 12:27:24.846068  5840 leader_election.cc:400] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16.
I20260709 12:27:24.846400  5840 leader_election.cc:403] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:27:24.845293  5873 rpcz_store.cc:276] 0709 12:27:23.038593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.038841 (+   248us) service_pool.cc:224] Handling call
0709 12:27:23.136389 (+ 97548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:23.136823 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:23.136854 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:23.137389 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 12:27:23.834521 (+697132us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:23.834572 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.834656 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:24.844589 (+1009933us) raft_consensus.cc:1685] finished
0709 12:27:24.844616 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.844791 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":310,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":8128,"thread_start_us":621337,"threads_started":1,"wal-append.queue_time_us":883225}
I20260709 12:27:24.818841  5946 raft_consensus.cc:2393] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:24.821030  6793 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: [16.9-18.10]
I20260709 12:27:24.821696  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:24.714339 11889 consensus_meta.cc:287] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.517s	user 0.015s	sys 0.001s
I20260709 12:27:24.847723  6020 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:24.848263 11955 log.cc:927] Time spent T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 12:27:24.843276 11846 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:24.848974 11949 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.245s
I20260709 12:27:24.850128 11966 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.847762 11889 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:24.850736 11966 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:24.851594  6018 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:24.833272  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:24.852681  6762 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:24.852914 11889 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:24.699766 11904 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.815s	user 0.009s	sys 0.003s
I20260709 12:27:24.839603 11680 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:24.647473 11804 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:24.854008  5871 rpcz_store.cc:276] 0709 12:27:22.333479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.333751 (+   272us) service_pool.cc:224] Handling call
0709 12:27:22.334292 (+   541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:23.326642 (+992350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.733097 (+1406455us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b54003890d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:24.733363 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1919476096}
I20260709 12:27:24.840802  5872 pending_rounds.cc:107] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:27:24.855280  5872 raft_consensus.cc:2889] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:24.855553 11804 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:24.856279  6778 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:24.856812 11902 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:24.855605  5872 pending_rounds.cc:107] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 17.9
W20260709 12:27:24.847167 11956 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 12:27:24.857580  5872 raft_consensus.cc:2889] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:24.853518 11965 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:24.853132 11966 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:24.849599 11846 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:24.862699 11846 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:24.864645  6611 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:24.832207  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:24.867112  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:24.717012 11827 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:24.874763  6020 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:24.734246 11607 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:27:24.876466  5990 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 2.191s (SENT) [suppressed 4 similar messages]
I20260709 12:27:24.877395  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:24.877595  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:24.880134  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 44598.3us
W20260709 12:27:24.882206  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 226618us
I20260709 12:27:24.883584  5943 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:27:24.903002  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:24.905213  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 198083us
I20260709 12:27:24.906903  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:24.909900 11902 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:24.910548 11971 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 12:27:24.911568 11902 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:24.912068 11902 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:24.912451  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 66514.7us
W20260709 12:27:24.915827 11983 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.257s	user 0.000s	sys 0.000s
W20260709 12:27:24.917172 11973 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 12:27:24.841997 11945 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:24.918556  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1211) took 1014 ms. Trace:
I20260709 12:27:24.919744 11950 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:24.918965  5868 rpcz_store.cc:276] 0709 12:27:23.904240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.904533 (+   293us) service_pool.cc:224] Handling call
0709 12:27:23.911881 (+  7348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:23.919158 (+  7277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:23.919193 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:23.919756 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 12:27:24.233483 (+313727us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:24.716576 (+483093us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.799540 (+ 82964us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.799658 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:24.918292 (+118634us) raft_consensus.cc:1685] finished
0709 12:27:24.918320 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.918496 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82540,"raft.queue_time_us":85498,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":12393,"thread_start_us":717912,"threads_started":2,"wal-append.queue_time_us":696611}
I20260709 12:27:24.923056 11678 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:24.927767  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1950) took 1503 ms. Trace:
I20260709 12:27:24.928262  5874 rpcz_store.cc:276] 0709 12:27:23.424073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.424233 (+   160us) service_pool.cc:224] Handling call
0709 12:27:23.449057 (+ 24824us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:23.517426 (+ 68369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:23.517482 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:23.539869 (+ 22387us) log.cc:844] Serialized 26 byte log entry
0709 12:27:24.103552 (+563683us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:24.103623 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.103726 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:24.927510 (+823784us) raft_consensus.cc:1685] finished
0709 12:27:24.927534 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.927695 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2729,"raft.queue_time_us":56153,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":14062,"thread_start_us":448890,"threads_started":1,"wal-append.queue_time_us":1174873}
I20260709 12:27:24.671906 11794 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:24.928313  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1444) took 1504 ms. Trace:
I20260709 12:27:24.935614  5866 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
W20260709 12:27:24.941221  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 50138.1us
I20260709 12:27:24.913369  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:24.941833  6016 rpcz_store.cc:276] 0709 12:27:23.423717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.424357 (+   640us) service_pool.cc:224] Handling call
0709 12:27:23.448861 (+ 24504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:23.449277 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:23.449315 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:23.550658 (+101343us) log.cc:844] Serialized 26 byte log entry
0709 12:27:24.221892 (+671234us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:24.321744 (+ 99852us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.361497 (+ 39753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.361609 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:24.928096 (+566487us) raft_consensus.cc:1685] finished
0709 12:27:24.928119 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.928254 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38369,"prepare.queue_time_us":79670,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":5034,"raft.queue_time_us":113745,"raft.run_cpu_time_us":433,"raft.run_wall_time_us":88960,"spinlock_wait_cycles":1429888,"thread_start_us":587556,"threads_started":1,"wal-append.queue_time_us":877418}
W20260709 12:27:24.952312  5987 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 2.024s (SENT)
I20260709 12:27:24.952888  5987 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:24.953272  6609 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:24.659722 11963 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:24.955643  6775 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:24.954269  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:24.875013 11711 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:24.862952 11680 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:24.883738 11804 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:24.885653 11587 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16.
I20260709 12:27:24.819727 11895 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:24.971530  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:24.966502 11950 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:24.969046  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:24.940729 11827 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:24.986479  5943 raft_consensus.cc:2410] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 21 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:27:24.912469 11902 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:24.932510 11473 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:24.876137 11957 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:24.942145 11607 raft_consensus.cc:697] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:24.883507 11942 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 12:27:24.993508  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:24.985392 11806 consensus_meta.cc:287] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.090s	user 0.011s	sys 0.005s
I20260709 12:27:24.986940 11950 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:25.005045  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 62024.7us
W20260709 12:27:25.021498  5987 leader_election.cc:336] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 2.088s (SENT)
I20260709 12:27:25.022034  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:24.966966 11952 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:24.994889 11678 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:24.946195 11904 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:27:24.986104 11775 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:25.032104  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 101477us
I20260709 12:27:25.040813  6017 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
W20260709 12:27:25.024554 11799 consensus_meta.cc:287] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.094s	user 0.012s	sys 0.002s
I20260709 12:27:24.986078 11953 consensus_queue.cc:1059] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:27:25.050455  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:25.025396 11967 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:25.130090 11982 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:27:25.131867 11716 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:25.038266 11857 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:25.137902  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:25.012952  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 136510us
I20260709 12:27:25.016412 11473 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:25.015282 11804 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:25.047466 11460 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:25.148015  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:25.023267 11806 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:25.052611 11952 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:25.155805  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:25.156507  5946 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:25.157888  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:25.158811  5946 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:25.160113  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:25.162406  5946 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:25.023021 11950 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:25.021310 11587 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:25.166179 11587 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:25.146534  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:25.166603 11587 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.167029  5941 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:25.167104 11587 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:25.148698  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:25.107690  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 138709us
I20260709 12:27:25.170176  6609 leader_election.cc:400] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21.
I20260709 12:27:25.171087 11473 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:25.150359 11804 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:25.172864 11977 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.129388 11794 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:25.174736 11804 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:25.176340  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:25.164918  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 114530us
I20260709 12:27:25.166548  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:25.113507  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 200208us
W20260709 12:27:25.148296 11835 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.305s	user 0.009s	sys 0.006s
I20260709 12:27:25.183468  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:25.190320  5942 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:25.192276  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.199239  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:25.203938  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.209873  5913 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:25.141595 11921 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:25.211016  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68268.3us
W20260709 12:27:25.178324 11879 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.257s	user 0.011s	sys 0.005s
I20260709 12:27:25.212803  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:25.150518 11920 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:25.214651  5942 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:25.114374 11711 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:25.140892 11827 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:25.219099  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:25.219234  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.225042  5987 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:25.230186  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.223974  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.229780  5868 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:25.237318  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:25.238643  6757 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 16: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:25.238965  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:25.218302  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:25.195276 11974 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:25.253611  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:25.258941  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 195913us
I20260709 12:27:25.259227  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:25.259656  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 45583.4us
I20260709 12:27:25.248088  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:25.247150  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:25.258167  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:25.267472  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:25.205511 11967 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.250415  6609 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:25.173460 11977 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:25.276120  6768 raft_consensus.cc:2410] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:27:25.275847  6773 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 12.
I20260709 12:27:25.276262  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:25.280730  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:25.282074  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:25.226205 11835 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:25.277391  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:25.295773  6766 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:25.224678 11887 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:25.220142 11953 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:25.222935 11895 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:25.202746 11904 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:27:25.289114  6762 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:27:25.289747  6793 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:25.295885  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:25.247030 11959 consensus_meta.cc:287] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.592s	user 0.009s	sys 0.002s
I20260709 12:27:25.261023 11772 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:25.296995  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:25.248272 11966 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:25.279719 11879 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:25.297195  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:25.279465 11965 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:25.312842  5990 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:25.317619  6766 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:27:25.317380  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:27:25.321935  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 112192us
I20260709 12:27:25.323779  6757 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:25.324481  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:25.339252  5867 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:25.313671 11680 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:25.326920 12003 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:27:25.327867  5873 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:27:25.150748 11607 consensus_queue.cc:237] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:25.298623 11704 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:25.343215  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.301723 11835 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:25.354823  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:25.362100  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:25.319831 11966 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.323071 11879 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:25.346800 12011 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:25.369275  6785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1583) took 2676 ms (client timeout 2191 ms). Trace:
I20260709 12:27:25.401533  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:25.409834  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 92589.4us
W20260709 12:27:25.414002  6785 rpcz_store.cc:269] 0709 12:27:22.687017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.687224 (+   207us) service_pool.cc:224] Handling call
0709 12:27:25.363318 (+2676094us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400008a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:25.363520 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2823395584}
I20260709 12:27:25.278132 11974 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:25.284907 12005 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.131s
W20260709 12:27:25.368271 11808 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.872s	user 0.011s	sys 0.003s
I20260709 12:27:25.287247 11967 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:25.408263 11966 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:25.348096 11772 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:25.410800  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 124793us
I20260709 12:27:25.367360 11835 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:25.319550 11959 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:25.360076 11704 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:25.230518 11920 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:25.361685 11887 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:25.313967 11952 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:25.328881 11902 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:25.476567  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:25.413479 11879 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:25.452489  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:25.452816  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.432636 11957 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:25.464676  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:25.502238  6017 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:25.427150 11963 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:25.439124 12005 consensus_queue.cc:1243] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:25.417975  6785 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:25.473573  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:25.439075 11772 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.474443  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:25.466105 11981 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:27:25.486291  6020 raft_consensus.cc:2410] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:27:25.518584  5873 raft_consensus.cc:2368] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 13. Current term is 14.
I20260709 12:27:25.488440  5869 raft_consensus.cc:2410] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 12:27:25.485173 11887 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:25.489429 12006 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 12:27:25.490233 12010 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:25.485647 11904 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.467701 11704 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:25.495189 12017 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.498400 11977 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:25.499366  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.502586  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:25.438299 11808 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:25.492316 11276 consensus_meta.cc:287] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.426s	user 0.013s	sys 0.000s
I20260709 12:27:25.523161 11772 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:25.490726 12002 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.528365  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.485309 11989 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:25.538132  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:25.422487 12011 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:25.525451  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.519863 11965 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:25.544778  5872 raft_consensus.cc:2410] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:27:25.533696 12005 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:25.560855  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 58397.7us
I20260709 12:27:25.560870  5873 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:25.490233 12009 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.102s
I20260709 12:27:25.540215 12010 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.580587  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.580595  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.549407 11947 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:25.576097 11556 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.221s	user 0.014s	sys 0.002s
I20260709 12:27:25.612720  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.575721 11940 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:25.613860  6021 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:25.614125  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:25.587361  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 232614us
I20260709 12:27:25.610478  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:25.618219  5947 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:25.618925 11967 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:25.619657  5947 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:25.622160 11963 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.611114  5869 raft_consensus.cc:2393] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:25.622879 11963 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:25.622984 11587 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:25.546007 12017 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:25.624194  5867 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:25.624292  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:25.625735  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.627622  5867 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:25.625756  5872 raft_consensus.cc:2393] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:25.585685 11827 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.626312 12017 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:25.626713 11942 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:27:25.610512 12010 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:25.632349  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:25.633845  5872 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 21: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:25.541031 11904 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:25.639364  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:25.639349  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.636694  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.642186  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.643540  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.644497  5872 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:25.645167  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:25.643692  5867 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:25.588106 12005 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:25.649308 12011 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:25.650374 12024 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:27:25.650590 12005 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:25.651901  5868 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 10. (term mismatch)
I20260709 12:27:25.652392  5868 pending_rounds.cc:85] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 9
I20260709 12:27:25.652680  5868 pending_rounds.cc:107] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 20.10
I20260709 12:27:25.652925  5868 raft_consensus.cc:2889] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:25.657214  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.658310  5869 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 19: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:25.657109  5867 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:27:25.655740  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:25.659898 12025 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:27:25.668798 11797 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:25.669323 11797 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:25.587003 11965 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:25.670300 11797 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:25.673004 11965 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:25.674085 11887 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:25.628654 11894 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:27:25.563557 12002 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:25.664781  5865 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:25.665082  5868 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:25.638142 11827 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:25.681187 11808 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:25.682159 11587 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:25.682493 11827 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:25.684134 11850 consensus_meta.cc:287] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.145s	user 0.012s	sys 0.002s
I20260709 12:27:25.684011 11704 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:25.685011  5871 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
W20260709 12:27:25.678403  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:25.685730 11791 consensus_meta.cc:287] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.382s	user 0.008s	sys 0.007s
I20260709 12:27:25.686209 11791 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:25.687258 11904 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:25.687520 11704 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:25.688418 11791 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:25.696651 11984 raft_consensus.cc:3055] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:25.697216 11984 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:25.698630  5873 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:25.699572  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:25.700634  5873 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:25.689365  5868 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:25.689785 12002 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:25.693861  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.694483 11797 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:25.698177 11772 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:25.704586  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:25.707743  5941 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:25.713295  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:25.719532  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 63863.7us
W20260709 12:27:25.721359  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 183809us
I20260709 12:27:25.722536  6017 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:25.722671  5873 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:27:25.724380  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.724570  5873 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:25.727730  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.728099  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:25.725018  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:25.728711  5866 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:25.730188  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:25.730548  5866 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:25.730839  6017 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:27:25.731854  5866 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:25.731935  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:25.733069  5866 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:25.724421  5840 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:25.737051 11772 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:25.698278 11984 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:25.723428 11775 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:25.727167  5873 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:25.738365  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:25.755085  5945 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:27:25.742561 11854 consensus_meta.cc:287] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.900s	user 0.013s	sys 0.002s
I20260709 12:27:25.743042  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:25.723315 11950 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.557s	user 0.013s	sys 0.001s
I20260709 12:27:25.755945  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:25.760519  6020 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:25.760948 11984 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:25.761364 11950 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:25.762245  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:25.763638  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:25.764118  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.764763 11902 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:25.765203  6020 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:25.761662  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:25.768656  5941 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:25.765264 11950 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:25.771900  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1450) took 1406 ms. Trace:
I20260709 12:27:25.772414  6024 rpcz_store.cc:276] 0709 12:27:24.365328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.366033 (+   705us) service_pool.cc:224] Handling call
0709 12:27:24.464489 (+ 98456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:24.968164 (+503675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.968200 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:24.968805 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 12:27:25.282026 (+313221us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:25.282085 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.771403 (+489318us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b54003642d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:25.771460 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:25.771522 (+    62us) raft_consensus.cc:1685] finished
0709 12:27:25.771535 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:25.771825 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1157567232,"thread_start_us":267073,"threads_started":1,"wal-append.queue_time_us":328668}
I20260709 12:27:25.743844 11835 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:25.777606  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 53257.4us
I20260709 12:27:25.778959 12037 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:25.779192 11960 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 1.407s	user 0.007s	sys 0.005s
I20260709 12:27:25.780035  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1451) took 1414 ms. Trace:
I20260709 12:27:25.775694 11854 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:25.781682 11962 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 1.264s	user 0.007s	sys 0.015s
I20260709 12:27:25.783527 11966 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:25.784606 11945 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:25.788067  5940 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:25.788834 11674 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:25.792164  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:25.795563  6019 rpcz_store.cc:276] 0709 12:27:24.365653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.365783 (+   130us) service_pool.cc:224] Handling call
0709 12:27:24.366157 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:24.602980 (+236823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.603022 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:24.603080 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:24.603126 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.779741 (+1176615us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54006b9ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:25.779809 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:25.779967 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":544081792}
I20260709 12:27:25.802640  5941 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:25.827291  5872 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 16. Ops: [14.7-14.7]
I20260709 12:27:25.829380  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:25.834306  5942 raft_consensus.cc:2368] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
I20260709 12:27:25.836874  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:25.794982 12037 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.838568  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:25.839773  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:25.840754  6017 raft_consensus.cc:2368] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17.
I20260709 12:27:25.841203  5866 raft_consensus.cc:2376] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19. Re-sending same reply.
I20260709 12:27:25.842327  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.842447  6611 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:25.807138 11850 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.846088  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.846436  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:25.843072  5869 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:25.847028  5866 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:25.845034 11772 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:25.847194  6020 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:25.813117 11674 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:25.813136 11904 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:25.827883 11989 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:25.873873 12011 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
W20260709 12:27:25.877547 12039 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:25.878026 11854 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:25.879649  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.853757 11850 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:25.959512  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:25.844158 12037 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:25.961023 11276 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:25.962244 11967 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:25.963022 11967 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:25.963979 11967 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:25.880640  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:25.966596  5866 raft_consensus.cc:2410] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:27:25.967953 11713 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:25.968135  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:25.969120  5866 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:25.848963 11950 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:25.885988 11862 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:25.887956 11879 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:25.875386 11952 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:25.838140 11794 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:25.893910  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:25.795068 12010 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:25.971834  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:25.974036 12010 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:25.973340  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:25.974946  5945 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260709 12:27:25.975701 11985 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.859s	user 0.001s	sys 0.001s
I20260709 12:27:25.975831 11950 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:25.978425  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:25.978744  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:25.979751  6022 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:25.980403  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:25.982712  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:25.984282 11950 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:25.985013  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1861) took 2817 ms. Trace:
I20260709 12:27:25.985553  5948 rpcz_store.cc:276] 0709 12:27:23.168026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.168193 (+   167us) service_pool.cc:224] Handling call
0709 12:27:24.252377 (+1084184us) spinlock_profiling.cc:243] Waited 6.97 ms on lock 0x7b5800030398. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:27:24.252711 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:24.620700 (+367989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.984694 (+1363994us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54004fced4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:25.984909 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":290787072}
I20260709 12:27:25.850865  6611 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:25.948313  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:25.950644  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.907317  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:25.952276  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1596) took 2528 ms. Trace:
I20260709 12:27:25.851225 11827 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.851542 11829 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:25.960989  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:25.988488  6756 rpcz_store.cc:276] 0709 12:27:23.421364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.424357 (+  2993us) service_pool.cc:224] Handling call
0709 12:27:23.424762 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:23.455785 (+ 31023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:23.455819 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:23.456458 (+   639us) log.cc:844] Serialized 26 byte log entry
0709 12:27:24.000859 (+544401us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:24.000921 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.001014 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:25.780872 (+1779858us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:25.892509 (+111637us) raft_consensus.cc:1685] finished
0709 12:27:25.892534 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:25.949875 (+ 57341us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19757,"raft.queue_time_us":147281,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":27096,"spinlock_wait_cycles":7071488,"thread_start_us":451543,"threads_started":1,"wal-append.queue_time_us":993378}
W20260709 12:27:25.988310  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 137545us
I20260709 12:27:25.993269  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1139) took 1770 ms. Trace:
I20260709 12:27:25.999406  5949 rpcz_store.cc:276] 0709 12:27:24.222496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.223324 (+   828us) service_pool.cc:224] Handling call
0709 12:27:24.223752 (+   428us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:24.255399 (+ 31647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.255482 (+    83us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:24.277250 (+ 21768us) log.cc:844] Serialized 52 byte log entry
0709 12:27:24.708982 (+431732us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:24.994093 (+285111us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.994312 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.994391 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:25.172482 (+178091us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b1800416980. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:27:25.975047 (+802565us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:25.992905 (+ 17858us) raft_consensus.cc:1685] finished
0709 12:27:25.992936 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:25.993171 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30457,"prepare.queue_time_us":406933,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":46396,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":32024,"spinlock_wait_cycles":12803712,"thread_start_us":420792,"threads_started":2,"wal-append.queue_time_us":643142}
W20260709 12:27:26.002664  6018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 1446) took 2546 ms (client timeout 1845 ms). Trace:
W20260709 12:27:26.003146  6018 rpcz_store.cc:269] 0709 12:27:23.441650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.457230 (+ 15580us) service_pool.cc:224] Handling call
0709 12:27:25.987844 (+2530614us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400201a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:25.987970 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2470585088}
I20260709 12:27:25.984339  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:25.993786  6020 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:27:25.994285  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:26.005007  5874 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 21: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:25.989612  6024 raft_consensus.cc:2410] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:27:25.849427 11772 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.015255  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:26.015631 12041 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:27:26.016757 11835 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:25.872488 11887 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.129s
I20260709 12:27:26.006484 11556 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:26.002097 11829 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:26.018069 11674 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:25.988997 11966 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:26.007674 11827 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:25.954366 12016 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.020131 11713 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:25.996851 11950 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:25.851086 11775 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.157s
I20260709 12:27:26.025853  6020 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:26.036890  6022 raft_consensus.cc:2410] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:27:26.039541  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:26.043340  5913 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:26.000469 11965 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.996707 12017 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:26.048418  6024 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:26.058462  6611 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:26.045590 11772 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:26.061055  5866 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:26.061316 11989 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:26.062469 12010 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:26.063154  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:26.048206 11829 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:26.065500  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:26.047978 11556 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:26.067888  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:26.069339  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:26.072769  5869 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:26.073344  6785 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:27:26.075098  6018 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:26.075764  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:26.084007  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:26.084668  6785 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:26.085397  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:26.085901  6785 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:26.086609  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:26.086704  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:26.087855  6785 raft_consensus.cc:2410] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:27:26.087798  5840 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:26.088526  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:26.089241  5840 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:26.090106  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:26.086314  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:26.059327 11952 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:26.064450 11674 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:26.059648 11713 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:26.070613 11950 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:26.095263  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:26.057065 12016 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:26.071928 11989 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.098927  5948 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:26.102375  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:26.104928 11713 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:26.066887 11965 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:26.107712 11965 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:26.108632 11952 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.109272 11952 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:26.112701  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:26.076344 11556 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:26.116441  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:26.113441  5944 raft_consensus.cc:2376] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17. Re-sending same reply.
I20260709 12:27:26.090315 11681 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:26.093374 11904 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:26.068902 12020 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:27:26.111824 11952 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:26.121778  5866 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:26.142252  5949 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:26.069154 11902 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:27:26.123001  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:26.095595 12027 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:26.131837  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:26.143371  5866 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:26.070855 12017 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:26.138841  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:26.145419  5945 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:26.147676  5869 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:26.153005  5948 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:26.153079  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:26.118685 11989 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:26.091131 11835 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:27:26.154769  5944 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:26.121275 11950 raft_consensus.cc:697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:26.129371 11904 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.155521 11997 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:26.149935 12027 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.179627  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:26.181769  5941 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:26.182026  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:26.154501 11935 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:26.129606 11932 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:27:26.192881  5941 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:26.173832 11904 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:26.196813 11935 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:26.197819 11935 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:26.200824 11935 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:26.202330  6775 raft_consensus.cc:606] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:26.202935 11932 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:26.203536 11932 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:26.204471 11932 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:26.206753 12016 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:26.187559 12027 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:26.202981  6775 raft_consensus.cc:614] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:26.213348 12002 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:26.214116 12002 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:26.215368 12002 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:26.192417 11997 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:26.222039 11827 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:26.255982 11997 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:26.235195  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:26.258826  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.236977  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:26.268625  5945 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
W20260709 12:27:26.286528  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:26.248315 12006 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:27:26.262768  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:26.249172 11989 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:26.253986 11950 consensus_queue.cc:237] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:26.286206 12020 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.226326 11932 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:26.239706 12017 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:26.290889  5913 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:26.240262 11797 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.301985 11959 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:26.305562  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:26.309275  5872 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:27:26.201390 11829 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:26.318363  5872 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:26.252718 12037 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:26.309922 11996 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.133s	user 0.022s	sys 0.023s
I20260709 12:27:26.288466 11902 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.022s
W20260709 12:27:26.328161  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.328095  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:26.314555 11992 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:26.332052  5869 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:26.330061  5872 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:26.317139 11797 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:26.310787 12020 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:26.340265  5866 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: [16.8-18.9]
W20260709 12:27:26.330385 11996 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.149s	user 0.022s	sys 0.027s
I20260709 12:27:26.309082 11904 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:26.347987  5941 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:26.341861 11846 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:26.355389 11995 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:26.355988 11995 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:26.364703 11827 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.365469 11827 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:26.370720 11827 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:26.344139 11992 raft_consensus.cc:697] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:26.364063 11829 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:26.372341 11992 consensus_queue.cc:237] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:26.375371 11797 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:26.376058  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1571) took 1230 ms. Trace:
I20260709 12:27:26.376576  5943 rpcz_store.cc:276] 0709 12:27:25.145372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.145590 (+   218us) service_pool.cc:224] Handling call
0709 12:27:25.146004 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:25.280112 (+134108us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:25.280164 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:26.233317 (+953153us) spinlock_profiling.cc:243] Waited 6.26 ms on lock 0x7b2c001b1e88. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:27:26.233832 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 12:27:26.338568 (+104736us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:26.338626 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.338821 (+   195us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:26.375669 (+ 36848us) raft_consensus.cc:1685] finished
0709 12:27:26.375693 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.375989 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":60337,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"spinlock_wait_cycles":14387456,"thread_start_us":232640,"threads_started":2,"wal-append.queue_time_us":139490}
I20260709 12:27:26.357081 11995 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:26.379202 11995 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:26.379727 11772 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:26.374720 12027 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:26.389323 12027 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:26.391530  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:26.392232  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.393050  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:26.394629  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:26.395320  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:26.397015  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:26.398520  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:26.401551  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1581) took 1043 ms. Trace:
I20260709 12:27:26.398288 11989 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:26.402268  5942 rpcz_store.cc:276] 0709 12:27:25.357964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.358210 (+   246us) service_pool.cc:224] Handling call
0709 12:27:26.401261 (+1043051us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b5400833954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:26.401457 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1077039488}
I20260709 12:27:26.403991  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:26.405958 12046 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:27:26.407641 11904 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:26.408629  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:26.397248 11963 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:26.414875  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:26.417788  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:26.418814 12037 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:26.419864  5941 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:26.420323 11771 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:26.422768 11974 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:26.419171  5870 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:26.428602  5913 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:26.429412  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1423) took 2850 ms. Trace:
I20260709 12:27:26.435833 11989 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:26.436439 11989 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:26.437328 11989 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:26.438228  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:26.439162  6756 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:26.439244  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:26.439985  6793 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 22: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:26.440351  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:26.440793  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:26.440960  6756 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:26.441398  6793 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:26.441881  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:26.435556  6021 rpcz_store.cc:276] 0709 12:27:23.579328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.819525 (+240197us) service_pool.cc:224] Handling call
0709 12:27:23.838702 (+ 19177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:25.614310 (+1775608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.429084 (+814774us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400562a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:26.429334 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4081340288}
I20260709 12:27:26.442637  6785 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:26.449328  5873 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:26.452953  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1448) took 2088 ms. Trace:
I20260709 12:27:26.453434  6015 rpcz_store.cc:276] 0709 12:27:24.364698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.368593 (+  3895us) service_pool.cc:224] Handling call
0709 12:27:24.369081 (+   488us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:24.584548 (+215467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.584595 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:24.585456 (+   861us) log.cc:844] Serialized 52 byte log entry
0709 12:27:24.804223 (+218767us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:24.804299 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.452475 (+1648176us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b5400363dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:26.452535 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:26.452593 (+    58us) raft_consensus.cc:1685] finished
0709 12:27:26.452606 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.452879 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17864,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":184,"spinlock_wait_cycles":494190848,"thread_start_us":218218,"threads_started":1,"wal-append.queue_time_us":437695,"wal-append.run_cpu_time_us":2194,"wal-append.run_wall_time_us":1106143}
I20260709 12:27:26.457772 11806 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:26.459898 11920 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:26.460727 12011 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:26.461167 12011 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:26.429900 12003 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:26.468075  6756 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:26.431331 11556 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:26.433342  5869 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:27:26.431433 11933 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:26.472833 11933 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:26.476725 11989 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:26.487775  5870 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:26.469560  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:26.489779  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:26.491555  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:26.492430  5870 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:26.510102  6793 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:26.514185  5840 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:26.515261  5840 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:26.483001 11933 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:26.524345 11989 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:26.523952 11685 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:26.527963  5870 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:26.529158  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:26.498428 11950 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.535404 11921 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.480580 11556 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:26.513445 11904 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:26.521903 12037 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:26.524117 11933 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:26.477208 11902 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:27:26.539551  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:26.541953  5870 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: [17.10-17.10]
W20260709 12:27:26.542858  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.562552  6019 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:26.564502  6024 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:26.555867 11967 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:26.566701  6015 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:26.566879 11967 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:26.569921 12037 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:26.572362  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.530272  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1186) took 1722 ms. Trace:
I20260709 12:27:26.573087  6778 rpcz_store.cc:276] 0709 12:27:24.807566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.807745 (+   179us) service_pool.cc:224] Handling call
0709 12:27:24.808085 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:24.856712 (+ 48627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.529875 (+1673163us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b5400470c54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:26.530184 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110384896}
I20260709 12:27:26.550629  6016 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:26.539641 12011 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:26.575256 12011 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:26.578123 11879 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:26.557328 11904 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.579350 11904 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:26.579535 11850 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:26.579813  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:26.581298  5869 raft_consensus.cc:2410] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:27:26.583000  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1453) took 2216 ms. Trace:
I20260709 12:27:26.534263 11685 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:26.559551 11921 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:26.571413 12037 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:26.583117  5869 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:26.592696  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:26.606761  5870 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 19: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:26.583629  6023 rpcz_store.cc:276] 0709 12:27:24.366252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.366428 (+   176us) service_pool.cc:224] Handling call
0709 12:27:24.498699 (+132271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:24.817804 (+319105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.817836 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:24.818348 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 12:27:25.014341 (+195993us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:25.014391 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.582437 (+1568046us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b540026fa54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:26.582513 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:26.582587 (+    74us) raft_consensus.cc:1685] finished
0709 12:27:26.582608 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.582930 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35957,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":117,"spinlock_wait_cycles":732426112,"thread_start_us":88321,"threads_started":1,"wal-append.queue_time_us":321572,"wal-append.run_cpu_time_us":2725,"wal-append.run_wall_time_us":1197932}
I20260709 12:27:26.512611 12020 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:26.610934  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 49302.1us
I20260709 12:27:26.587137 11947 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:26.629542  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:26.674230  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.604725 11995 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:27:26.693377 12046 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:27:26.560292 11950 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:26.714636  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.691421 11947 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:26.765117  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 427844us
W20260709 12:27:26.816722  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.846488  6785 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:26.848446  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:26.611253  5873 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 12:27:26.855551  5873 pending_rounds.cc:85] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 9
I20260709 12:27:26.856001  5873 pending_rounds.cc:107] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:27:26.856396  5873 raft_consensus.cc:2889] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:26.865424  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1871) took 1616 ms. Trace:
I20260709 12:27:26.874657  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1471) took 1418 ms. Trace:
W20260709 12:27:26.932861  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.915232  5870 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:27:26.940658  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 60573.9us
I20260709 12:27:26.940881  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:26.927659 11799 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 12:27:26.872109 11587 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.937968 11556 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:27:26.943202  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1454) took 1223 ms. Trace:
I20260709 12:27:26.943701  6017 rpcz_store.cc:276] 0709 12:27:25.719757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.731676 (+ 11919us) service_pool.cc:224] Handling call
0709 12:27:26.942981 (+1211305us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400171bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:26.943132 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248244608}
I20260709 12:27:26.878659 12012 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:26.952455  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:26.956197  5873 rpcz_store.cc:276] 0709 12:27:25.456174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.706624 (+250450us) service_pool.cc:224] Handling call
0709 12:27:25.722536 (+ 15912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:25.887312 (+164776us) spinlock_profiling.cc:243] Waited 9.01 ms on lock 0x7b5400114020. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bf8a1d 00007f39d2bdf1a4 00007f39d2c61447 00007f39d2c79fe7 00007f39d2c7b7de 00007f39d2c85b49 00007f39d2c86416 00007f39d2c82d8f 00007f39d2c8283e 00007f39d7c47419
0709 12:27:26.449111 (+561799us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b58004970d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bdf0aa 00007f39d2c61447 00007f39d2c79fe7 00007f39d2c7b7de 00007f39d2c85b49 00007f39d2c86416 00007f39d2c82d8f 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4
0709 12:27:26.874247 (+425136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.874569 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093601408}
I20260709 12:27:26.880198  5946 rpcz_store.cc:276] 0709 12:27:25.248968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.249136 (+   168us) service_pool.cc:224] Handling call
0709 12:27:25.249437 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:25.249863 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:25.288782 (+ 38919us) log.cc:844] Serialized 14 byte log entry
0709 12:27:26.382578 (+1093796us) spinlock_profiling.cc:243] Waited 6.33 ms on lock 0x7b18002004a0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:27:26.864869 (+482291us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:26.864952 (+    83us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:26.865024 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.865128 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.865328 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16612,"prepare.queue_time_us":37541,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":69,"raft.queue_time_us":16750,"spinlock_wait_cycles":14566656,"thread_start_us":420267,"threads_started":1}
I20260709 12:27:26.959707 11940 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:26.960378 11895 raft_consensus.cc:2764] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:27:26.960901 11772 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:26.961143 12012 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:26.943280 11587 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:26.945444 11850 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.970129  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:26.970309  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:26.936542 11879 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.884707 11835 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.859341 11995 raft_consensus.cc:697] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:26.978698  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 489474us
I20260709 12:27:26.878539 11997 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:26.997102  6773 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.015s	sys 0.011s
W20260709 12:27:27.006683  5871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 1960) took 2122 ms (client timeout 1731 ms). Trace:
W20260709 12:27:27.018851  5871 rpcz_store.cc:269] 0709 12:27:24.875104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.876978 (+  1874us) service_pool.cc:224] Handling call
0709 12:27:26.997179 (+2120201us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b54002f94d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:26.997325 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1829862912}
I20260709 12:27:26.990226 12012 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:27.024597  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:27.024922  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:27.026286  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.013314 11895 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:27.002154 11850 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:27.006415 11879 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:27.018764  6773 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.016s	sys 0.011s
I20260709 12:27:26.977788 11940 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.016902  5838 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:27.072969  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 56142.1us
I20260709 12:27:27.074227  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:27.013207  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:27.012212  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:27.076386  5949 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 18: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:27.013942 11835 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:27.081951  5873 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
W20260709 12:27:27.044204 11992 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.010s	sys 0.007s
W20260709 12:27:27.015782  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.038177 11967 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:27.090579  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.095022 12053 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:27:27.102106 12051 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
W20260709 12:27:27.103034  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.105551 11685 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.522s	user 0.014s	sys 0.000s
I20260709 12:27:27.106014 11950 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:27.107414 12011 consensus_meta.cc:287] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.531s	user 0.011s	sys 0.004s
W20260709 12:27:27.068881 12027 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.006s	sys 0.020s
W20260709 12:27:27.110134 12027 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.006s	sys 0.020s
I20260709 12:27:27.112140 11977 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:27.112816 11977 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:27.114120  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.115384 11977 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:27.116518  5913 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:27.117134 11977 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:27.118067 11977 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:27.119612 11879 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:27.120471 11977 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:27.121836 11977 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:27.122339 11835 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:27.122589 11977 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:27.123226 11835 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:27.123864 11835 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:27.123982 11977 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:27.126543 11989 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:27.128084 12020 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:27.128558 12020 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:27.128715  5870 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:27.129796 11995 consensus_queue.cc:237] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:27.129911 12020 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:27.131716 12020 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:27.140890  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55542 (request call id 51) took 2185 ms. Trace:
I20260709 12:27:27.143455 11678 raft_consensus.cc:3055] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:27.144111 11678 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:27.147509  5913 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:27.149068 11854 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:27.156994 11587 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:27.020443 11997 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:27.158398 12040 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 1.282s	user 0.008s	sys 0.010s
I20260709 12:27:27.159752 11997 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:27.079979 11940 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:27.141575  6775 rpcz_store.cc:276] 0709 12:27:24.955254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.955475 (+   221us) service_pool.cc:224] Handling call
0709 12:27:27.140558 (+2185083us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54002c1ad4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c7d853 00007f39d7c4b60a 00007f39d11d17d4 00007f39d11d1762 00007f39d11d1691 00007f39d11d160c 00007f39d11d08c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:27.140751 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2858540288}
W20260709 12:27:27.182193  5948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.013s	sys 0.019s
W20260709 12:27:27.183125  5948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.014s	sys 0.019s
W20260709 12:27:27.089893 11992 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.011s	sys 0.007s
I20260709 12:27:27.144516 12020 consensus_queue.cc:1059] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:27:27.080071 11895 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:27.120568 11879 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:27.199165 12047 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:27:27.215602 11921 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:27.213119 11895 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:27.199931 11587 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:27.218264 11587 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:27.219342 11895 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:27.223351 11932 consensus_meta.cc:287] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.913s	user 0.013s	sys 0.000s
I20260709 12:27:27.226996 11587 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:27.229821 11933 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.644s	user 0.015s	sys 0.000s
I20260709 12:27:27.230113 11835 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:27.235945  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.267187  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.262768 11974 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.071s	sys 0.081s
I20260709 12:27:27.145360 11678 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:27.275753 11974 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.072s	sys 0.082s
W20260709 12:27:27.288725  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 662308us
I20260709 12:27:27.294708  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1971) took 1294 ms. Trace:
I20260709 12:27:27.295761  5866 rpcz_store.cc:276] 0709 12:27:26.000860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.146468 (+145608us) service_pool.cc:224] Handling call
0709 12:27:26.146780 (+   312us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:26.342357 (+195577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:27.294305 (+951948us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400027e54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:27.294637 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444162816}
I20260709 12:27:27.286836 11678 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:27.310400 11904 consensus_meta.cc:287] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.731s	user 0.017s	sys 0.000s
I20260709 12:27:27.311015 11904 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:27.298926 11967 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:27.314477 11967 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:27.316730 11967 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:27.316552 11992 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:27.313994  5869 raft_consensus.cc:2393] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:27:27.320048  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 466299us
I20260709 12:27:27.317763 11933 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:27.333581  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:27.334319  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:27.334661  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:27.334966  6024 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:27.335283  6023 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:27.328450  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:27.329200  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:27.336615  6017 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 17: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:27.333114  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:27.313439 11904 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:27.337323  6019 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:27.333775 11933 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:27.338647 11933 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:27.340075 11997 consensus_queue.cc:1059] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:27.339780  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:27.341277 11997 raft_consensus.cc:3055] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:27.341611 11997 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:27.343009 11904 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.343580 11904 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:27.343798 11997 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:27.345808 11904 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:27.346216  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:27.347431  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:27.348522  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:27.334343  5874 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:27.349205  6015 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:27.348862 11997 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:27.355540  6018 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:27.356446  6021 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:27.356081  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:27.362962 11902 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:27.365046  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:27.365530  5987 connection.cc:256] RPC call timeout handler was delayed by -1.1069800430000214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2242s for a response
I20260709 12:27:27.366017  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:27.366153  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:27.367538  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:27.368374  5940 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:27.372558  6024 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:27.363432 11902 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:27:27.375170  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 46257.6us
W20260709 12:27:27.377133  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 424744us
I20260709 12:27:27.378079  5840 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:27.339850 11933 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:27.380538  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.387428  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.388229  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:27.396225  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:27.382844  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:27.397166  6019 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:27:27.383932  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:27.380831  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:27.398972  6021 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:27.399582  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:27.418046  6608 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.001s	sys 0.037s
W20260709 12:27:27.396540  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.465255  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.433924 12037 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:27:27.436769 11921 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.425923 11827 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:27.375676 11902 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:27.445140  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1596) took 1265 ms. Trace:
I20260709 12:27:27.481971  5948 rpcz_store.cc:276] 0709 12:27:26.179420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.179616 (+   196us) service_pool.cc:224] Handling call
0709 12:27:26.180160 (+   544us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:26.191929 (+ 11769us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:26.191985 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:26.194589 (+  2604us) log.cc:844] Serialized 26 byte log entry
0709 12:27:26.484502 (+289913us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b18002962c0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:27:27.183773 (+699271us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:27.183853 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:27.184032 (+   179us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:27.444676 (+260644us) raft_consensus.cc:1685] finished
0709 12:27:27.444709 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:27.445046 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12284,"raft.queue_time_us":12148,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":137,"spinlock_wait_cycles":2775808,"thread_start_us":698940,"threads_started":1,"wal-append.queue_time_us":1181466}
W20260709 12:27:27.403103 11947 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.662s	user 0.010s	sys 0.004s
I20260709 12:27:27.376525 11949 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:27.429834  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.379254 11806 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:27:27.436033 11808 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:27.492270  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.465809  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 62562.1us
W20260709 12:27:27.464648  6608 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.002s	sys 0.037s
I20260709 12:27:27.487914 11949 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.504261  6015 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:27.476420  5867 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17. Ops: []
W20260709 12:27:27.479866  5865 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1474) took 2018 ms (client timeout 2242 ms). Trace:
I20260709 12:27:27.507869 11949 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:27.486703  6018 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:27.487079 11827 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.464773 11587 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.495043  5872 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:27:27.496112  6611 leader_election.cc:400] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17.
I20260709 12:27:27.526351  6611 leader_election.cc:403] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:27:27.493933 11556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.013s	sys 0.027s
W20260709 12:27:27.497007  5990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.026s	sys 0.015s
W20260709 12:27:27.529311  5990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.027s	sys 0.015s
I20260709 12:27:27.485318 11921 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:27.496784 11808 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:27.486830 11674 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:27.480980 11846 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:27.467248 11965 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:27.522171 11879 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:27.520290  5865 rpcz_store.cc:269] 0709 12:27:25.458994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.636354 (+177360us) service_pool.cc:224] Handling call
0709 12:27:27.477271 (+1840917us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b540018a954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:27.477409 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54329088}
W20260709 12:27:27.465823  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.491140 11797 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:27:27.489348 11997 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:27:27.488273 11947 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:27.584633 11947 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:27.468241  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:27.531052  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 547496us
W20260709 12:27:27.586326  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 118201us
I20260709 12:27:27.587191  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:27.587705  6609 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:27.588088  6018 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:27.481019 11895 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.589066 11895 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:27.590279  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.590129  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:27.590648  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:27.591426  6024 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:27.591728 11895 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:27.590304  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:27.592594  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:27.593119  6016 raft_consensus.cc:2410] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:27:27.593294  6015 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 18: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:27.590159  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:27.594177  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:27.594781  6019 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:27.598374  6018 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:27.599160 11808 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:27.562559 11674 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:27.601171 11808 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:27:27.602875 11997 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:27.603899 11997 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:27.607182 11921 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:27.528316 11587 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:27.610845 11587 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:27.614347 11674 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:27.616557  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.529197 11556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.014s	sys 0.027s
I20260709 12:27:27.520231 12012 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:27.564070  5915 connection.cc:256] RPC call timeout handler was delayed by -1.3766198260000237s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
W20260709 12:27:27.621587  5915 connection.cc:256] RPC call timeout handler was delayed by -1.3379198259999612s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
W20260709 12:27:27.520781 11986 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.006s	sys 0.015s
I20260709 12:27:27.562937 11846 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:27.529347 11827 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:27.576355 11806 raft_consensus.cc:3055] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:27.532553 11940 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:27.635993  5941 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
W20260709 12:27:27.627898 11986 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.006s	sys 0.016s
I20260709 12:27:27.634368 11806 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:27.688072  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1466) took 2232 ms. Trace:
I20260709 12:27:27.688706  5867 rpcz_store.cc:276] 0709 12:27:25.455099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.677555 (+222456us) service_pool.cc:224] Handling call
0709 12:27:25.677968 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:27.476164 (+1798196us) spinlock_profiling.cc:243] Waited 5.94 ms on lock 0x7b54001a5f20. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bfadc0 00007f39d2c851c0 00007f39d2c86231 00007f39d2c82d8f 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c
0709 12:27:27.519869 (+ 43705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:27.687665 (+167796us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54007350d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:27.687978 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377016960}
W20260709 12:27:27.743553  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.784291  5838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.025s	sys 0.028s
W20260709 12:27:27.785058  5838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.025s	sys 0.028s
W20260709 12:27:27.785856  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 711671us
I20260709 12:27:27.817507  5838 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:27.821899  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.692296 11854 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.016s	sys 0.022s
W20260709 12:27:27.685774 12011 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.017s	sys 0.021s
I20260709 12:27:27.843123 11879 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:27:27.877579 11806 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:27.877820 11797 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.889132 11806 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:27.843768 11965 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:27.888855 11797 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:27.690203 12027 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.012s	sys 0.025s
W20260709 12:27:27.904203  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.846977 11854 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.016s	sys 0.022s
W20260709 12:27:27.875428 12011 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.017s	sys 0.022s
W20260709 12:27:27.892906 11713 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.023s	sys 0.040s
W20260709 12:27:27.926178 11713 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.023s	sys 0.040s
I20260709 12:27:27.889498  5838 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:27.861989 11933 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:27.940948 11933 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:27.941354 11771 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:27.942430 11771 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:27.943748 12002 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:27.943555 11771 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:27.945461 11827 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:27.913235 12027 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.013s	sys 0.028s
I20260709 12:27:27.843927 12004 consensus_queue.cc:1059] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 12:27:27.911501 11854 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:27.947899  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 58470.3us
I20260709 12:27:27.929340 11902 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:27.961109 11933 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:27.950881 12037 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:27.950623 12070 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:27.972885  5987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.031s	sys 0.020s
W20260709 12:27:27.973433  5987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.031s	sys 0.020s
I20260709 12:27:27.975899  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1239) took 1006 ms. Trace:
I20260709 12:27:27.976109  5873 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:27:27.964141 11940 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:27.976536  5869 rpcz_store.cc:276] 0709 12:27:26.969822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.969985 (+   163us) service_pool.cc:224] Handling call
0709 12:27:27.975577 (+1005592us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b5400026a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:27.975817 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":740058368}
I20260709 12:27:27.962245 12012 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:27.986733  5866 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.012s	sys 0.030s
W20260709 12:27:27.992295  5866 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.012s	sys 0.031s
W20260709 12:27:27.992578  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.968627 11977 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.023s	sys 0.018s
W20260709 12:27:27.959538 12020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.025s	sys 0.029s
W20260709 12:27:28.003321 12020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.025s	sys 0.030s
I20260709 12:27:28.002871  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:28.004613  5840 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.011s	sys 0.010s
I20260709 12:27:27.970131 11933 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:28.007555  5942 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.005s	sys 0.014s
W20260709 12:27:27.986279 11992 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.010s	sys 0.032s
W20260709 12:27:27.993549 11974 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.016s	sys 0.025s
W20260709 12:27:28.004333 11977 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.023s	sys 0.018s
I20260709 12:27:27.981858 12027 raft_consensus.cc:3055] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:27:28.013553  5840 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.011s	sys 0.010s
W20260709 12:27:28.057801  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 605596us
I20260709 12:27:27.970429 12070 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:28.061540  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:28.063877  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:28.066773  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:28.001838 11947 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:28.018240  6608 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.011s	sys 0.024s
W20260709 12:27:28.071143  6608 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.012s	sys 0.024s
W20260709 12:27:28.044008 11974 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.016s	sys 0.029s
I20260709 12:27:28.010748 12071 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:28.030942  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.077142  6773 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:28.077826  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:28.015436  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:28.079356  5871 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:28.079741  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:28.080430  6756 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:28.079242  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:28.082283  5874 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:28.083581  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:28.084276  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.084985  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:28.085254  6757 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:28.046710 11587 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:28.017988 11992 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.011s	sys 0.032s
I20260709 12:27:28.068980  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:28.006121 11904 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:28.105772  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.106678  6762 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:28.070629 11967 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:28.113252  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:28.088802  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 73442.7us
I20260709 12:27:28.074721  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:28.017365 11933 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:28.016959 11921 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:28.073065 11977 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.088840  6766 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:28.118950  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:28.122668  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:28.118958  5865 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 19: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:28.120662  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:28.013271 12012 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:28.092700  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.089792  5868 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:28.015293 11854 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:28.070398 11995 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.009s	sys 0.019s
I20260709 12:27:28.094666  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:28.073024 11685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.002s	sys 0.023s
I20260709 12:27:28.095677  5874 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:28.059062 12027 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:28.103106  6611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.018s	sys 0.029s
I20260709 12:27:28.141736  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.062952  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:28.109543  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 1989) took 1098 ms. Trace:
I20260709 12:27:28.143544  5873 rpcz_store.cc:276] 0709 12:27:27.011213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.082872 (+ 71659us) service_pool.cc:224] Handling call
0709 12:27:27.975772 (+892900us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:27.976659 (+   887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.109336 (+132677us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b5400026a50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:28.109488 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917177088}
I20260709 12:27:28.070283  5869 raft_consensus.cc:2368] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18.
I20260709 12:27:28.114992  5872 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:28.018000 12067 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.088078 12065 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:28.014887  5942 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.006s	sys 0.015s
I20260709 12:27:28.102823 11587 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:28.098250 12030 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:28.093618 12071 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:28.078531  6775 raft_consensus.cc:2376] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15. Re-sending same reply.
I20260709 12:27:28.124405  6778 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:28.116406  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:28.161126  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 44776.9us
I20260709 12:27:28.094477  6757 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:28.162508  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:28.162676  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:28.164013  6766 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:28.105506 11997 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.129333 11895 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:28.077994 11947 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:28.133289  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.109369 11797 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:28.114015 11965 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:28.137912  6785 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
W20260709 12:27:28.138183  5913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.010s	sys 0.019s
W20260709 12:27:28.170368  5913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.011s	sys 0.019s
W20260709 12:27:28.170718  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 553194us
I20260709 12:27:28.170773  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:28.138830  6609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.014s	sys 0.021s
I20260709 12:27:28.127112  6756 raft_consensus.cc:2410] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:27:28.140934 11904 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:28.180855  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:28.127903 12068 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
W20260709 12:27:28.142071  6611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.018s	sys 0.029s
I20260709 12:27:28.103678  5870 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:28.105093 11940 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.117772 11967 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:28.133946 11995 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.010s	sys 0.021s
I20260709 12:27:28.104254 11984 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:28.139209 11685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.002s	sys 0.024s
W20260709 12:27:28.164372  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.012s	sys 0.045s
I20260709 12:27:28.166570  6757 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:27:28.167901  6768 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 18: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:28.168152  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:28.138669 11921 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
W20260709 12:27:28.171823  6609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.014s	sys 0.022s
W20260709 12:27:28.295406  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 707698us
I20260709 12:27:28.296388  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:28.138432  6017 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:28.139551  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.183012  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:28.197176  5944 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24. Ops: []
W20260709 12:27:28.236074  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:28.248553  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 76959us
I20260709 12:27:28.248490  5946 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 8. (term mismatch)
I20260709 12:27:28.299746  5946 pending_rounds.cc:85] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 7
I20260709 12:27:28.300107  5946 pending_rounds.cc:107] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 12:27:28.300408  5946 raft_consensus.cc:2889] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:28.300819  5946 pending_rounds.cc:107] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 12:27:28.301113  5946 raft_consensus.cc:2889] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:28.302145  6778 raft_consensus.cc:2410] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 12:27:28.250883  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 754894us
I20260709 12:27:28.133868 12012 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:28.249850 11904 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:28.156891 12067 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:28.174340 11674 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.330819  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.156870 11902 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:28.166779 11894 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.272712  6785 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:28.346388  6785 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:28.346592  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 50324.8us
W20260709 12:27:28.249392 11986 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.008s	sys 0.068s
I20260709 12:27:28.284822  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:28.291600  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.013s	sys 0.045s
W20260709 12:27:28.360481  5915 connection.cc:256] RPC call timeout handler was delayed by -1.0292198259999168s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19630000000000003s for a response
I20260709 12:27:28.361897  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:28.167332 11895 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:28.169255 11850 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:28.250957  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:28.394862  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.395403  6756 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 18: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:28.313743  6778 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:28.314061  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:28.248560 12004 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:28.317263  6019 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:28.130844 11977 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:28.275326 12073 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 12:27:28.335031  6016 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:28.262996 11989 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.129s
I20260709 12:27:28.345252  6757 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 17: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:28.368343  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:28.156708 12065 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:28.347440  6785 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:28.271083 11940 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:28.404332 11932 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:27:28.404857  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.404938 11895 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:28.405534 12088 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.407194 12004 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.169468 11997 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:28.515081  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.416527  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1161) took 1049 ms. Trace:
I20260709 12:27:28.417832  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:28.516649  6023 raft_consensus.cc:2410] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
W20260709 12:27:28.423902  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 261256us
I20260709 12:27:28.248886 11827 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.247943 11965 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:28.359375 11986 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.010s	sys 0.068s
I20260709 12:27:28.440070  6785 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:28.388047 11894 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:28.437155  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:28.441648  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 73460.1us
I20260709 12:27:28.447496  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:28.448335  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 147184us
I20260709 12:27:28.369263 11674 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:28.292676 11984 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:28.407758  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:28.474298 11995 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:28.476243  5949 raft_consensus.cc:3055] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:28.544399  5949 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:28.447098 12030 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:28.479482 11835 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:28.485525 11716 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:28.501147 11772 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:28.518011  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:28.519767  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:28.420642 12090 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:28.523123  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.521112  6775 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:28.525872  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.545473  5949 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:28.530929 11895 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:28.527343  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:28.576476  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 49233.2us
I20260709 12:27:28.280285 12027 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:28.439720 11685 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:28.583310 11685 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:28.584241  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.520457 12004 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:28.550540  6773 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:28.587427 11997 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:28.559433  6785 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 12:27:28.463090 12093 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:28.431216 12074 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 12:27:28.563934  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.399056 12083 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:28.595110 12074 raft_consensus.cc:3055] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:27:28.545943 12073 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:27:28.595589 12074 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:28.596684 12074 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:28.525326 11827 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:28.598492 12074 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:28.603245  6785 pending_rounds.cc:85] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 10
I20260709 12:27:28.529210 11797 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.610770  6785 pending_rounds.cc:107] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:27:28.610924 11797 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:28.611550  6785 raft_consensus.cc:2889] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:28.613407 11797 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:28.604929 12070 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:28.607929 12004 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:28.608350 11947 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:28.609798 12094 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:27:28.568755 11716 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:28.618456 11716 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:28.621641 12004 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:28.627265 12067 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:28.631174 11967 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:28.582116 12090 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:28.566816 11835 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:28.635494 12012 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:28.639814 11771 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:28.512672 11932 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:28.520342 12088 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:28.536998 12077 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.687521  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:28.657269  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.694674  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:28.658592  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 80728.2us
I20260709 12:27:28.695080  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.695611 12088 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:28.566077 12081 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 12:27:28.564975 11933 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:28.641417 11894 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:28.685834 11940 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:28.704783 12037 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.739s	user 0.012s	sys 0.000s
I20260709 12:27:28.655759  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.718768  6785 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:28.696939  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:28.655094 11965 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:28.656271  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.720222  6756 raft_consensus.cc:2393] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:28.681581  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:28.697587 12064 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:28.648172 12027 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:28.698586 11932 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:28.643210 11977 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:28.643981 11587 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:28.578709 12071 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:28.700156 12002 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.756s	user 0.008s	sys 0.006s
I20260709 12:27:28.575197 12089 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:28.653525 11827 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:28.661366 11674 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:28.665710 12065 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:28.666952 12079 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.141s	user 0.000s	sys 0.004s
I20260709 12:27:28.637212 11835 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:28.710471 11846 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.072s	user 0.012s	sys 0.003s
I20260709 12:27:28.670328 12030 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:28.668449 11806 consensus_meta.cc:287] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.779s	user 0.013s	sys 0.000s
W20260709 12:27:28.711684 11949 consensus_meta.cc:287] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.182s	user 0.010s	sys 0.006s
I20260709 12:27:28.700392 12077 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:28.651187 11808 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.029s	user 0.013s	sys 0.003s
I20260709 12:27:28.753738 11808 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17.
I20260709 12:27:28.717077 11685 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:28.659667 11984 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:28.738288 12090 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.750453 11949 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:28.756531 12090 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:28.745922 11556 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.758249 11556 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:28.759001 11949 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:28.750427 11846 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:28.752831 12077 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:28.736933 12037 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:28.738052 11940 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:28.769687 12090 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:28.389585  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 12:27:28.464669  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 190347us
W20260709 12:27:28.820694  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:28.827518  5839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.030s	sys 0.028s
W20260709 12:27:28.828629  5839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.031s	sys 0.028s
I20260709 12:27:28.828765  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:28.831310  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:28.833415  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.515991  5942 rpcz_store.cc:276] 0709 12:27:27.367495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.367712 (+   217us) service_pool.cc:224] Handling call
0709 12:27:27.382307 (+ 14595us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:27.382573 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:28.152192 (+769619us) log.cc:844] Serialized 14 byte log entry
0709 12:27:28.415987 (+263795us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:28.416073 (+    86us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:28.416141 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.416240 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:28.416435 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21630,"prepare.queue_time_us":727234,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":2160,"raft.queue_time_us":63931,"thread_start_us":1007567,"threads_started":2}
I20260709 12:27:28.570510  5949 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:28.837971  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 319812us
I20260709 12:27:28.840638  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:28.856042  5944 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:27:28.860776  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:28.862470  5947 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:28.861012  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:28.870446  5942 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
W20260709 12:27:28.887181  6021 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.012s	sys 0.039s
I20260709 12:27:28.868831  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:28.887866  6021 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.013s	sys 0.040s
W20260709 12:27:28.890017  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.895345  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:28.720506  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:28.898943  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 178359us
I20260709 12:27:28.899582  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:28.875646  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.875929  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:28.900738  5872 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:28.565579  6016 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:28.899785  5873 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:27:28.904531  6018 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:28.906900  5865 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:28.669956  6757 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:28.664474  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:28.696244  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.942920  6766 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:28.942967  6773 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:28.944243  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.944609  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.945835  6773 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:28.874109  5987 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:28.923067  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.927587 11587 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:28.928608  6762 raft_consensus.cc:2410] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:27:28.929773  6778 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:28.929358 11895 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:28.930354  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.930912 11835 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:28.931496 12002 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:28.931286  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:28.932410 12037 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:28.933750 12071 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:28.938817  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:28.974017  5840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.006s	sys 0.004s
I20260709 12:27:28.971922  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:28.976476  5840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.007s	sys 0.004s
W20260709 12:27:28.982340  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 80972.6us
W20260709 12:27:28.998865  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 125020us
I20260709 12:27:29.001479 12110 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:29.004793 11556 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:29.005780 11835 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:29.009953 12002 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:29.012147 11846 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:29.014427 12108 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:29.001840 11587 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:29.016844  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:29.017251  6793 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:29.020654  5872 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:29.025936  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.027498  6018 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
W20260709 12:27:29.028424 12003 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:27:29.030001  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:29.031749  6785 raft_consensus.cc:2410] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:27:29.033411  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:29.033737 12064 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.034135  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.034849 12064 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:29.037397  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 869231us
I20260709 12:27:29.039731  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:29.034523  6756 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 17: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:29.043668  6756 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:29.046372  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:29.047683 12089 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:29.049114  5947 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
W20260709 12:27:29.041777  6020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.007s	sys 0.007s
W20260709 12:27:29.050153  6020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.007s	sys 0.007s
W20260709 12:27:29.052099  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.000s	sys 0.025s
W20260709 12:27:29.061699  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.000s	sys 0.026s
I20260709 12:27:29.052577  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:29.057977  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:29.060964  6766 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:29.035811  6785 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:29.020640 12090 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:27:29.052419  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.017402 12108 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:29.035126  6016 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:29.053138 12092 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:29.076345  5870 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:29.020635 11967 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:29.093356  5872 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:29.095021 11950 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:29.094599 12108 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:29.097946 12071 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:29.012312 12002 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:29.096777  5990 leader_election.cc:336] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 2.191s (SENT)
W20260709 12:27:29.097142  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 75653.6us
I20260709 12:27:29.098649 12071 raft_consensus.cc:697] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:29.074184 12090 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.105583  5987 leader_election.cc:400] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
I20260709 12:27:29.105991  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.106031  5987 leader_election.cc:403] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 12:27:29.108954  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:29.114234  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 52387.7us
I20260709 12:27:29.107911  5990 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:29.090095 11967 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:29.119849  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.131027 11879 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:29.131393  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1490) took 1072 ms. Trace:
I20260709 12:27:29.133978 12089 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:29.134802  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:29.134889  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 95305.4us
I20260709 12:27:29.135560  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.136106  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:29.136449  6018 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:29.138100 12117 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:29.138878 12117 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:29.135071 11967 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:29.143548  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:29.144826  5944 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:29.144949 12112 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:29.132215 11879 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:29.152854  5913 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:29.156569 12071 consensus_queue.cc:237] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:29.157508 12070 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
I20260709 12:27:29.159003  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:29.155100  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 49808.6us
W20260709 12:27:29.131560 11904 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.740s	user 0.012s	sys 0.009s
I20260709 12:27:29.165758 11904 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:29.166474 11587 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:29.111096 11835 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:29.168969 11904 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:29.108546 12090 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:29.078222 12092 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:29.175628  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.185267 11879 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:29.186267 11854 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.186908  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 1395) took 1061 ms. Trace:
I20260709 12:27:29.187526 11854 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:29.189704  5837 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.002s	sys 0.022s
W20260709 12:27:29.190270  5837 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.002s	sys 0.023s
I20260709 12:27:29.191033 11854 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:29.193063  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:29.187361  6762 rpcz_store.cc:276] 0709 12:27:28.127758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.127898 (+   140us) service_pool.cc:224] Handling call
0709 12:27:29.186561 (+1058663us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b54005758d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:29.186787 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1805142784}
I20260709 12:27:29.202065  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:29.203600  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 77313.9us
I20260709 12:27:29.140738 12117 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:29.174461 12112 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:29.213958 12120 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:29.215093 11894 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.216745 12064 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:29.216603  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:29.217828  5944 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:29.218222  6020 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:29.218641  6611 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:29.131111 12105 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:29.227298  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.202075 12030 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:29.182103 12118 consensus_queue.cc:1059] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:27:29.132202  5866 rpcz_store.cc:276] 0709 12:27:28.058931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.131499 (+ 72568us) service_pool.cc:224] Handling call
0709 12:27:28.131835 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:28.512860 (+381025us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:28.513279 (+   419us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:28.514441 (+  1162us) log.cc:844] Serialized 26 byte log entry
0709 12:27:29.111616 (+597175us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:29.120662 (+  9046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:29.130788 (+ 10126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:29.130861 (+    73us) raft_consensus.cc:1685] finished
0709 12:27:29.130877 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:29.131151 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39502,"raft.queue_time_us":289071,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":7541,"thread_start_us":452462,"threads_started":2,"wal-append.queue_time_us":484727}
I20260709 12:27:29.228701  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:29.245900  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:29.246474  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:29.247920 11904 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:29.248418 11904 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.248806 11904 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:29.250391 11895 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:29.251006 11895 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.251372 11895 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:29.258111  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:29.259630 11965 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:29.260254 11965 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:29.262300  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 165888us
I20260709 12:27:29.227789 12105 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.265430  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.265961 12105 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:29.266861  6021 raft_consensus.cc:2410] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:27:29.270717 11904 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:29.273459  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:29.275372  5697 catalog_manager.cc:5697] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 6 to 10, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 10 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:27:29.223479 12120 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:29.288728  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 70402.9us
W20260709 12:27:29.295743  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 79271.3us
I20260709 12:27:29.298990  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.240886 11879 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.299913  6020 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:29.303932  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:29.273582 11904 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:29.289384 12120 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:29.303648 12064 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:27:29.314153  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:29.315145 12002 raft_consensus.cc:2764] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:27:29.315534  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:29.316748  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:29.313055  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:29.288395 11894 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:29.319022 11556 consensus_queue.cc:1059] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:29.319519  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:29.319864 11894 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:29.320151 11556 raft_consensus.cc:3055] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:29.320592  5947 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 21: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:29.320669 11556 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:29.317744 11977 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:29.322237 11977 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.322775 11977 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:29.323097 12111 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:29.323568 12111 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.322314 11992 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:29.323983 12111 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:29.288404 12118 consensus_queue.cc:1243] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:29.300323 11846 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.326164 11846 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:29.328163 12112 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:29.328975 11846 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:29.330034 12112 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:29.332226  6611 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:29.332741  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.333370 11950 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:29.333690  6021 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:29.334187 11950 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:29.335601  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.336830  6016 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 17: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:29.337723 11879 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:29.338336 12103 raft_consensus.cc:3055] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:29.338816 12103 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:29.339110  6611 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:29.339212  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:29.340171  5940 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:29.339924 12103 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:29.340598 11879 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:29.341310  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:29.302549 12089 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:29.317953  5948 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:29.337241 11950 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:29.311553 12003 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:27:29.350018  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.351089  6018 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:29.351295 12003 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:29.351966 12003 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:29.353073 11949 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:29.353796 11949 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.354867  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.354197  5949 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
W20260709 12:27:29.357450  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 50220.9us
I20260709 12:27:29.325444  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:29.316975 12030 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.361050  5870 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:29.341403 12110 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:29.365454  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:29.354292 11949 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:29.366406  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:29.388664  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.342641 12103 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:29.377415  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:29.377589  5872 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:29.348466 12089 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.383409  5840 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:29.383872  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:29.385771 11940 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:29.375831  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.261479 11965 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:29.394861 11904 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.395349  6017 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 17: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:29.397461  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:29.400856  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.402331  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:29.400911  5947 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:29.405982  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:29.407078  5949 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:27:29.409122 11556 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:29.411044 11556 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:29.413954  5873 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:29.422021  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:29.375432 12110 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.391409 12089 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:29.430716  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 1.06876e+06us
I20260709 12:27:29.438473 12077 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:29.433949 11879 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:29.439375 11846 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:29.440151 11846 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:29.441071  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:29.439373  6016 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:29.393782 12030 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:29.440178 11879 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:29.446936 11879 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:29.447913 12030 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:29.448616 12030 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:29.448949 12030 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:29.449292 12030 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:29.441280 11846 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:29.450938 11846 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:29.434171  6018 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 13.
I20260709 12:27:29.453960  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:29.428303 12110 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:29.470377  5866 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:29.393363 12118 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:27:29.467451  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:29.472800  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 147573us
I20260709 12:27:29.353371 12003 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:29.432868 11950 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:29.474300  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:29.482112  5944 raft_consensus.cc:2410] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:27:29.433178 12092 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:29.481678  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.495555  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:29.432498 11904 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:29.498289  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:29.498905  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:29.502472  6609 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:29.502837 11965 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:29.489296 11854 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:29.506124  5870 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:29.508144  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:29.508538  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:29.546520  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 163310us
I20260709 12:27:29.511610  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:29.512200 12105 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:29.513198  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:29.514096  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:29.502596  5697 catalog_manager.cc:5697] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:29.519286  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:29.517127 12030 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:29.527570 11949 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:29.529649  6773 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:29.524771 12110 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:29.510903  5945 pending_rounds.cc:85] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 9
I20260709 12:27:29.546931  5871 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:29.480126 12003 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:29.472293 12077 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:27:29.565313  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:29.567476  5869 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:29.563839  5873 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:29.571781  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:29.572345  6016 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:29.568290  5872 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:29.563812  5945 pending_rounds.cc:107] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:27:29.553380 11854 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.597393  5945 raft_consensus.cc:2889] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:29.603164  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:29.604166  5870 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 13.
I20260709 12:27:29.605357 12110 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:29.606916 12110 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:29.610056  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 111318us
I20260709 12:27:29.611186  5869 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: [20.12-20.12]
W20260709 12:27:29.613019  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 93866.5us
I20260709 12:27:29.614634  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:29.614970 11904 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:29.615556  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:29.616427 12128 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:27:29.617318  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:29.618257  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:29.570449 11967 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:29.624111 11967 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:29.625890 12077 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:29.626566 12077 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:29.603432 11949 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:29.627399  5874 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:29.631259  6757 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:29.629787 11854 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:29.631986  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:29.657712  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:29.627888 12112 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:27:29.650669  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 148329us
W20260709 12:27:29.656343  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 43753.3us
I20260709 12:27:29.657966  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:29.666517  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51981.7us
I20260709 12:27:29.633322  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:29.664811  6762 raft_consensus.cc:2376] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17. Re-sending same reply.
I20260709 12:27:29.659122  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:29.666750  5942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.008s	sys 0.037s
W20260709 12:27:29.675104  5942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.008s	sys 0.037s
I20260709 12:27:29.672951  6768 raft_consensus.cc:2376] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17. Re-sending same reply.
I20260709 12:27:29.680433  6778 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:29.650816 11967 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:29.691638  5987 leader_election.cc:400] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18.
I20260709 12:27:29.693820  5987 leader_election.cc:403] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:27:29.696702 12027 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:29.697474 12027 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:29.700114 12027 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:29.702359  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:29.703395  5941 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 19: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:27:29.704452  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.710256  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:29.710803  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:29.711500  5867 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:29.712359  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:29.712901 11950 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:29.713544 11950 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.713980 11950 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:29.715865  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:29.723688 11950 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:29.689332 12077 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:29.743563  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.744979  6020 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:29.743563  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.746153  6021 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:29.748173 12105 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:29.750162  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.751741  6018 raft_consensus.cc:2410] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:27:29.754892  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:29.755861  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:29.750666 11950 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:29.766806 12030 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:29.768208  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:29.770210  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:29.773311 12090 consensus_meta.cc:287] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.549s	user 0.010s	sys 0.002s
I20260709 12:27:29.774052 12090 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:29.769220  6017 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:29.701089 12120 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:29.786284  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:29.787313  6016 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:29.789932  5871 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:29.777149 12090 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:29.741675 12110 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:29.783427  6609 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:29.804677  5944 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:27:29.819386  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 82758.8us
I20260709 12:27:29.793533 11835 raft_consensus.cc:3055] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:27:29.826079  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:29.822383  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 483449us
I20260709 12:27:29.803332 12110 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.851233  6609 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:29.849992  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 363058us
I20260709 12:27:29.876796  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:29.799356 11992 raft_consensus.cc:697] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:29.826920 12108 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.015s	sys 0.030s
W20260709 12:27:29.884537  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 62014.9us
W20260709 12:27:29.852505  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 168777us
W20260709 12:27:29.892401  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:29.905465  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.803990 12030 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18.
I20260709 12:27:29.841295 11835 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:29.928012 11835 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:29.917644  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:29.930114  6778 raft_consensus.cc:2368] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
W20260709 12:27:29.930528  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 87420.9us
I20260709 12:27:29.932525  6611 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:29.938405  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:29.939504  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:29.941350  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:29.952593 12092 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:29.853154 11950 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:29.954665 11950 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:29.955246 11950 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:29.939524  6768 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:29.881440 12110 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:29.955127  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:29.961884  5696 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40818 (request call id 150) took 36545 ms (client timeout 15000 ms). Trace:
W20260709 12:27:29.973066 12089 consensus_meta.cc:287] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.001s
I20260709 12:27:29.982182  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:29.962634  5696 rpcz_store.cc:269] 0709 12:26:53.416269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.439218 (+ 22949us) service_pool.cc:224] Handling call
0709 12:27:29.961511 (+36522293us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:21.038182 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:21.039135 (+   953us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:21.039210 (+    75us) write_op.cc:435] Acquired schema lock
0709 12:27:21.039238 (+    28us) tablet.cc:662] Decoding operations
0709 12:27:21.040804 (+  1566us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:21.040955 (+   151us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:21.041005 (+    50us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:27:21.042764 (+  1759us) tablet.cc:701] Row locks acquired
0709 12:27:21.042845 (+    81us) write_op.cc:260] PREPARE: finished
0709 12:27:21.043111 (+   266us) write_op.cc:270] Start()
0709 12:27:21.043342 (+   231us) write_op.cc:276] Timestamp: P: 1783600041043016 usec, L: 0
0709 12:27:21.043398 (+    56us) op_driver.cc:348] REPLICATION: starting
0709 12:27:21.044502 (+  1104us) log.cc:844] Serialized 4737 byte log entry
0709 12:27:21.240880 (+196378us) op_driver.cc:464] REPLICATION: finished
0709 12:27:21.766829 (+525949us) spinlock_profiling.cc:243] Waited 9.64 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39cfb790ac 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d66b9ef0
0709 12:27:22.220357 (+453528us) write_op.cc:301] APPLY: starting
0709 12:27:22.220503 (+   146us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:23.617081 (+1396578us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:23.631086 (+ 14005us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:24.984755 (+1353669us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:25.140489 (+155734us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 12:27:25.154070 (+ 13581us) write_op.cc:312] APPLY: finished
0709 12:27:25.329052 (+174982us) log.cc:844] Serialized 112 byte log entry
0709 12:27:25.457076 (+128024us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:25.457250 (+   174us) write_op.cc:454] Released schema lock
0709 12:27:25.457332 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:25.457867 (+   535us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:25.457900 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":682840,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":978999,"apply.run_cpu_time_us":60862,"apply.run_wall_time_us":3237542,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54335,"num_ops":12,"prepare.queue_time_us":86499,"prepare.run_cpu_time_us":15212,"prepare.run_wall_time_us":47706,"raft.queue_time_us":47726,"raft.run_cpu_time_us":22191,"raft.run_wall_time_us":814183,"replication_time_us":197225,"spinlock_wait_cycles":22169472,"thread_start_us":1052583,"threads_started":5,"wal-append.queue_time_us":348393,"wal-append.run_cpu_time_us":37356,"wal-append.run_wall_time_us":2327455}]]}
W20260709 12:27:29.975672  5990 leader_election.cc:336] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 2.242s (SENT)
W20260709 12:27:29.902936 12108 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.016s	sys 0.030s
I20260709 12:27:29.981564  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1632) took 1087 ms. Trace:
I20260709 12:27:29.818687 11977 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:29.915246 12090 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:29.998572  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:29.986271 11949 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:30.012663  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 80247.8us
W20260709 12:27:30.014286  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 110336us
I20260709 12:27:29.992003  6756 rpcz_store.cc:276] 0709 12:27:28.894367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.054695 (+160328us) service_pool.cc:224] Handling call
0709 12:27:29.112679 (+ 57984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:29.170398 (+ 57719us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:29.171482 (+  1084us) log.cc:844] Serialized 14 byte log entry
0709 12:27:29.981180 (+809698us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:29.981255 (+    75us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:29.981299 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:29.981360 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:29.981498 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5762,"prepare.queue_time_us":379,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":47273,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":124,"thread_start_us":787217,"threads_started":2,"wal-append.queue_time_us":653295}
I20260709 12:27:29.997979 12092 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:30.013782 12103 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.611s	user 0.011s	sys 0.004s
W20260709 12:27:30.015690 11895 consensus_meta.cc:287] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.764s	user 0.014s	sys 0.004s
I20260709 12:27:30.036893  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:30.004591 12090 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.992993 12089 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:30.046470  6016 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:30.044629  6611 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:30.055433  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 281435us
W20260709 12:27:30.063845  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 70945.3us
W20260709 12:27:30.066915  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 351106us
I20260709 12:27:30.049412 12090 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:30.071039  6775 raft_consensus.cc:2368] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18.
I20260709 12:27:30.042906 12108 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:30.055245 12064 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.012s	sys 0.039s
W20260709 12:27:30.078514  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 52641us
I20260709 12:27:30.092942  6773 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:30.046257 11895 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:30.083791 12064 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.013s	sys 0.039s
I20260709 12:27:30.105867  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:30.107165  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.113298  5866 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:27:30.109313  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 77669.5us
I20260709 12:27:30.113324  5871 raft_consensus.cc:2410] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:27:30.082010 12108 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.053659 12092 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:30.133308 12070 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:30.121930 12108 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:30.143644 11895 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:30.090719 12120 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:30.129072  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:30.173175 12137 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:30.126734 12111 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.802s	user 0.015s	sys 0.004s
W20260709 12:27:30.149374 11965 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.001s
W20260709 12:27:30.174553  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 64520.4us
I20260709 12:27:30.177218  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:30.179330  5866 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:27:30.185715 12138 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:27:30.188376  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 67999.6us
I20260709 12:27:30.191700 11835 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:30.198163  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:30.199105  5872 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
W20260709 12:27:30.200037  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.200582  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:30.204460 12142 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:30.205168 12142 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.205803 12136 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:30.206821 12136 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:30.207306 12136 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.207737 12136 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:30.209174 12133 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 12:27:30.211010 12129 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.102s	user 0.000s	sys 0.002s
I20260709 12:27:30.212767  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2008) took 1228 ms. Trace:
I20260709 12:27:30.126528 12002 consensus_queue.cc:1059] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 12:27:30.213204  5865 rpcz_store.cc:276] 0709 12:27:28.984514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.984731 (+   217us) service_pool.cc:224] Handling call
0709 12:27:29.024418 (+ 39687us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:29.107386 (+ 82968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:29.107475 (+    89us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:29.146636 (+ 39161us) log.cc:844] Serialized 26 byte log entry
0709 12:27:29.631904 (+485268us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:29.631967 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:29.632046 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:30.212432 (+580386us) raft_consensus.cc:1685] finished
0709 12:27:30.212460 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:30.212650 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46671,"raft.queue_time_us":49685,"raft.run_cpu_time_us":475,"raft.run_wall_time_us":88788,"thread_start_us":484610,"threads_started":1,"wal-append.queue_time_us":706040}
I20260709 12:27:30.197125  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:30.218159  5870 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:27:30.220463 11940 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.004s	sys 0.037s
I20260709 12:27:30.067373 11992 consensus_queue.cc:237] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:30.228756  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.246752  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:30.247714  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:30.248885  5944 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 14. Current term is 16. Ops: []
I20260709 12:27:30.249286  5941 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:30.250068  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:30.250545  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:30.251711  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:30.252058  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:30.253257  5949 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 16: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:30.250185  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:30.225602 11940 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.004s	sys 0.037s
I20260709 12:27:30.250279  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:30.255306 12030 raft_consensus.cc:3055] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:30.255772 12030 raft_consensus.cc:740] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:30.205617 12142 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:30.193348  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:30.265542  5872 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:27:30.228905  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 308053us
I20260709 12:27:30.279836  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:30.282708  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 89506us
W20260709 12:27:30.281232  5868 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.020s	sys 0.040s
I20260709 12:27:30.285761 12105 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:30.286298 12105 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.286680 12105 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:30.287286  5865 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:27:30.285733  5868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.020s	sys 0.040s
I20260709 12:27:30.216491 12111 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:30.181737 12137 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:30.301390 12089 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:30.277994 11977 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:30.309614 12002 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:30.310156  5870 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:30.240859 12120 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:27:30.316953  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:30.318693  5874 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:30.284696 12003 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.708s	user 0.015s	sys 0.001s
I20260709 12:27:30.330281  5840 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:30.331986 11904 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:30.334766  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:30.339488  5944 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:27:30.341919  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 237570us
I20260709 12:27:30.344260  5840 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:30.375799  5840 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:30.386401  5945 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:27:30.388952  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 274510us
I20260709 12:27:30.391633  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:30.400043  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:30.401924  5872 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
W20260709 12:27:30.393682 11854 consensus_meta.cc:287] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.702s	user 0.017s	sys 0.000s
I20260709 12:27:30.388072 11835 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:30.385195 11977 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:30.411651  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:30.406046  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:30.406131  5840 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:30.347604 12120 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:27:30.408674  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:30.411391  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:30.406734  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:30.414615  5866 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:30.422353  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.422705  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:30.424018  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:30.409893 11965 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:30.412561 11835 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:30.416843 12002 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:30.430459 11965 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:30.419384 11895 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:30.458999 12153 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:27:30.460436  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:30.463330  5840 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 2.097s (SENT) [suppressed 27 similar messages]
I20260709 12:27:30.412308 12110 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:30.409879 11854 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:30.480536  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:30.504176  5865 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:30.452469 11835 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:30.455889 12077 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:27:30.487725  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 60392.4us
I20260709 12:27:30.494362 12154 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:30.496021  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:30.496938 12002 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:30.498636 11949 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:30.452127 11950 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:30.554032  5942 raft_consensus.cc:2368] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
I20260709 12:27:30.554440 11950 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:30.519258  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:30.524572  5866 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:30.482215 11965 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:30.536240  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:30.475816 11895 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:30.537458  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 127840us
I20260709 12:27:30.525467 12117 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.512800 12089 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:30.551249 12110 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:30.552363 12151 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:27:30.552070 12071 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:30.541544  5873 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:30.559077  5874 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:30.503023 12158 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:30.562656  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 107352us
I20260709 12:27:30.562693  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:30.564095  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:30.566313  6609 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:30.516247 12077 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.525199 12153 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:30.591960  6015 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:30.593012  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:30.599220  6018 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:30.566874 11895 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:30.561348 11879 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:30.560997 12154 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:30.612607  5941 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:30.613327  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:30.598861 12027 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:30.636897  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 100769us
I20260709 12:27:30.599464 12077 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:30.649511  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 43737.5us
W20260709 12:27:30.665773  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.624488 11854 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:30.569677 12117 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:30.674654 12170 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:27:30.712733 12172 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:30.681769  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:30.715080  6019 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:27:30.693811  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:30.716269  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:30.696015  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.716996  5940 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:30.694602  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:30.603816 12110 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:30.593281 11949 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:30.678035 12135 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:30.665673 12166 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:30.727123 12135 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:30.727682 12135 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:30.727325 12166 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:30.729185 12174 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.729933 12174 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:30.697402  5865 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:30.702975  6773 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:30.707542  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:30.710757 12173 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:30.614485 12158 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:30.716497  6023 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:30.696302 12154 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:30.736462  5949 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:30.738052 12158 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:30.739598 12173 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:30.739827  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:30.740458  5944 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:30.712760 11895 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:30.745024  5871 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:30.704823  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:30.746359  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:30.747246  6015 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:30.752496  6018 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:30.723308  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:30.756134  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:30.756691  5941 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 17: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:30.716063 11904 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:30.762840  6019 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
W20260709 12:27:30.777532  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 77013us
I20260709 12:27:30.781813  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:30.783677  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 76279.3us
I20260709 12:27:30.784797  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:30.755373 12103 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:30.744360 11895 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:30.762233 12118 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.719588 12027 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:30.763427 11854 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.798172  5944 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:30.734175 12105 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:30.803982  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:30.807823  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:30.803128  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:30.808209  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:30.788041 11977 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:30.802248  5915 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:30.815255  6778 raft_consensus.cc:2393] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 17: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:30.787787 12089 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:30.763720 11949 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:30.792353 12103 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.800921 12070 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:27:30.826784  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 122041us
I20260709 12:27:30.820554  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:30.830353  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:30.832103  5949 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:30.836114  5872 raft_consensus.cc:2393] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:27:30.812469 12090 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.702s	user 0.015s	sys 0.000s
I20260709 12:27:30.845523  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:30.835119 11846 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:30.848119 11846 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:30.848758 11846 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:30.850291  5943 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 24: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:30.851496 12157 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:30.852200 12157 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:30.854799 12157 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:30.855536 12168 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:27:30.855849  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:30.856796  6768 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 17: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:30.859256 11949 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:30.863059  5940 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:30.815476 12117 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:30.868902 12002 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:30.868754 12169 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:30.873978  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 56541us
I20260709 12:27:30.874245 11949 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.874995 11949 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:30.875301  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:30.876367 12169 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:30.877475  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:30.879166 12136 consensus_meta.cc:287] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.671s	user 0.012s	sys 0.004s
W20260709 12:27:30.879907  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 80128.7us
I20260709 12:27:30.881233  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:30.890020  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:30.808804 11854 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:30.818897 11895 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:30.817313 12166 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:30.817546 11977 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:30.877099  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:30.884975  6766 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:30.834267 12103 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:30.882192 12139 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.654s	user 0.005s	sys 0.000s
W20260709 12:27:30.892890  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 55970.7us
I20260709 12:27:30.919375  6609 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:30.802801 12118 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:30.903684  6023 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:30.838323 11950 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:30.834513 12089 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:30.920663  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 176392us
I20260709 12:27:30.923694  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:30.862911 12090 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:30.931318  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 157293us
I20260709 12:27:30.935554  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:30.941800  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:30.910610 11977 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:30.810278 12153 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:30.963901  6609 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:30.939965  6611 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:30.968516  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.954000  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:30.949872 12164 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:27:30.950717  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:30.952016  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:30.971951  5940 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:27:30.953585 12118 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:30.950724 12167 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:27:30.954685 12137 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:30.954952 12142 consensus_meta.cc:287] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.662s	user 0.012s	sys 0.002s
I20260709 12:27:30.972563  6775 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:30.934228 12166 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:30.988545  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 48656.2us
I20260709 12:27:30.894344 12136 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:30.942072 12172 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:30.992725  6611 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:31.011402 12157 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:31.011565 11940 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:31.000437  6611 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:31.013007  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 91730.3us
I20260709 12:27:31.013427  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:30.944758 12174 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:30.986883 12142 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:30.995149 12108 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.857s	user 0.007s	sys 0.007s
W20260709 12:27:31.024204  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 60548.6us
W20260709 12:27:31.034755  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 95901.8us
I20260709 12:27:30.995361 11950 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:30.994851 12137 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.045447  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:31.045212 12175 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:31.012892 12172 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:31.012526 12157 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:31.051334  6611 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:31.052378  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1907) took 1657 ms. Trace:
I20260709 12:27:31.055035 12136 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:31.048287 12120 raft_consensus.cc:2764] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 12:27:31.059898 12137 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:31.078313 12117 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.078909 12117 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:31.080197 12137 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:31.081108 12117 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:31.082000  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:31.012586 11940 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:31.086419 11940 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:31.086838  5838 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:31.066749 12090 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:31.067898 11854 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:31.068073 12142 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:31.000636 12135 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:31.072557 11835 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.517s	user 0.004s	sys 0.007s
I20260709 12:27:31.046221 11950 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:31.073969 12089 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:31.060111 11556 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:31.110237  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:31.110688 11556 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:31.111122  6757 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:31.111845 11556 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:31.112705 12027 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:31.091012  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:31.113759  5944 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 17: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:31.113984  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:31.097160 12118 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:27:31.115082 12118 consensus_queue.cc:1243] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:31.115223  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:31.115654 12118 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:31.116005 12118 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:31.117847 12111 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:31.097734 11894 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:31.118584 11894 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.119029 11894 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:31.099059  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:31.121960 11949 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:31.122753 11949 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:31.096031 12105 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:31.123220 11949 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:31.123839 11949 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:31.108304 12183 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:31.124557  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:31.092132 12175 raft_consensus.cc:697] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:31.099483 12135 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:31.099711 12136 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:31.063784 12172 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:31.104331  5990 leader_election.cc:336] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.527s (SENT)
I20260709 12:27:31.063006  5946 rpcz_store.cc:276] 0709 12:27:29.382246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.382448 (+   202us) service_pool.cc:224] Handling call
0709 12:27:29.382941 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:29.383766 (+   825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:29.383800 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:29.384466 (+   666us) log.cc:844] Serialized 26 byte log entry
0709 12:27:29.827228 (+442762us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:29.827299 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:29.827444 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:30.881339 (+1053895us) spinlock_profiling.cc:243] Waited 5.56 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:31.029820 (+148481us) raft_consensus.cc:1685] finished
0709 12:27:31.029857 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.040034 (+ 10177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1704,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":130,"spinlock_wait_cycles":12780800,"thread_start_us":442050,"threads_started":1,"wal-append.queue_time_us":737664}
I20260709 12:27:31.108580  6766 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:31.131639  5990 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:31.109102 12141 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:27:31.110033 11950 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:31.040642 12108 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:31.130517  6756 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:31.138917  6773 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:31.112967 11556 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:31.116948 12118 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:31.161244  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:31.108769 11965 consensus_meta.cc:287] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.541s	user 0.015s	sys 0.000s
I20260709 12:27:31.172132  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:31.163297  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 112189us
I20260709 12:27:31.164484  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:31.167016  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.168180  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:31.111065 12070 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:31.117136 11846 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:31.180258  6016 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:27:31.187691  6785 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 18: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:31.138522 12027 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:31.185895  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 81639.1us
I20260709 12:27:31.173998 12158 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:31.127064 11835 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:31.182200 12173 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:31.195206  5949 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:31.172513 12142 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:31.177505 12137 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:31.180786 12070 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:31.173696 12136 raft_consensus.cc:697] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:31.196393  5990 leader_election.cc:336] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.573s (SENT)
I20260709 12:27:31.183725 12141 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:31.124408 12157 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:31.192389 12118 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:31.176790 11965 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:31.217773  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 93320.8us
I20260709 12:27:31.269557  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:31.270722  5943 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:31.276690  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:31.265707  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:31.280939  5872 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:31.284053  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:31.284849  5946 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:31.200726 12027 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:31.217543 12141 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:31.292546 11835 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:31.273955 12175 consensus_queue.cc:237] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:31.307380  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:31.308434  5872 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
W20260709 12:27:31.311069  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.313656  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:31.320046  5943 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:31.323004 12090 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:31.324091 12141 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:31.325214 12141 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:31.278707 12158 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:31.327955  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:31.330593  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:31.338404  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:31.339883  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:31.343930  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 154928us
W20260709 12:27:31.344961  5990 leader_election.cc:336] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.585s (SENT)
I20260709 12:27:31.344830  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:31.360778  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 47611.8us
I20260709 12:27:31.347648 12142 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:31.368521 12158 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:31.381531 12108 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:31.382290 12194 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:27:31.458150 12142 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:31.462949 12199 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.386241  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:31.463526 12199 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:31.464080  5870 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:31.391151 11965 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:31.390472 11977 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:31.392242 11846 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:31.318801 12173 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.467260  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:31.467130 12173 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:31.468386  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:31.468771 11977 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:31.397866 12136 consensus_queue.cc:237] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:31.470131  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:31.470964  6768 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 17: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:31.400332  6609 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:31.449745  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1647) took 1477 ms. Trace:
W20260709 12:27:31.472987  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 72817.4us
I20260709 12:27:31.372291 12030 consensus_queue.cc:260] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:31.408738 12169 consensus_meta.cc:287] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.532s	user 0.012s	sys 0.003s
I20260709 12:27:31.411080  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:31.417163 11950 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:31.422852 11904 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.423506  5874 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: [20.12-20.12]
W20260709 12:27:31.425905 12077 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.703s	user 0.007s	sys 0.008s
I20260709 12:27:31.426637 12117 consensus_queue.cc:1059] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:31.430068  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:31.431025  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 86106.9us
W20260709 12:27:31.435526 12103 consensus_meta.cc:287] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.502s	user 0.006s	sys 0.006s
I20260709 12:27:31.512398  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:31.440008  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 452465us
W20260709 12:27:31.513095  5990 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.589s (SENT)
I20260709 12:27:31.441068  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:31.441164 12172 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:31.447556  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.513715  5990 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:31.447988 12027 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:31.515985  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:31.516857  6019 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:31.451217  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.452273  5946 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:31.453663 11879 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:31.518982 11879 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:31.519273 11977 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:31.519884 11977 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:31.455124  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:31.469067  6766 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:31.475661  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:31.402149  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:31.478056 12202 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:31.475365  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:31.469484 12173 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:31.472795  6762 rpcz_store.cc:276] 0709 12:27:29.936843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.937769 (+   926us) service_pool.cc:224] Handling call
0709 12:27:30.102371 (+164602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:30.374581 (+272210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:30.382453 (+  7872us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:30.382521 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:30.399291 (+ 16770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.399398 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.413994 (+1014596us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54005758d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:31.414178 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":152614,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":167,"spinlock_wait_cycles":361725824,"thread_start_us":250069,"threads_started":1}
I20260709 12:27:31.486565  5949 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:31.502288  6768 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:27:31.492591  5915 connection.cc:256] RPC call timeout handler was delayed by -2.4368422249999639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1537s for a response
I20260709 12:27:31.438663  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:31.542101  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:31.563409  5915 connection.cc:256] RPC call timeout handler was delayed by -2.4323422249999567s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1542s for a response
W20260709 12:27:31.563997  5915 connection.cc:256] RPC call timeout handler was delayed by -2.4008422249999057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
W20260709 12:27:31.564528  5915 connection.cc:256] RPC call timeout handler was delayed by -2.3072422249997544s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16810000000000003s for a response
W20260709 12:27:31.565028  5915 connection.cc:256] RPC call timeout handler was delayed by -2.2667422250001437s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1726s for a response
I20260709 12:27:31.521610  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:31.565709  5915 connection.cc:256] RPC call timeout handler was delayed by -2.2478422250001131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
W20260709 12:27:31.567916  5915 connection.cc:256] RPC call timeout handler was delayed by -2.1884422250000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18130000000000002s for a response
W20260709 12:27:31.568200  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 54594.3us
W20260709 12:27:31.568413  5915 connection.cc:256] RPC call timeout handler was delayed by -2.1848422250000112s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1817s for a response
W20260709 12:27:31.570631  5915 connection.cc:256] RPC call timeout handler was delayed by -2.119142224999905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.189s for a response
W20260709 12:27:31.571252  5915 connection.cc:256] RPC call timeout handler was delayed by -2.099342224999873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1912s for a response
W20260709 12:27:31.571692  5915 connection.cc:256] RPC call timeout handler was delayed by -1.9211422250000396s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.211s for a response
W20260709 12:27:31.571998  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:31.572157  5915 connection.cc:256] RPC call timeout handler was delayed by -1.7789422249998097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2268s for a response
W20260709 12:27:31.573045  5915 connection.cc:256] RPC call timeout handler was delayed by -1.055342225000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30720000000000003s for a response
I20260709 12:27:31.573908  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:31.504452 12169 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:31.582101  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 106861us
W20260709 12:27:31.582696  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 127687us
W20260709 12:27:31.581189  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 59668.8us
I20260709 12:27:31.587894  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:31.509025 11965 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.590265  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:31.595541  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 82478.1us
I20260709 12:27:31.522413 11977 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:31.512488 12108 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.577816 12030 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:31.646345  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 72572.6us
I20260709 12:27:31.648156  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:31.569111 12103 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:31.651499 11894 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.532s	user 0.012s	sys 0.000s
I20260709 12:27:31.651911 12103 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:31.652060 11894 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:31.652812 12103 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:31.653617  5872 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 23: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:31.643380  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:31.654470  6768 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:27:31.647776  6757 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:27:31.590233 12136 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:31.615118  6019 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:31.514953 11904 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:31.455143 12070 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:31.663102  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:31.664856  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:31.666036 12172 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:31.605219 12199 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:31.667438 11950 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:31.668160 11904 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:31.603967 11965 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:31.670282 12092 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:31.671288  5865 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:31.671720 11965 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:31.676103  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:31.677049  5874 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:31.678113  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:31.678870  5873 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:31.684500  5867 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:31.686259  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:31.689929  5949 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:27:31.602097  5870 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:31.590379 12111 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:31.525930 12077 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:31.591681 12110 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:31.691075  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:31.665259 12070 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:31.660444  5990 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:31.586297 12202 raft_consensus.cc:697] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:31.698868  5987 leader_election.cc:400] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
I20260709 12:27:31.700497  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:31.701231  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:31.702643  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:31.709496  5865 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:27:31.604176 12157 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:31.638616 12117 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:31.712179 12136 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:31.742347 12136 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:31.744185  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:31.744807  6022 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:31.745121  6015 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:31.712935 12169 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
W20260709 12:27:31.746008  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 88235.6us
I20260709 12:27:31.747284 11904 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:31.747577  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:31.747789 12169 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:31.748270 12135 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:31.748713 11894 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:31.749414 12135 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:31.746248  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:31.749948 12135 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:31.711232 12092 raft_consensus.cc:2764] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:27:31.750538  6015 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:31.746210  5874 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 13. Current term is 14. Ops: []
W20260709 12:27:31.747179  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 86841.2us
I20260709 12:27:31.749958  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:31.752633  5915 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:31.752570  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:31.755244  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:31.756418  5990 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:31.757779  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:31.758553  5871 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:31.758036  5990 leader_election.cc:400] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
I20260709 12:27:31.757372  6024 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: [14.8-16.9]
I20260709 12:27:31.759485  5990 leader_election.cc:403] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:31.757851  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:31.758157  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:31.760439  6768 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:31.759483  5943 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:31.760394  6773 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:31.761650 12070 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:31.762068  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:31.753996  5913 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:31.763381  6023 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:31.712649  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:31.697213 12134 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:27:31.740913 11556 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.565s	user 0.012s	sys 0.001s
I20260709 12:27:31.657481 12108 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:31.750555 12135 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:31.714706  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 49943.6us
I20260709 12:27:31.776400  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:31.785792  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:31.785903  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:31.770200  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:31.774035  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:31.766067  6019 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:31.712743 12137 raft_consensus.cc:2764] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 12:27:31.713550  5987 leader_election.cc:403] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:27:31.749955 12180 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:31.789314  5874 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:31.762562 11992 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:31.789414  6015 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
W20260709 12:27:31.789976  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 91264.6us
I20260709 12:27:31.787418  6021 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
W20260709 12:27:31.792829  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 189213us
I20260709 12:27:31.793541  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:31.796404  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:31.771530 11977 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:31.795101  5990 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:31.802964  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:31.803164  5990 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:31.804246  5990 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:31.767004 12070 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.605616 11949 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:31.783715 12071 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
W20260709 12:27:31.827821  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.806231  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:31.807219  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 208767us
W20260709 12:27:31.808782  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 59149.7us
I20260709 12:27:31.713788 12111 raft_consensus.cc:697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:31.812057  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:31.829491  5949 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 19: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:31.795120 12117 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:31.797672 11556 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
I20260709 12:27:31.804513 12090 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:31.834707  6785 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:31.835284  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:31.813503 12134 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:31.840628 12134 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.841040 12134 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:31.841683 12071 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:31.842904  5868 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:31.842590 12071 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:31.844300  5868 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:31.846596  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:31.848937  6768 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:31.849474  5949 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 24: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:31.837288  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:31.850318  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:31.850620  5941 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:31.804459  6611 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:31.794179 11992 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:31.805195 11977 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.804342 12092 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:31.860445  6775 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 24: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:31.821836 12191 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:31.798532 12120 consensus_queue.cc:1059] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 12:27:31.864316  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:31.861231  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:31.823091 12108 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:31.871343  5941 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:31.861757  5873 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:31.863456  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:31.857509  5949 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
W20260709 12:27:31.874747  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 70419us
I20260709 12:27:31.857992  6020 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:27:31.815768 12208 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:31.886878  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 82669us
I20260709 12:27:31.810209 12173 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:31.823966 12077 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:31.875491  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:31.874378  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:31.868585 12191 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:31.854564 12071 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:31.857107 12183 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:31.879482  5873 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:31.861017 11977 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:31.881736  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:31.878664 12172 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:31.875866 12199 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:31.909940  5943 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:31.885061  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:31.883785 12210 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:27:31.831765 12169 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:31.821281 11879 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:31.890316  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 84117.7us
I20260709 12:27:31.812808 12070 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:31.903714  6773 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:31.884608 12108 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:31.860611 12092 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:31.911638  5941 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 18: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:31.912160  5949 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
W20260709 12:27:31.921437  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 47127.4us
I20260709 12:27:31.905529 12191 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:31.940268  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:31.940833  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:31.941097  6757 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:27:31.941567  6773 raft_consensus.cc:2376] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16. Re-sending same reply.
I20260709 12:27:31.942407  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:31.942634  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:31.943346  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:31.943737  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:31.944181  5870 raft_consensus.cc:2368] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
I20260709 12:27:31.925843 12111 consensus_queue.cc:237] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:31.926333  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:31.944332 12222 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:31.904781 11854 consensus_queue.cc:1059] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
W20260709 12:27:31.929612  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 98874.3us
I20260709 12:27:31.935133  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:31.905892 12117 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.936095  5943 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 12:27:31.950918  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:31.945533  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 74230.5us
I20260709 12:27:31.943368  6773 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:31.945042  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:31.964540  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:31.966239  5943 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:31.960083  5873 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 21: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:31.969676  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:31.969941  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:31.970749  6768 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
W20260709 12:27:31.951797 12182 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.893s	user 0.003s	sys 0.000s
I20260709 12:27:31.974007  6762 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:31.974766  6785 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 17: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:27:31.952929 12207 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:27:31.918217 11950 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:31.919351 12173 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:31.976536  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:31.927215 12108 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.003639 12108 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:32.003782 12217 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:31.952317 12092 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:31.926870 12172 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:32.011421  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:31.984359  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.984902  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:32.012727  6773 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:31.983467  5941 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:31.952951 11879 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:32.016047  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:32.016837  5870 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:32.020246 12027 raft_consensus.cc:3055] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:32.020766 12027 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:31.918542 11992 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:31.920495 12120 consensus_queue.cc:1059] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:27:31.999982  5873 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:32.026413  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:31.983871 12173 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.001777  5949 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:27:32.002871  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:32.030548  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:32.030642  6785 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:31.919173 12199 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:32.004969  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 112489us
I20260709 12:27:32.027956  5840 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:31.998023  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:32.030989  5868 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:31.878500 12202 consensus_queue.cc:237] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:32.024714  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 78570.3us
I20260709 12:27:31.965588 12117 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:32.035084  6768 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 14: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:32.037779  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:32.018383 12222 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:32.042411  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:31.952978 12191 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:32.036515 11879 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:32.046646 12183 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:32.047210 12183 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:32.047637 12183 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:32.049892 11879 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:32.050971 11879 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:32.051715  5945 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:27:32.052940 12141 consensus_meta.cc:287] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.727s	user 0.011s	sys 0.000s
I20260709 12:27:32.053308  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:32.043068  5987 leader_election.cc:400] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17.
I20260709 12:27:32.023599 11992 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:32.065940  6785 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:27:32.034209 12173 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:32.076433  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:32.078195  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:32.020931 12172 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:32.082556  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:32.091501  5872 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:32.084007  6023 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:32.085238  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:32.087778  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:32.089777 12221 raft_consensus.cc:3055] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:32.041659 11895 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:32.098716  5873 raft_consensus.cc:2410] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:27:32.024664 11950 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:32.102420  5865 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:27:32.095917  5868 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 21: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:32.069299 11949 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:32.108384  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:32.113117  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 75463.2us
W20260709 12:27:32.113369  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 114888us
I20260709 12:27:32.114212  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:32.113718  5940 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 24: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:27:32.124652  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 83702.7us
W20260709 12:27:32.124977  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 189955us
I20260709 12:27:32.123479  5840 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:32.129904  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 53589us
I20260709 12:27:32.130872  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:32.131790  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.131095  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:32.099224 12221 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:32.102701 11895 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:32.144357  5987 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:32.125893 12071 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:32.122182 12154 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:32.142701 12220 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:32.140568 11895 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:32.125540 12120 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:27:32.146418 12180 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:32.162124  6017 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 12:27:32.169875 12219 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:27:32.174512 12225 consensus_queue.cc:1059] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:27:32.140444 11879 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:32.180572  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.205904 12224 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:32.207547 12224 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:32.209318  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:32.210749 12221 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:32.214682  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1496) took 1707 ms. Trace:
I20260709 12:27:32.216177  5940 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:27:32.216552  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68590.6us
W20260709 12:27:32.212188  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 78208.3us
I20260709 12:27:32.215767  6016 rpcz_store.cc:276] 0709 12:27:30.485300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.485495 (+   195us) service_pool.cc:224] Handling call
0709 12:27:30.518068 (+ 32573us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:31.181878 (+663810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.191799 (+1009921us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b5400641754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:32.192169 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1522218240}
I20260709 12:27:32.184616 12174 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.194135 11879 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:32.242808  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 96985.3us
I20260709 12:27:32.213016 12071 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:32.268646  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:32.265424 12220 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:32.276276  5915 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:32.244750 11879 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:32.215590 12135 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:32.275122  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2025) took 1714 ms. Trace:
I20260709 12:27:32.274562  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:32.294800  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:32.295900  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:32.296869  6785 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:32.294339  6775 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
I20260709 12:27:32.289404  5869 rpcz_store.cc:276] 0709 12:27:30.560679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.562207 (+  1528us) service_pool.cc:224] Handling call
0709 12:27:30.599648 (+ 37441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:30.842683 (+243035us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:30.842717 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:30.877993 (+ 35276us) log.cc:844] Serialized 26 byte log entry
0709 12:27:30.920113 (+ 42120us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:30.920151 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.260230 (+1340079us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b540083f254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:32.260289 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:32.260355 (+    66us) raft_consensus.cc:1685] finished
0709 12:27:32.260370 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.274988 (+ 14618us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65631,"raft.queue_time_us":89947,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":9786,"spinlock_wait_cycles":526457728,"thread_start_us":41347,"threads_started":1,"wal-append.queue_time_us":148168,"wal-append.run_cpu_time_us":2572,"wal-append.run_wall_time_us":1049068}
W20260709 12:27:32.275701 12030 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.617s	user 0.008s	sys 0.005s
I20260709 12:27:32.302837  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:32.244958 12223 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:27:32.314913  5941 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:32.245332 12174 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:32.318645 12227 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.135s
I20260709 12:27:32.320362 12220 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:32.297890 12191 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:32.330895  6023 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:32.332187  6016 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:32.332439  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:32.333616  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:32.365119  6023 raft_consensus.cc:2393] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 17: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:32.338372  6785 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
W20260709 12:27:32.358115  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.368711  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:32.368727  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.377833  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:32.380240  6023 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:32.389313  6018 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:32.419520  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:32.428591  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:32.430552  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:32.431282  6019 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:32.419752  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:32.356360 11950 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:32.312088 12199 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:32.434002  6023 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:32.421084  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:32.339413 12135 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:32.441516  5940 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:32.368518 12030 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:32.441886  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 232621us
I20260709 12:27:32.333938 12117 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:32.449892 12120 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:32.448896  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:32.450939  5869 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:27:32.452327 12225 raft_consensus.cc:3055] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:32.455651  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:32.455987  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:32.456236  6773 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:32.456938  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:32.457734 12235 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
W20260709 12:27:32.477384 11977 consensus_meta.cc:287] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.553s	user 0.007s	sys 0.007s
I20260709 12:27:32.489336 11992 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:27:32.506336  5948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 1176) took 3149 ms (client timeout 2492 ms). Trace:
I20260709 12:27:32.499672 12174 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:32.506438  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:32.494156 12208 raft_consensus.cc:3055] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:32.489457 12077 raft_consensus.cc:3055] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:27:32.550235  5948 rpcz_store.cc:269] 0709 12:27:29.357219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.357417 (+   198us) service_pool.cc:224] Handling call
0709 12:27:32.506009 (+3148592us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b5400308ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:32.506211 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4110108032}
I20260709 12:27:32.510807 12172 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:32.445829 12223 raft_consensus.cc:3055] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:32.506757  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:32.500792 12234 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:27:32.450776  6018 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:27:32.512274 11949 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:32.512691 12157 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:27:32.485196  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.500154  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:32.519532  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:32.590920  6019 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:32.591527  5866 raft_consensus.cc:2393] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:32.521140  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:32.525928  5873 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:32.592307  5867 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:32.530193  6023 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 17: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:27:32.514070 12134 consensus_meta.cc:287] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.673s	user 0.005s	sys 0.012s
I20260709 12:27:32.550884  5868 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:32.579046 12223 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:32.607158  5868 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:27:32.606703 12223 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:32.608644 12223 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:32.486402 12071 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.555806  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:32.609938  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 54239.8us
W20260709 12:27:32.531008 12108 consensus_meta.cc:287] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.527s	user 0.010s	sys 0.004s
I20260709 12:27:32.610461 11895 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:32.477799  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.612375 12174 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:32.612942 11895 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:32.613006 12174 raft_consensus.cc:697] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:32.570375  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:32.541951 12070 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.004s
I20260709 12:27:32.524453 12229 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.564031  5949 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:32.524596 12120 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:32.569360 12077 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:32.576496  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:32.567874 12208 raft_consensus.cc:740] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:32.564814 12237 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.056s	user 0.000s	sys 0.002s
I20260709 12:27:32.530073 12225 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:32.578747  5944 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 14: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:32.595422  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:32.600186  5873 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:32.602021  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:32.609688 12071 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:32.610766 12108 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:32.570005  6021 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:32.634758  5948 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
I20260709 12:27:32.615916  5868 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:32.524818 12092 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:32.550253 11977 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:32.657673  5942 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:32.660511  6609 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:32.552426 12238 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:27:32.626266  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:32.664780  5949 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:32.634245 12222 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.003s
I20260709 12:27:32.638284  5865 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
I20260709 12:27:32.639791 12077 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:32.702044  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:32.640465 12183 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.592s	user 0.012s	sys 0.004s
I20260709 12:27:32.626036 12229 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:32.642707 12071 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:32.642820  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:32.704612  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:32.706241  5944 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:32.632398 12166 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:32.645824 12108 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:32.646023  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1622) took 1125 ms. Trace:
I20260709 12:27:32.625535 12157 consensus_queue.cc:1059] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:27:32.646589  6768 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:32.625965  5869 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:32.605258 12134 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:32.662653 12193 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:27:32.663753  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.663930  6015 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:32.626880  5866 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:32.671492  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:32.675712  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 144435us
I20260709 12:27:32.633976 12172 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.079s
W20260709 12:27:32.656919 12154 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.505s	user 0.012s	sys 0.000s
I20260709 12:27:32.645828 12232 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:32.657307 12030 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:32.711303  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:32.742632  5942 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:32.623454 12070 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:32.662995  6023 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:32.619024 11879 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:32.632498 11949 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 12:27:32.710879  5943 rpcz_store.cc:276] 0709 12:27:31.520331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.967104 (+446773us) service_pool.cc:224] Handling call
0709 12:27:31.967413 (+   309us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:31.981571 (+ 14158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:31.981618 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:32.019871 (+ 38253us) log.cc:844] Serialized 52 byte log entry
0709 12:27:32.260185 (+240314us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:32.287269 (+ 27084us) log.cc:844] Serialized 14 byte log entry
0709 12:27:32.585184 (+297915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.585271 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:32.645723 (+ 60452us) raft_consensus.cc:1685] finished
0709 12:27:32.645748 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.645950 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7712,"prepare.queue_time_us":34578,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":217355,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":10506,"spinlock_wait_cycles":29888768,"thread_start_us":497190,"threads_started":2,"wal-append.queue_time_us":423031}
I20260709 12:27:32.726970  5840 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:32.700488  5874 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:32.656445 12227 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
W20260709 12:27:32.717288 12173 consensus_meta.cc:287] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.618s	user 0.013s	sys 0.000s
I20260709 12:27:32.707312 12222 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:32.707935 12166 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:32.759624  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:32.761040  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:32.709169 12202 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:27:32.712783 12183 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:32.762967  6018 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:32.767897  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:32.768721 12191 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:32.734385 12120 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:32.747071 12154 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:32.776684  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 520456us
I20260709 12:27:32.784111  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:32.785154 12248 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:32.774768  5949 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:27:32.757140 12193 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:32.787418  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:32.754040 12077 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:32.737335 12180 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:27:32.788954  6022 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:32.715648 12071 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:32.791615  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:32.794050  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 122688us
W20260709 12:27:32.798942  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 70049.4us
I20260709 12:27:32.804131  5870 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:32.762902 12173 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:32.877715  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.902437  6020 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:27:32.734210 12174 consensus_queue.cc:237] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:32.792824 12193 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 24 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:32.800441 12153 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:27:32.919459  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 132112us
I20260709 12:27:32.773674 12251 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:32.980710  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1516) took 1056 ms. Trace:
I20260709 12:27:32.962203  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1625) took 1298 ms. Trace:
I20260709 12:27:32.755834 12208 consensus_queue.cc:260] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:32.966215  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1512) took 1046 ms. Trace:
W20260709 12:27:32.978641  6778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 1406) took 1848 ms (client timeout 1743 ms). Trace:
I20260709 12:27:32.958252  6016 raft_consensus.cc:2368] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20.
I20260709 12:27:33.003772  5870 rpcz_store.cc:276] 0709 12:27:31.924556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.104032 (+179476us) service_pool.cc:224] Handling call
0709 12:27:32.124099 (+ 20067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:32.804727 (+680628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.980304 (+175577us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b54004f5bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:32.980607 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1563288960}
W20260709 12:27:33.015806  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 247974us
W20260709 12:27:32.953532  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.004129  5941 rpcz_store.cc:276] 0709 12:27:31.664331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.024497 (+360166us) service_pool.cc:224] Handling call
0709 12:27:32.040534 (+ 16037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:32.315616 (+275082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.961865 (+646249us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b5400602a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:32.962122 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":630307200}
I20260709 12:27:33.012022  5867 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:33.019326  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:33.021382  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:33.017935  6778 rpcz_store.cc:269] 0709 12:27:31.132227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.132464 (+   237us) service_pool.cc:224] Handling call
0709 12:27:32.978319 (+1845855us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400328cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:32.978530 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":894827264}
I20260709 12:27:33.018303  5872 rpcz_store.cc:276] 0709 12:27:31.919636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.926432 (+  6796us) service_pool.cc:224] Handling call
0709 12:27:31.926767 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:32.092188 (+165421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.965887 (+873699us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b5400193054. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:32.966129 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378305408}
I20260709 12:27:33.032970  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:32.967964 12226 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.825s	user 0.002s	sys 0.000s
I20260709 12:27:32.991355 11846 raft_consensus.cc:3055] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:32.970652 12169 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:33.038295  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:32.998370 12092 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:33.039418  6611 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:32.907498 12154 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.919866 12134 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:32.991700 11879 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:32.995129 11977 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:33.036405  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.108811 12193 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:32.955366 12173 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:32.987241 12183 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:32.992965 12070 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:33.068917 11835 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:33.045594 11846 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:33.095552  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:33.096236 12208 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:33.097138 12243 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:33.098172 12160 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 12:27:33.002892 12166 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:32.949000 11949 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:32.804994 12071 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:33.106343 12232 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:32.922799 12120 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:32.915182 12111 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:33.115893  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 77701.4us
W20260709 12:27:33.117307  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 78005.5us
I20260709 12:27:33.217103  5840 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:33.117506  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:33.114632  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:33.027791 12153 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.120162  5942 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:32.953716 12200 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.345s
I20260709 12:27:33.126039  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:33.127249  6019 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:27:32.900372 12202 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:33.047847 12191 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:32.996294 12027 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:33.037735 12251 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:33.112488 12227 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.069181 12169 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:33.126861 12180 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:33.099195 11950 raft_consensus.cc:3055] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:33.130189 12198 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:27:33.160542  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 360663us
I20260709 12:27:33.158746 12244 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.119596 12154 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:33.143069 11879 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:33.136515 12223 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:33.196061  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 100593us
I20260709 12:27:33.222011 12243 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.154649 11977 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:33.156509 12229 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:33.202080 12257 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:27:33.155665 11949 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:33.206226 12225 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:33.199574 11835 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.219215  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:33.223336  5946 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:33.223941  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:33.227267  5947 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:33.227559  6021 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:33.270152  5943 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:27:33.162276 12166 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:33.225713 12255 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.318s	user 0.001s	sys 0.000s
I20260709 12:27:33.250939  6609 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:33.250909  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:33.256222  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:33.223495 12153 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:33.269294  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 50162.8us
I20260709 12:27:33.213964 11846 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:33.247367 12198 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.245208 12180 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:33.246079 12191 raft_consensus.cc:697] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:33.222786 12183 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:27:33.247611 11950 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:33.303639 12214 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:33.304517  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1526) took 1231 ms. Trace:
I20260709 12:27:33.262007 12243 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:33.305073 12175 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.232762 12158 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:27:33.305274  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:33.309479  5946 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
W20260709 12:27:33.311426  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 55388.5us
I20260709 12:27:33.229902 12202 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:33.342378  6022 rpcz_store.cc:276] 0709 12:27:32.072915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.113150 (+ 40235us) service_pool.cc:224] Handling call
0709 12:27:32.133162 (+ 20012us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:32.800049 (+666887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.304164 (+504115us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b5400363dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:33.304429 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1489655680}
I20260709 12:27:33.347957  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1471) took 1019 ms. Trace:
I20260709 12:27:33.348624  6016 rpcz_store.cc:276] 0709 12:27:32.328439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.415932 (+ 87493us) service_pool.cc:224] Handling call
0709 12:27:33.347697 (+931765us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54006b99d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:33.347886 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1219368192}
I20260709 12:27:33.350754  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1522) took 1286 ms. Trace:
I20260709 12:27:33.254509 12064 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 12:27:33.341741  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:33.334918  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.359390  6775 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:33.248286 12227 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:33.339845  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:33.370743  6778 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:33.338518  6793 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:27:33.322640  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:33.376390  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 71228us
I20260709 12:27:33.376514  5943 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:33.377599  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:33.351339  6020 rpcz_store.cc:276] 0709 12:27:32.064209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.075946 (+ 11737us) service_pool.cc:224] Handling call
0709 12:27:32.099216 (+ 23270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:32.909780 (+810564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.350395 (+440615us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54001612d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:33.350670 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1845882752}
I20260709 12:27:33.364419  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:33.379617  6785 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:33.279806 11879 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:33.264814 12089 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.051s
W20260709 12:27:33.334600 12256 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 12:27:33.311077 12198 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:33.389890  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 47865.1us
I20260709 12:27:33.280184 12225 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:33.260768 12223 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:33.313424 12180 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:33.363436 12175 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:33.402611 12175 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:33.404336 12175 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:33.406452 12243 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:33.407114 12261 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:27:33.336062 12251 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:33.410005 12266 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:33.411731 12266 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.412185 12266 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:33.413089 12254 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.498s	user 0.000s	sys 0.004s
W20260709 12:27:33.413779  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.414059 11879 raft_consensus.cc:697] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:33.415364 12227 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:33.415313 11879 consensus_queue.cc:237] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:33.261158 12244 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:33.339421 11846 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:33.398490  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:33.335228 12169 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:33.388763 11854 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:33.334913 12260 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.152s
W20260709 12:27:33.399791 12267 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 12:27:33.389031  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 166785us
I20260709 12:27:33.397410 12158 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:33.310679 12157 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:33.275250 11835 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:33.431007  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:33.424697  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:33.434337  5866 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:33.370425 12265 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:33.450496  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:33.434710 12117 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.365451 11977 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:27:33.464164  5874 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:27:33.489138 12117 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:33.489920  5871 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:27:33.491770 12117 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:33.480271 12248 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:33.468806 12169 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:33.495807 12191 consensus_queue.cc:237] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:33.470717 12258 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.471099 12262 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.313773 11949 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:33.467718 11950 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:33.475397 11835 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:33.477272  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:33.478811 12183 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:33.421774  6775 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
W20260709 12:27:33.481031  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 50146.9us
I20260709 12:27:33.509009  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:33.435055 12160 raft_consensus.cc:2764] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
W20260709 12:27:33.481853  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 82726.4us
I20260709 12:27:33.512434  5870 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:33.512996  5915 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:33.492631 12244 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:33.501296  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1241) took 1501 ms. Trace:
I20260709 12:27:33.465188  6785 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:27:33.482669  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:33.518308  6016 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:33.519591  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:33.515597  6762 rpcz_store.cc:276] 0709 12:27:31.999853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.013117 (+ 13264us) service_pool.cc:224] Handling call
0709 12:27:32.032267 (+ 19150us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:32.032445 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:32.032460 (+    15us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:32.992507 (+960047us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b2c0001ffa8. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:27:32.993124 (+   617us) log.cc:844] Serialized 26 byte log entry
0709 12:27:33.138000 (+144876us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:33.158690 (+ 20690us) log.cc:844] Serialized 14 byte log entry
0709 12:27:33.160224 (+  1534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.160350 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:33.493647 (+333297us) raft_consensus.cc:1685] finished
0709 12:27:33.500936 (+  7289us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b18000b1b20. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb96522 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:27:33.501048 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.501223 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7687,"prepare.queue_time_us":20032,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":2321,"raft.queue_time_us":4512,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":10959,"spinlock_wait_cycles":16986496,"thread_start_us":144351,"threads_started":1,"wal-append.queue_time_us":234593}
I20260709 12:27:33.481725 12157 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:33.530331  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:33.531430  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:33.524786 12214 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:33.537261  5870 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:27:33.538763  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:33.538513  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:33.539593  6609 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:33.525468 11977 raft_consensus.cc:2764] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:27:33.525444 11949 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:33.546185 12223 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:33.547048 12266 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:33.547151  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.521569 12174 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:33.548008  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.525421 12175 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:27:33.550714 12174 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:33.551167 12174 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:33.552641 12278 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:33.553113 12202 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:33.553453 12154 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:33.554060 12280 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:33.553844 12223 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:33.554633 12280 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:33.555202 12223 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:33.549374  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:33.557106  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:33.557461 12157 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:33.528781 11835 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.558570 11835 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:33.560714 11835 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:33.558003 12117 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:33.561882 12244 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:33.562222 12227 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:33.562513 11977 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:33.563515 11977 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:33.562855 12227 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:33.550575 12175 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:33.555616 12223 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:33.567261  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:33.567387  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:33.569562  5869 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:33.573652  6778 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:33.574195  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:33.575206  5946 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:33.505434 12258 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:33.502642 12248 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:33.594385  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:33.534407 12265 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:33.582855  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:33.583014  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:33.583469 12232 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:33.583361 12223 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:33.584169 12221 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:33.584717 12227 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:33.584193  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:33.585908  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:33.585299  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:33.586257 12202 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:33.586503 12030 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:33.507098 12262 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:33.518921 12064 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
W20260709 12:27:33.587507  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.594120 12266 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:33.592329  5869 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:33.562112 12117 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.626331 12117 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:33.626570 12214 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:33.579493  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 127600us
I20260709 12:27:33.536054 12160 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 12:27:33.600132 12229 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:33.603617  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:33.605202  5915 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:33.646453  5949 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:33.577108 12157 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:33.610754  5944 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:33.608075  5948 raft_consensus.cc:2376] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20. Re-sending same reply.
I20260709 12:27:33.607304  5940 raft_consensus.cc:2376] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14. Re-sending same reply.
I20260709 12:27:33.589598 12258 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:33.635047  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.552038 12174 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:33.562638 12244 raft_consensus.cc:697] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:33.608340 12265 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:33.640209 12071 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:33.642541 12276 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:33.651717  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46709.7us
I20260709 12:27:33.659740 12223 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:33.634485  5838 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:33.689123  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 54720.9us
I20260709 12:27:33.689141  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:33.597517 11895 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:27:33.616868 12175 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.616374 12108 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:33.636082 12217 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:33.656560 12110 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.677070  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:33.694633  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:33.719745  5940 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:33.649752 12030 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.607661 12232 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:33.645458 12229 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:33.603622 12203 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 12:27:33.701886  5838 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:33.748273  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 46498.7us
I20260709 12:27:33.694810  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1531) took 1034 ms. Trace:
W20260709 12:27:33.711406  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.649828 12070 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:33.676766  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:33.660328 12077 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:33.658349 12266 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 22 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:33.555636 12280 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:33.728895  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.756278  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:33.703119 12282 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:33.751809  6024 rpcz_store.cc:276] 0709 12:27:32.660419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.661902 (+  1483us) service_pool.cc:224] Handling call
0709 12:27:33.688965 (+1027063us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b5800480098. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:27:33.694704 (+  5739us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3193600}
I20260709 12:27:33.648404 12153 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:33.759260  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 104038us
I20260709 12:27:31.132397  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:33.713747 12217 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:33.721570 12137 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.747323 12229 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:33.780009  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:33.777328  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:33.696041 12158 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 12:27:33.720965 12193 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:33.746723 11879 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.802546  6019 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 12:27:33.913556  6019 pending_rounds.cc:85] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 13
I20260709 12:27:33.913892  6019 pending_rounds.cc:107] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:27:33.914130  6019 raft_consensus.cc:2889] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:33.780992  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:33.915560  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.789892 12251 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:33.917297 12214 raft_consensus.cc:3055] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:33.917802 12214 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:33.918431 12261 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:33.918684 12214 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:33.726653 12030 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:33.896659  5867 raft_consensus.cc:2368] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21.
I20260709 12:27:33.717239 11835 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
W20260709 12:27:33.896821  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 351809us
I20260709 12:27:33.777087 12174 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:33.703022 12277 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
W20260709 12:27:33.798180 12208 consensus_meta.cc:287] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.647s	user 0.009s	sys 0.005s
W20260709 12:27:33.905004  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 125221us
W20260709 12:27:33.790808 12237 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:27:33.717995 12108 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:33.716794 12175 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:33.727061 12157 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 12:27:33.727310 12110 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:33.925299  5990 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:33.804865  5873 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:34.014389 12077 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:33.895673 12120 consensus_meta.cc:287] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.675s	user 0.012s	sys 0.001s
I20260709 12:27:33.947793 12229 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:33.951928  5840 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:33.985525  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 235134us
W20260709 12:27:33.937009 12180 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.537s	user 0.015s	sys 0.008s
I20260709 12:27:33.774868 12070 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:33.903813 12227 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.693781 12276 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:33.991649 12217 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:34.056881  5943 raft_consensus.cc:2410] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 12:27:33.935418  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:34.058708  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:34.035138  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 109952us
I20260709 12:27:34.059970  5990 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:33.988312 11977 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:33.993706 12169 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.807674 12137 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:33.999948 12279 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.922660 12193 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:33.999503 12251 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:34.013808 12173 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.939240 11879 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:34.067652  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.894625 12244 consensus_queue.cc:237] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:34.038756 12281 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.042742 12258 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:34.040544 11835 consensus_queue.cc:1243] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:27:34.087512  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 410799us
W20260709 12:27:34.085906  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 86518.4us
I20260709 12:27:34.045789 12294 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:27:34.092267  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.105597  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:34.166750  6018 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:34.068675 11940 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:34.086799 12225 consensus_meta.cc:287] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.665s	user 0.012s	sys 0.000s
W20260709 12:27:34.179838  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 74395.3us
I20260709 12:27:33.953964 12265 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:34.183305  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:34.080328 12203 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.199169  5945 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:27:34.199637  5945 pending_rounds.cc:85] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 8
I20260709 12:27:34.199885  5945 pending_rounds.cc:107] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:27:34.200114  5945 raft_consensus.cc:2889] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:34.013556 12266 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:34.076136 12248 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:34.205463  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.222472 12225 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.222958 12300 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.223439 12225 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:34.218866  6016 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:34.223611 12300 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:34.226563 12225 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:34.226702 12300 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:34.228323 12300 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:34.229542 12300 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:34.210242  6019 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
W20260709 12:27:34.232173  5915 connection.cc:256] RPC call timeout handler was delayed by -1.6203535810000176s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
W20260709 12:27:34.235543  5915 connection.cc:256] RPC call timeout handler was delayed by -1.4727535810002337s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
I20260709 12:27:34.239075  5944 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:34.239207  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:34.240437  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:34.237596  5915 connection.cc:256] RPC call timeout handler was delayed by -1.4493535810001958s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1769s for a response
W20260709 12:27:34.280148  5915 connection.cc:256] RPC call timeout handler was delayed by -1.4250535810001566s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1796s for a response
W20260709 12:27:34.280572  5915 connection.cc:256] RPC call timeout handler was delayed by -1.3458535810000285s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1884s for a response
W20260709 12:27:34.280943  5915 connection.cc:256] RPC call timeout handler was delayed by -1.3017535809999572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19330000000000003s for a response
W20260709 12:27:34.281334  5915 connection.cc:256] RPC call timeout handler was delayed by -1.210853581000265s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2034s for a response
W20260709 12:27:34.281889  5915 connection.cc:256] RPC call timeout handler was delayed by -1.2090535810002621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2036s for a response
W20260709 12:27:34.282368  5915 connection.cc:256] RPC call timeout handler was delayed by -1.1550535810001747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2096s for a response
I20260709 12:27:34.083386 12169 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:34.296271  6022 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:34.080978 12227 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:34.069175 12180 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:34.198942 12120 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.292668  6024 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:34.197953 12282 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:34.198679 12217 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:34.293534  5915 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:34.327924  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:34.203033 12262 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:34.211746 11977 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:34.161994 12173 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:34.329279  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:34.296533 11846 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.862s	user 0.014s	sys 0.000s
I20260709 12:27:34.334738  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:34.342595  5872 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:34.198567 11940 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:34.344064  5913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.025s	sys 0.056s
W20260709 12:27:34.345317  5913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.025s	sys 0.056s
W20260709 12:27:34.345717  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 617431us
I20260709 12:27:34.346985 11977 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:34.347528  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:34.348094 12137 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:34.349725 12286 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
W20260709 12:27:34.350435 12301 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
W20260709 12:27:34.350731 12289 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
W20260709 12:27:34.351882 12293 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.307s	user 0.000s	sys 0.000s
I20260709 12:27:34.353179 12304 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:34.093205 12279 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:34.358347  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:34.356511  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:34.349233  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:34.354794  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:34.359779  6021 raft_consensus.cc:2410] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:27:34.360821  5944 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:34.331665 12258 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:34.366844  6019 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:27:34.363662  6024 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:34.374416  6016 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:27:34.378536  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:34.349370 11977 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:34.336246 12232 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:34.387665  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.182819 12281 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:34.382215  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 201222us
I20260709 12:27:34.440939  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:34.336704 12282 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:34.386899  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:34.444880  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.382560 12318 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:34.388738  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:34.421525 12030 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:34.365752 12222 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.015s	sys 0.010s
I20260709 12:27:34.329391 12120 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:34.218854 12203 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:34.291379 12077 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:34.432541  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 51714.3us
I20260709 12:27:34.441071 12305 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.436151  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:34.451469 12222 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.016s	sys 0.010s
I20260709 12:27:34.379571 12300 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:34.469204  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:34.341373 12262 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:34.355829 12280 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:27:34.440001  5942 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.015s	sys 0.036s
I20260709 12:27:34.388235 12248 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:27:34.426118 12070 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:34.422035  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:34.467720 12198 consensus_meta.cc:287] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.037s	user 0.007s	sys 0.007s
W20260709 12:27:34.446832  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 494972us
I20260709 12:27:34.332456 12251 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:34.449703  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:34.379464 11846 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
W20260709 12:27:34.439085 12272 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.429s	user 0.001s	sys 0.001s
I20260709 12:27:34.425238 12232 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:34.385943 12304 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:27:34.455587  5944 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:34.460847 12282 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:34.463589 12294 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.442646 12108 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:34.436565  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:34.466761 12292 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:34.465775 12291 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.422s	user 0.000s	sys 0.002s
W20260709 12:27:34.465576 12223 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.772s	user 0.009s	sys 0.000s
W20260709 12:27:34.646927 12309 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.259s	user 0.000s	sys 0.000s
I20260709 12:27:34.654037 12180 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:34.656461 12203 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:34.653321  6022 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
W20260709 12:27:34.658982 12313 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:34.484193  6015 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:27:34.486403  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:34.661252  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 174989us
W20260709 12:27:34.493315  5942 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.015s	sys 0.036s
I20260709 12:27:34.661942 11846 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:34.662606  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1635) took 1096 ms. Trace:
I20260709 12:27:34.662747 11846 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:34.663673  6015 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:34.663190  5942 rpcz_store.cc:276] 0709 12:27:33.565962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.566119 (+   157us) service_pool.cc:224] Handling call
0709 12:27:33.566488 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:33.566731 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:33.658540 (+ 91809us) log.cc:844] Serialized 14 byte log entry
0709 12:27:34.662284 (+1003744us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:34.662336 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:34.662379 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.662443 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:34.662547 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48462,"prepare.queue_time_us":240756,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":327998,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":8332,"thread_start_us":1083346,"threads_started":2,"wal-append.queue_time_us":16112}
W20260709 12:27:34.467515 11949 consensus_meta.cc:287] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.921s	user 0.006s	sys 0.005s
I20260709 12:27:34.664237 11846 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:34.429821 11977 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:34.467940 12296 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.391s	user 0.002s	sys 0.000s
I20260709 12:27:34.637104 12173 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:34.662052  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:34.554697  5840 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:34.667534  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 168030us
I20260709 12:27:34.494921  6021 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:34.469916 12110 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:34.481356 11879 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:34.470525 12175 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:34.491369 12303 raft_consensus.cc:3055] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:34.669410 11835 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:34.670082 11835 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 22 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:34.597968 12311 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.671133 11835 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:34.671183 12311 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:34.672767 12117 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.046s	user 0.013s	sys 0.004s
I20260709 12:27:34.493397 12280 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:34.673401  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:34.673889 12280 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:34.674326 11846 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:34.625700  5948 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:34.630698  5913 leader_election.cc:400] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21.
I20260709 12:27:34.675078  5913 leader_election.cc:403] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:27:34.675352 12225 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:34.675628  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 48432.4us
I20260709 12:27:34.676136 12225 raft_consensus.cc:2764] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:27:34.676802 12225 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21.
I20260709 12:27:34.623245  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:34.678524 12120 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:34.678730  5944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3"
dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:48330
I20260709 12:27:34.680507  5944 raft_consensus.cc:606] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:34.681079  5944 raft_consensus.cc:614] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:34.634835 12294 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:34.683548 12311 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:34.687440  5915 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:34.688690  5944 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
W20260709 12:27:34.689579  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.626309 12232 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:34.691897  6608 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.019s	sys 0.063s
W20260709 12:27:34.702915  6608 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.019s	sys 0.063s
I20260709 12:27:34.491310 12262 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:34.689569  5944 pending_rounds.cc:85] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 9
I20260709 12:27:34.714419  5944 pending_rounds.cc:107] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:27:34.714888  5944 raft_consensus.cc:2889] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:34.695744  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:34.476341 12305 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:34.611503 12193 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:34.701463  5987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.017s	sys 0.017s
I20260709 12:27:34.706596  6775 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:34.716569  5987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.017s	sys 0.017s
I20260709 12:27:34.706835  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:34.709319  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:34.624533 12092 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:34.715973  5865 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
W20260709 12:27:34.718761  5840 connection.cc:256] RPC call timeout handler was delayed by -1.0800589099999343s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18430000000000002s for a response
I20260709 12:27:34.718254  6773 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:34.719915  5913 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:34.723995  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:34.724779  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.727844  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:34.732602  5840 leader_election.cc:400] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20.
I20260709 12:27:34.733173  5840 leader_election.cc:403] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:27:34.743551  6773 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:34.726682  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:34.747299  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 59966.9us
I20260709 12:27:34.750428  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:34.750725  5866 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:34.750499  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:34.752147  5867 raft_consensus.cc:2410] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:27:34.751919  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:34.751809  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:34.757254  6778 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:27:34.752933  5870 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:34.762143  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:34.757601  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:34.764199  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:34.764660  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:34.765046  6793 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:34.765544  6775 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:34.767774  5870 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 14: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:34.767943  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:34.768716  6762 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:34.769018  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:34.770079  5867 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 21: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:34.757470  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:34.669611 12303 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:34.773274  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:34.770228  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:34.773985  6766 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:34.771869  5868 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:27:34.775656  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 55854.9us
I20260709 12:27:34.775949  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:34.777518  5865 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:34.777659  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:34.673310 12117 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:34.778767  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:34.713941 12173 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:34.714188 12175 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.780588  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:34.590245 12198 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:34.781159  6773 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 18: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:27:34.782389  5947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.019s	user 0.090s	sys 0.111s
W20260709 12:27:34.783517  5947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.090s	sys 0.111s
I20260709 12:27:34.726603 11879 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.756594 12223 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:34.759609 12227 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:34.785794  5990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.007s	sys 0.007s
I20260709 12:27:34.715444 12169 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:34.788165  6778 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:34.790616  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1634) took 1225 ms. Trace:
I20260709 12:27:34.784247  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:34.665922 11977 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:34.793573  6793 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 12:27:34.713901 12311 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 12:27:34.826153  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:34.827265  6766 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:34.813299  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:34.777136 12258 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:34.778162 11940 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:34.812253  5990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.007s	sys 0.007s
I20260709 12:27:34.811353  5947 rpcz_store.cc:276] 0709 12:27:33.565564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.565755 (+   191us) service_pool.cc:224] Handling call
0709 12:27:33.594410 (+ 28655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:33.594646 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:33.748173 (+153527us) log.cc:844] Serialized 14 byte log entry
0709 12:27:34.790147 (+1041974us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:34.790217 (+    70us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:34.790268 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.790346 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:34.790523 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6003,"prepare.queue_time_us":399434,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":6147,"thread_start_us":1151770,"threads_started":2}
W20260709 12:27:34.831748  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 771802us
I20260709 12:27:34.834770  6775 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:34.791640 12222 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:34.838902  6775 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: [17.9-18.10]
I20260709 12:27:34.786729 12279 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:34.836786  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:34.834197  6610 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.008s	sys 0.008s
I20260709 12:27:34.856658  6778 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: [21.11-21.11]
W20260709 12:27:34.857475  6610 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.008s	sys 0.008s
W20260709 12:27:34.833590 12111 consensus_meta.cc:287] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.607s	user 0.010s	sys 0.006s
I20260709 12:27:34.861711  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:34.786824 12175 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:34.864614  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:34.862448  5990 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:34.862927  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:34.779949 12173 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:34.830295 12300 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:34.785290 12070 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:27:34.850468 12304 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:34.802342 12120 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:27:34.831418 12258 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.802956 11879 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:34.855142 12222 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.882552  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:34.881603  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:34.867334 12166 consensus_meta.cc:287] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.589s	user 0.017s	sys 0.000s
I20260709 12:27:34.883667  6778 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
W20260709 12:27:34.896365 12221 consensus_meta.cc:287] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.281s	user 0.011s	sys 0.003s
W20260709 12:27:34.896608 12261 consensus_meta.cc:287] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.977s	user 0.009s	sys 0.002s
I20260709 12:27:34.897058 12261 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:34.897269 12134 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.898205 12307 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20.
I20260709 12:27:34.898041 12134 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:34.899097 12315 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:34.899586 12315 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.900017 12315 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:34.900302  6023 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:34.864204 12111 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:34.833420 12225 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:27:34.874434 12169 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:34.909304  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:34.875332 12304 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.868069 12281 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:34.874828 11950 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:34.899533  5867 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:34.943626 12262 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:34.945026 12175 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:34.904505 12227 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:27:34.947194  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:34.950598 12305 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:34.954756  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:35.031327  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 76664.6us
I20260709 12:27:35.033900 12305 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:35.034685 12305 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:35.035821 12305 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:34.956028 12232 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:34.956557 12320 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:35.040236 12320 raft_consensus.cc:697] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:35.040889  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.957597  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:34.959307 12251 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:35.043027 12251 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:34.965946  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 83101.8us
I20260709 12:27:35.046638  5990 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:34.970569 12223 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:34.972114  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2041) took 1549 ms. Trace:
I20260709 12:27:34.972947 12278 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:35.058023 12278 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:35.058279  5990 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:35.058461  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.060127  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:34.966857 12294 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:34.973672  6766 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:34.975675  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1535) took 2306 ms. Trace:
I20260709 12:27:34.889937 12222 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:34.980572 12214 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:35.065786  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.008865 12070 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:35.076683 12070 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:35.079381 12070 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:34.984993  6762 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:34.981945  6775 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:35.080868  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:34.986598  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.082588 12244 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:35.083222 12244 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.083642 12244 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:35.085117 12117 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:34.878940 12258 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:35.087997  6015 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:34.881484 12300 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:34.857286 12303 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:34.871161 12327 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:35.002681  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.094182  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:35.048141 12262 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:35.003638  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 190390us
I20260709 12:27:35.095086  6015 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:34.937873 12137 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.050253  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:34.991048 12304 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:35.065524 12175 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.104193  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:35.104435  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:35.055737  6778 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:35.051025 12223 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:35.061470  6023 rpcz_store.cc:276] 0709 12:27:32.669805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.756212 (+ 86407us) service_pool.cc:224] Handling call
0709 12:27:32.756608 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.900783 (+2144175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.975353 (+ 74570us) spinlock_profiling.cc:243] Waited 2.14 s on lock 0x7b540041df54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:34.975570 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4928929536}
I20260709 12:27:35.066870  6766 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
W20260709 12:27:34.928814 12174 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.935s	user 0.007s	sys 0.006s
I20260709 12:27:35.059543  6793 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:35.109169  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:35.071192  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:35.074694  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.105118  6015 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:35.088156  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:35.114228  5947 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 21: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:35.114032 12279 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:35.081188  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.116649  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:35.007892  6785 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:35.097164  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:35.066416 12071 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:35.051250  5872 rpcz_store.cc:276] 0709 12:27:33.422670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.423519 (+   849us) service_pool.cc:224] Handling call
0709 12:27:33.423947 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.343147 (+919200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.971783 (+628636us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b54007f30d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:34.972032 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2112099072}
I20260709 12:27:35.101166  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:35.105590  6778 raft_consensus.cc:606] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:35.119547  6778 raft_consensus.cc:614] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:35.107875  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.119956 12092 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:35.118328  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:35.070356  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.091545 12251 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:35.094650 12263 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.123369  5872 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:35.127954  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:35.122418  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.132249  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:35.176931  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.177587  5874 raft_consensus.cc:2410] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 21 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:27:35.177891  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:35.187604  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.187963  6778 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
W20260709 12:27:35.162031  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.190176  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:35.194237  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.195441  6793 raft_consensus.cc:2410] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:27:35.195118 12300 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:35.196475  5866 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:35.118533 12311 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:27:35.198608  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 66532.5us
I20260709 12:27:35.201822  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:35.128580 12169 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.206688  5865 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:35.203315  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.041419 12320 consensus_queue.cc:237] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:35.204064  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:35.196337 12077 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:35.126199 12030 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.126104 12166 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:35.135227 12071 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:35.232652  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:35.196520 12280 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.522s	user 0.015s	sys 0.002s
I20260709 12:27:35.234221  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:35.261049  5940 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:35.234656  5871 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:27:35.238013  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.263224  6756 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:35.128952 12175 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:35.267388 12160 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:35.267958 12160 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:35.268337 12160 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:35.228130 12120 consensus_queue.cc:1243] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:35.103256 12261 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.099s
I20260709 12:27:35.099073 12262 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:35.259212 12070 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:35.273845 11846 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.599s	user 0.013s	sys 0.000s
I20260709 12:27:35.274360  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:35.275004  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.275398 11879 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:35.275668  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:35.276059  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.229020 12174 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:35.256655  6766 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:27:35.105010 12137 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:35.239748 12134 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:35.118956 12334 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:35.277974  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:35.251665  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 175797us
I20260709 12:27:35.277459  6768 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:35.231415 12307 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.287464  6766 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:27:35.276760 12071 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:35.234401 12077 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:35.299019  6611 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:35.308554 12329 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:27:35.199914 12263 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:35.309441 12321 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:27:35.326802  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.253798 12311 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:35.283321 12281 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.259542 12166 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:35.308990 12332 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 12:27:35.321434  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:35.321671  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 203409us
I20260709 12:27:35.289021 12108 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.322052  6768 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 12:27:35.307642 12217 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.973s	user 0.010s	sys 0.005s
I20260709 12:27:35.242717 12169 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:35.252017 12030 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:35.253070 12294 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:27:35.430668  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:35.273900 12070 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:35.432966  5874 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:35.241057 12223 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:35.318665 12334 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:35.346439  6611 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:35.281353 12280 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:35.324712 12323 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:35.442767  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 241062us
I20260709 12:27:35.445708  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:35.479735  5866 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:35.449594  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:35.480556 12337 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 12:27:35.318809 12174 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.120s
I20260709 12:27:35.429363 12200 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.482147 12030 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:35.482358 12200 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:35.379715 12251 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:35.462021 12108 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:35.486131 12251 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:35.302461 12307 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:35.454695  5915 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:35.428503 11950 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
W20260709 12:27:35.472965  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 150109us
I20260709 12:27:35.473791 12261 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:35.446692 12311 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:35.485428 12169 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:35.489292  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 58756us
I20260709 12:27:35.507160  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:35.489171  6609 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:35.498629  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.485284 12200 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:35.502447  5840 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:35.509442  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 54846us
I20260709 12:27:35.465859 12232 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.510454  5915 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:35.482537 12336 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:27:35.509207  6756 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:35.512276  5840 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:35.510735  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.519126  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.520090  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.518273 12110 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:35.520720  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.454471 12281 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:35.488538 12323 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.526398  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.513945  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:35.532742  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1954) took 1224 ms. Trace:
I20260709 12:27:35.538105  5946 rpcz_store.cc:276] 0709 12:27:34.308505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.683240 (+374735us) service_pool.cc:224] Handling call
0709 12:27:34.683630 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:34.684045 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:34.684076 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:34.684562 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 12:27:34.817390 (+132828us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:34.817544 (+   154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.817618 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.532434 (+714816us) raft_consensus.cc:1685] finished
0709 12:27:35.532460 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.532657 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":317,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"thread_start_us":132371,"threads_started":1,"wal-append.queue_time_us":314818}
I20260709 12:27:35.499105 12325 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:35.507908 12311 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:35.507189 12261 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.555333  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:35.557760  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.562288  6757 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:35.528231 12169 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:35.478777 12222 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:35.561761  5946 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 21: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:27:35.541056 12339 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:27:35.527034 12248 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:35.541323  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.463445 12077 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:35.537722 12323 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:35.541085 12120 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.531741 12340 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:35.579445 12282 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:35.580407  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:35.558379 12261 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:35.590567  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 62900.2us
I20260709 12:27:35.540642 12175 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:35.591588  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1636) took 2016 ms. Trace:
I20260709 12:27:35.592341  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:35.527812 12232 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:35.597200  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:35.592057  5943 rpcz_store.cc:276] 0709 12:27:33.566828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.566986 (+   158us) service_pool.cc:224] Handling call
0709 12:27:35.582689 (+2015703us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54001bdfd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:35.582887 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1121615744}
I20260709 12:27:35.549523 12200 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:27:35.614161  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:35.577287 12108 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:35.645579  6762 raft_consensus.cc:2376] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18. Re-sending same reply.
I20260709 12:27:35.646687  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:35.591130 12340 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:35.662657  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:35.684032  5946 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
W20260709 12:27:35.644716 12315 consensus_meta.cc:287] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.744s	user 0.014s	sys 0.003s
I20260709 12:27:35.598184 12282 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:35.611757 12248 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:35.644141 12202 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:35.651804 12279 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.688184 12340 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:35.596328 12120 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:35.724766  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.722146  5946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7"
dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:48330
I20260709 12:27:35.652776 12344 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.747294  5946 raft_consensus.cc:606] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:35.749512  5946 raft_consensus.cc:614] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:35.689589 12263 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:35.723456 12251 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:35.730701 12282 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:35.764971  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:35.767891  6049 heartbeater.cc:646] Failed to heartbeat to 127.2.4.254:41049 (5 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.2.4.254:41049 timed out after 15.000s (SENT)
I20260709 12:27:35.768996 12349 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:35.662775 12334 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:35.747929 12166 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:35.738621 12202 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:35.759533 12338 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.297s	user 0.002s	sys 0.000s
I20260709 12:27:35.769593 12222 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:35.778717 12070 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:35.783095 12346 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.247s	user 0.002s	sys 0.000s
I20260709 12:27:35.789158  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:35.795924  6609 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:35.797987  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:35.797740 12263 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.781208 12281 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:35.706830 12315 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:35.821288  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:35.783987 12344 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:35.785455 12354 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:35.749760 12279 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:35.800463 12280 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:35.794337 12261 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:35.882041  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 60858.1us
I20260709 12:27:35.782878 12323 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:35.887398  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:35.890455  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:35.886648 12365 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:27:35.800141 12199 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:27:35.894728 12315 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:35.895583 12199 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:35.896030 12199 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:35.896286 12357 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:27:35.832288 12157 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.898069 12248 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:35.798014 11950 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:27:35.899647  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1213) took 2300 ms. Trace:
I20260709 12:27:35.945583  5874 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:35.898329 12157 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:35.896920 12199 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:35.892807  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:35.828197 12070 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:35.940862 12222 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:35.952219 12070 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:35.952752 12302 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 1.639s	user 0.000s	sys 0.003s
I20260709 12:27:35.953722  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:35.955549  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:35.956542 12244 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.872s	user 0.014s	sys 0.000s
I20260709 12:27:35.957278  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1544) took 2302 ms. Trace:
I20260709 12:27:35.887442 12307 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:35.957713  6020 rpcz_store.cc:276] 0709 12:27:33.654918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.655157 (+   239us) service_pool.cc:224] Handling call
0709 12:27:33.655600 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:33.688973 (+ 33373us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:33.717443 (+ 28470us) log.cc:844] Serialized 14 byte log entry
0709 12:27:34.015958 (+298515us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:34.016472 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:27:34.016636 (+   164us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:34.016679 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.016770 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.945903 (+1929133us) spinlock_profiling.cc:243] Waited 5.23 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:35.956901 (+ 10998us) raft_consensus.cc:1685] finished
0709 12:27:35.957039 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.957213 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35061,"prepare.queue_time_us":28606,"prepare.run_cpu_time_us":185,"prepare.run_wall_time_us":12551,"raft.queue_time_us":14568,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":21596,"spinlock_wait_cycles":12030720,"thread_start_us":225679,"threads_started":1,"wal-append.queue_time_us":374939}
I20260709 12:27:35.960009  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:35.961208  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:35.962044  6020 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:35.962947  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:35.900838 12166 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:35.964700  6020 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:35.965693  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:35.966801  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:35.967325  6020 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:35.956275  6018 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:35.968600  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:35.970206  5866 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:35.970824 12229 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:35.975503  5871 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:35.945881  5949 rpcz_store.cc:276] 0709 12:27:33.599305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.657421 (+ 58116us) service_pool.cc:224] Handling call
0709 12:27:33.657802 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:33.764293 (+106491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:33.764322 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:35.086782 (+1322460us) spinlock_profiling.cc:243] Waited 3.72 ms on lock 0x7b2c001b1e88. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:27:35.094211 (+  7429us) log.cc:844] Serialized 26 byte log entry
0709 12:27:35.300759 (+206548us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:35.300815 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.300919 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.781945 (+481026us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:35.899358 (+117413us) raft_consensus.cc:1685] finished
0709 12:27:35.899385 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.899552 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":245801,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":76756,"spinlock_wait_cycles":32839680,"thread_start_us":208503,"threads_started":2,"wal-append.queue_time_us":358715}
I20260709 12:27:35.980901 11854 raft_consensus.cc:3055] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:35.981413 11854 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:35.952786 12070 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:35.982860 12294 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.983477 12294 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:35.984429  5915 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.196:34835 timed out after 1.500s (SENT) [suppressed 12 similar messages]
I20260709 12:27:35.980139  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:35.986133  6018 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:35.947909  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 435733us
I20260709 12:27:35.830094 12347 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:35.994078  6766 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:35.888032 12281 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:35.995942  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 103088us
I20260709 12:27:35.994634  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1549) took 1546 ms. Trace:
I20260709 12:27:35.957093 12244 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:35.990757  5840 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:36.022188 12363 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:27:36.023243  6762 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:36.023630  5866 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
W20260709 12:27:36.025785 12368 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:27:36.026738 12027 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:36.026012 12258 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.940s	user 0.010s	sys 0.005s
I20260709 12:27:35.832444 12263 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:36.036118 12263 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:36.036284 12294 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:36.036837 12263 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:36.018895 12229 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:36.037869 12229 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.038316 12229 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:36.038868 12311 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:36.003805 12347 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.046626  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:35.990878 12092 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.006428  6024 rpcz_store.cc:276] 0709 12:27:34.447709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.553515 (+105806us) service_pool.cc:224] Handling call
0709 12:27:34.623748 (+ 70233us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:34.624339 (+   591us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:34.624390 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:34.963060 (+338670us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b2c001ddd28. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:27:34.963714 (+   654us) log.cc:844] Serialized 26 byte log entry
0709 12:27:35.264119 (+300405us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:35.264201 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.264301 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.994304 (+730003us) raft_consensus.cc:1685] finished
0709 12:27:35.994331 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.994572 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":850,"raft.queue_time_us":1318,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":8567,"spinlock_wait_cycles":9424128,"thread_start_us":279392,"threads_started":1,"wal-append.queue_time_us":409219}
I20260709 12:27:36.044642 12258 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:36.126825 12279 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:35.982585 11854 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:36.127522  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:36.128007 11854 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:36.128509 12258 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:36.131125 12157 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:36.023540 12232 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:36.063642  6023 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:36.030933 12199 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:36.135732 12344 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:36.135907 12294 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.136816 12294 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:36.019165 12166 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.020350 12281 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:36.061765 12347 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:36.072935  6018 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:35.950364 12202 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:35.951140 12354 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.083033  6020 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:36.151264  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:36.139421 12092 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:36.119071 12214 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.033s	user 0.007s	sys 0.007s
I20260709 12:27:36.152464  6773 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:36.153311 12354 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:36.153772 12174 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.154094  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:36.155272  5866 raft_consensus.cc:2410] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
W20260709 12:27:36.122236 12304 consensus_meta.cc:287] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.996s	user 0.016s	sys 0.000s
I20260709 12:27:36.019259 12280 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.103s
I20260709 12:27:36.122684 12244 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:36.125430 12160 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.857s	user 0.015s	sys 0.000s
I20260709 12:27:36.126221  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:36.137800  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1556) took 1053 ms. Trace:
I20260709 12:27:36.162539  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:36.162371  6015 rpcz_store.cc:276] 0709 12:27:35.084271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.136307 (+ 52036us) service_pool.cc:224] Handling call
0709 12:27:35.136792 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:35.498227 (+361435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:35.498266 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:35.498769 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 12:27:35.665077 (+166308us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:35.665138 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.782924 (+117786us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b540041da54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:35.782998 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:36.137392 (+354394us) raft_consensus.cc:1685] finished
0709 12:27:36.137416 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.137715 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14505,"raft.queue_time_us":14698,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":23260,"spinlock_wait_cycles":796510592,"thread_start_us":127774,"threads_started":1,"wal-append.queue_time_us":247638}
I20260709 12:27:36.168135  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:36.156123 12229 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:36.171955 12229 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:36.154304 12174 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:36.154754 12311 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:36.164973 12202 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:36.145053 12166 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:36.179441 12169 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.004s
W20260709 12:27:36.194878  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:36.195909  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.175889 12120 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:36.184150 12374 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.173681 12160 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:36.200640  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:36.205179  6018 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:36.204320  6024 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:36.202647 12374 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:36.215348 12374 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:36.162034 12304 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:36.224941  6757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1674) took 1516 ms (client timeout 1698 ms). Trace:
I20260709 12:27:36.198580 12169 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:36.225538  6757 rpcz_store.cc:269] 0709 12:27:34.708869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.709002 (+   133us) service_pool.cc:224] Handling call
0709 12:27:36.224674 (+1515672us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b5400008a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:36.224861 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1940559104}
W20260709 12:27:36.229454  5838 leader_election.cc:336] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Timed out: RequestConsensusVote RPC to 127.2.4.195:43329 timed out after 1.680s (SENT)
I20260709 12:27:36.232422  5838 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:36.233438  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:36.234622  6018 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:36.232633 12261 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:36.245882  5940 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 18: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:36.257755 12154 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.258466 12154 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:36.262650  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:36.263507  6018 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:36.238976 12298 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:27:36.268116  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.265024 12349 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:36.265444 12160 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:36.277240  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 247961us
I20260709 12:27:36.244133 12070 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:36.287294  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 302885us
I20260709 12:27:36.283565  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:36.299563  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:36.304837  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1547) took 1857 ms. Trace:
I20260709 12:27:36.305869  6018 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:36.307519 12362 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:36.307709 12280 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:36.308148 12362 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.308586 12362 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:36.309793  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:36.310596  6757 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:36.305344  6019 rpcz_store.cc:276] 0709 12:27:34.447160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.448626 (+  1466us) service_pool.cc:224] Handling call
0709 12:27:34.454663 (+  6037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.774187 (+319524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:34.774218 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:34.774259 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:34.774291 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.304579 (+1530288us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54001711d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:36.304630 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.304773 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":734335360}
I20260709 12:27:36.291404 12294 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:36.324266  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:36.325510  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:36.326267  6793 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:27:36.326735  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:36.328112  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:36.328563  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1488) took 1073 ms. Trace:
I20260709 12:27:36.299494 12281 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:36.330200  6018 rpcz_store.cc:276] 0709 12:27:35.255407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.325517 (+1070110us) service_pool.cc:224] Handling call
0709 12:27:36.328502 (+  2985us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.337275  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:36.336541  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:36.338788  6019 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:36.342813  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:36.323381 12277 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:27:36.346706  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:36.325697 12298 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:27:36.350540  6019 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:36.344594 12027 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 12:27:36.354789 12070 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:36.331514 12166 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:36.358505  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.369815 12369 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.370651 12369 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:36.367889 12278 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:36.378703  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1561) took 1293 ms. Trace:
I20260709 12:27:36.379698  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:36.385079  6018 raft_consensus.cc:2410] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:27:36.384563  6019 rpcz_store.cc:276] 0709 12:27:35.085587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.376677 (+1291090us) service_pool.cc:224] Handling call
0709 12:27:36.377603 (+   926us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.377902 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.377943 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:36.378008 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:36.378056 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.378135 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.378606 (+   471us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.385969  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1485) took 1131 ms. Trace:
I20260709 12:27:36.386381  6018 rpcz_store.cc:276] 0709 12:27:35.254552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.341401 (+1086849us) service_pool.cc:224] Handling call
0709 12:27:36.385895 (+ 44494us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.386430  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:36.345618 12370 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:36.395313  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:36.395998  5869 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:36.396741  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:36.397187  5869 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:36.403978  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:36.404863  5874 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:36.387661 12304 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:36.406899 11977 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:36.398959  6609 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:36.409415  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:36.416617  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:36.416833  6018 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:36.419149 12172 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:36.419723 12172 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.420087 12323 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:36.420143 12172 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:36.427574  6018 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:36.429494  6018 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:27:36.430058  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:36.434242  5874 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:36.435767  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:36.438470  6762 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:36.394611 12278 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:36.432974  5948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1950) took 2236 ms (client timeout 1680 ms). Trace:
I20260709 12:27:36.440830  5913 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:36.441426  6775 raft_consensus.cc:2410] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
W20260709 12:27:36.455444  5948 rpcz_store.cc:269] 0709 12:27:34.196994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.238686 (+ 41692us) service_pool.cc:224] Handling call
0709 12:27:36.432761 (+2194075us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54003098d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:36.432891 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240933760}
I20260709 12:27:36.437925 12359 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:36.430325  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1480) took 1182 ms. Trace:
I20260709 12:27:36.444962  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1557) took 1360 ms. Trace:
I20260709 12:27:36.458698  6018 rpcz_store.cc:276] 0709 12:27:35.248248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.428766 (+1180518us) service_pool.cc:224] Handling call
0709 12:27:36.429217 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:36.429997 (+   780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.430258 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.459148  6023 rpcz_store.cc:276] 0709 12:27:35.084520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.108776 (+ 24256us) service_pool.cc:224] Handling call
0709 12:27:36.444637 (+1335861us) spinlock_profiling.cc:243] Waited 953 ms on lock 0x7b54006b8fd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:36.444833 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2192350592}
I20260709 12:27:36.460991  6018 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:36.446185  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1649) took 1586 ms. Trace:
I20260709 12:27:36.461609  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1484) took 1212 ms. Trace:
I20260709 12:27:36.446349  5946 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:36.462183  6018 rpcz_store.cc:276] 0709 12:27:35.249537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.460298 (+1210761us) service_pool.cc:224] Handling call
0709 12:27:36.460686 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.461359 (+   673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.461554 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.445530 12354 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:36.461931  5940 rpcz_store.cc:276] 0709 12:27:34.859529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.275414 (+415885us) service_pool.cc:224] Handling call
0709 12:27:36.445949 (+1170535us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b54003089d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:36.446122 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2217434368}
I20260709 12:27:36.482359  5869 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:36.464306  6023 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:36.464565  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1960) took 2024 ms. Trace:
I20260709 12:27:36.486491  5948 rpcz_store.cc:276] 0709 12:27:34.439876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.463309 (+2023433us) service_pool.cc:224] Handling call
0709 12:27:36.463715 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.464032 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.464076 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:36.464150 (+    74us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:36.464208 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.464290 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.464452 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.488952  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1490) took 1233 ms. Trace:
I20260709 12:27:36.490942  6023 rpcz_store.cc:276] 0709 12:27:35.255983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.460914 (+1204931us) service_pool.cc:224] Handling call
0709 12:27:36.461402 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.488597 (+ 27195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.488889 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.493185  6023 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:27:36.458627 12359 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.398474 12370 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:36.496649  5948 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:36.498317  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1962) took 2027 ms. Trace:
I20260709 12:27:36.480547 12321 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:36.499603  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1961) took 2029 ms. Trace:
I20260709 12:27:36.498554  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:36.502238  5948 rpcz_store.cc:276] 0709 12:27:34.470878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.489098 (+2018220us) service_pool.cc:224] Handling call
0709 12:27:36.496470 (+  7372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.498042 (+  1572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.498256 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.502059  5940 rpcz_store.cc:276] 0709 12:27:34.470295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.486910 (+2016615us) service_pool.cc:224] Handling call
0709 12:27:36.499084 (+ 12174us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.499302 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.499327 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:36.499370 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:36.499399 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.499451 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.499546 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.508438  5947 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:36.500800 12359 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:36.511198  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1645) took 1683 ms. Trace:
I20260709 12:27:36.519608  5947 rpcz_store.cc:276] 0709 12:27:34.828094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.494873 (+1666779us) service_pool.cc:224] Handling call
0709 12:27:36.508084 (+ 13211us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.508855 (+   771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.511083 (+  2228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.476886 11977 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:27:36.517835 12321 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:36.523371 12321 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:36.496053 12173 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:36.501598 12174 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:36.530085 11977 consensus_queue.cc:1243] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:36.531673 12347 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:36.533085  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1650) took 1601 ms. Trace:
I20260709 12:27:36.533514  5948 rpcz_store.cc:276] 0709 12:27:34.932207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.503349 (+1571142us) service_pool.cc:224] Handling call
0709 12:27:36.518330 (+ 14981us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.532735 (+ 14405us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:36.532766 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:36.532807 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:36.532840 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.532902 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.533027 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":110,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103}
I20260709 12:27:36.535509 12276 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:36.536046  5940 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:36.536762  6762 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:36.537876 12347 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:36.538554 12137 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:36.539252  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:36.540230  5874 raft_consensus.cc:2410] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
W20260709 12:27:36.541160 12285 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 2.379s	user 0.014s	sys 0.000s
I20260709 12:27:36.541320  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:36.542770  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:36.543972  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:36.545064  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:36.547147  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1478) took 3264 ms. Trace:
I20260709 12:27:36.548061 12370 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:36.549191  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1536) took 1794 ms. Trace:
I20260709 12:27:36.548116  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 1434) took 1023 ms. Trace:
I20260709 12:27:36.549680  5867 rpcz_store.cc:276] 0709 12:27:34.754569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.781354 (+ 26785us) service_pool.cc:224] Handling call
0709 12:27:34.781761 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.001206 (+219445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.548928 (+1547722us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54002f62d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:36.549126 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":269974272}
I20260709 12:27:36.550243  6766 rpcz_store.cc:276] 0709 12:27:35.524960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.525831 (+   871us) service_pool.cc:224] Handling call
0709 12:27:35.526212 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.994656 (+468444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.547851 (+553195us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b5400003a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:36.548045 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1075390848}
I20260709 12:27:36.551484  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:36.547712  6017 rpcz_store.cc:276] 0709 12:27:33.282374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.282642 (+   268us) service_pool.cc:224] Handling call
0709 12:27:33.283227 (+   585us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:33.309957 (+ 26730us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:33.515009 (+205052us) log.cc:844] Serialized 14 byte log entry
0709 12:27:33.769214 (+254205us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:33.788956 (+ 19742us) log.cc:844] Serialized 26 byte log entry
0709 12:27:33.789137 (+   181us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:33.789368 (+   231us) log.cc:844] Serialized 14 byte log entry
0709 12:27:33.791279 (+  1911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.791389 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.030718 (+1239329us) spinlock_profiling.cc:243] Waited 4.37 ms on lock 0x7b18001dde20. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb96522 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:27:36.069097 (+1038379us) spinlock_profiling.cc:243] Waited 4.66 ms on lock 0x7b18001dde20. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:27:36.540305 (+471208us) spinlock_profiling.cc:243] Waited 6.16 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:36.546883 (+  6578us) raft_consensus.cc:1685] finished
0709 12:27:36.546913 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.547076 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37334,"prepare.queue_time_us":104581,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":88,"raft.queue_time_us":37798,"raft.run_cpu_time_us":525,"raft.run_wall_time_us":32913,"spinlock_wait_cycles":35740544,"thread_start_us":380591,"threads_started":2,"wal-append.queue_time_us":505738}
W20260709 12:27:36.554381 12282 consensus_meta.cc:287] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.746s	user 0.013s	sys 0.000s
I20260709 12:27:36.554916  6015 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: [17.8-17.8]
I20260709 12:27:36.556255  5915 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:36.580184  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.581049  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1507) took 1042 ms. Trace:
I20260709 12:27:36.581491  6015 rpcz_store.cc:276] 0709 12:27:35.539344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.554244 (+1014900us) service_pool.cc:224] Handling call
0709 12:27:36.554631 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:36.571452 (+ 16821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.580967 (+  9515us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:36.583221  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.556875  5867 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:36.499279 12369 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:36.545617  5874 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:36.560912  6017 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:36.601151  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:36.601228  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 45020.5us
I20260709 12:27:36.603941  5915 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:36.606992  6015 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:36.600131 12282 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:36.607820 12276 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:36.608388  5868 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:36.608451 12276 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.609200 12276 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:36.529103 12172 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:36.610227  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:36.611733 12172 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:36.613128  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1509) took 1048 ms. Trace:
I20260709 12:27:36.611765  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:36.613689  6017 rpcz_store.cc:276] 0709 12:27:35.565871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.560109 (+994238us) service_pool.cc:224] Handling call
0709 12:27:36.560610 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.612800 (+ 52190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.613062 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.614667  5867 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
W20260709 12:27:36.614961  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 103907us
I20260709 12:27:36.615736  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:36.616451  5874 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:36.619330  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:36.613722 12305 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.621939 12305 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:36.623449  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1953) took 2315 ms. Trace:
I20260709 12:27:36.624032  5944 rpcz_store.cc:276] 0709 12:27:34.308100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.682224 (+374124us) service_pool.cc:224] Handling call
0709 12:27:34.682626 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.745551 (+ 62925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.623042 (+1877491us) spinlock_profiling.cc:243] Waited 5.32 ms on lock 0x7b5400833954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:36.623350 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12229888}
I20260709 12:27:36.625391 12305 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:36.630283  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:36.632380  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:36.632964  6022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 1495) took 2292 ms (client timeout 1500 ms). Trace:
I20260709 12:27:36.633397  5867 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 25: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:36.643651  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1654) took 1084 ms. Trace:
I20260709 12:27:36.659471  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1218) took 2466 ms. Trace:
I20260709 12:27:36.662130  5947 rpcz_store.cc:276] 0709 12:27:35.559040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.591723 (+1032683us) service_pool.cc:224] Handling call
0709 12:27:36.642983 (+ 51260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.643224 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.643264 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:36.643318 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:36.643355 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.643415 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.643574 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:36.645120  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 1432) took 1125 ms. Trace:
I20260709 12:27:36.649216  5941 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:36.661105  6022 rpcz_store.cc:269] 0709 12:27:34.342365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.356129 (+ 13764us) service_pool.cc:224] Handling call
0709 12:27:36.632611 (+2276482us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b5400562554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:36.632807 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":608413056}
I20260709 12:27:36.653415  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:36.665059  6756 rpcz_store.cc:276] 0709 12:27:35.519580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.520316 (+   736us) service_pool.cc:224] Handling call
0709 12:27:36.644340 (+1124024us) spinlock_profiling.cc:243] Waited 8.29 ms on lock 0x7b580001f898. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5eeee 000000000036eb39 00007f39d7bd21e0 00007f39d7c9b0c4 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:27:36.644528 (+   188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.644734 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.644795 (+    61us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:36.644844 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:36.644878 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.644937 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.645050 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19056896}
I20260709 12:27:36.668272  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:36.662416  5942 rpcz_store.cc:276] 0709 12:27:34.193141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.679311 (+486170us) service_pool.cc:224] Handling call
0709 12:27:34.679767 (+   456us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:34.834927 (+155160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:34.834963 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:34.858400 (+ 23437us) log.cc:844] Serialized 52 byte log entry
0709 12:27:34.969598 (+111198us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b1800384340. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:27:35.094252 (+124654us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:35.474172 (+379920us) log.cc:844] Serialized 14 byte log entry
0709 12:27:35.511627 (+ 37455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.658904 (+1147277us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b540081b854. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:36.659004 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:36.659086 (+    82us) raft_consensus.cc:1685] finished
0709 12:27:36.659107 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.659379 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29672,"prepare.queue_time_us":489942,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":68742,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":25157,"spinlock_wait_cycles":355808000,"thread_start_us":487855,"threads_started":2,"wal-append.queue_time_us":397327}
W20260709 12:27:36.676043  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56820.4us
I20260709 12:27:36.682229  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:36.768610  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1563) took 1244 ms. Trace:
I20260709 12:27:36.728497  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:36.769290  6018 rpcz_store.cc:276] 0709 12:27:35.523943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.463644 (+939701us) service_pool.cc:224] Handling call
0709 12:27:36.487076 (+ 23432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.487359 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:36.490086 (+  2727us) log.cc:844] Serialized 14 byte log entry
0709 12:27:36.768086 (+278000us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:36.768171 (+    85us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:36.768235 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.768337 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.768523 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12475,"prepare.queue_time_us":4805,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"thread_start_us":185361,"threads_started":1}
I20260709 12:27:36.775122 12158 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:36.779402 12158 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.779938 12158 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:36.776697  5941 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:36.777285  6609 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:36.645723 12340 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.886s	user 0.014s	sys 0.002s
I20260709 12:27:36.636597 12311 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:36.615024 12282 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:36.638679 12092 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:36.780540 12334 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:36.788064 12347 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:36.728652 11940 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:36.767498 12111 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:36.775561 12153 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:36.821975  5874 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:36.791033 12110 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.780593 12202 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:27:36.843781  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 72310.2us
I20260709 12:27:36.796059 12311 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:36.846912  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.811645 12160 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:36.785980 12280 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:27:36.794243 12340 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:36.813090 12111 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:36.791913 12294 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
W20260709 12:27:36.856395  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.815253 12334 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:36.815984 12092 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
W20260709 12:27:36.870694 12362 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.562s	user 0.012s	sys 0.001s
I20260709 12:27:36.875286 12362 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:36.846319 12153 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:36.876807  5869 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:36.854642 12160 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.877504 12160 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:36.876485 12153 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:36.883252 12092 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.883912 12092 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:36.884776 12153 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:36.862391 12111 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:36.895350 12362 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:36.815613 12347 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:36.843885 11940 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:36.875411 12334 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:36.892890 12160 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:36.913558  5949 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:36.852622 12311 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:36.848821 12110 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:36.851645 12202 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:36.932927 11846 raft_consensus.cc:3055] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:36.933581 11846 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:36.934851 11846 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:36.936733 11846 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:36.939677  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1560) took 1854 ms. Trace:
I20260709 12:27:36.942935  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:36.944674 12173 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.945255 12173 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:36.947575 12173 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:36.948401 12221 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:36.948782 12221 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.949169 12221 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:36.951391  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1958) took 2641 ms. Trace:
I20260709 12:27:36.952041  5946 rpcz_store.cc:276] 0709 12:27:34.309821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.828772 (+1518951us) service_pool.cc:224] Handling call
0709 12:27:36.446122 (+617350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.472821 (+ 26699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.950993 (+478172us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b54003089d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:36.951273 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1410172288}
I20260709 12:27:36.940256  6021 rpcz_store.cc:276] 0709 12:27:35.085307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.119874 (+ 34567us) service_pool.cc:224] Handling call
0709 12:27:35.120195 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.471737 (+351542us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:35.647627 (+175890us) log.cc:844] Serialized 14 byte log entry
0709 12:27:35.814315 (+166688us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:35.815709 (+  1394us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:35.815780 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.939321 (+1123541us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b5400364cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:36.939389 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.939585 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":273876,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":68,"raft.queue_time_us":10530,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":8903,"spinlock_wait_cycles":807648512,"thread_start_us":303623,"threads_started":2,"wal-append.queue_time_us":291579}
I20260709 12:27:36.955046 11940 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:36.956940 12303 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:36.958031 12092 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:36.941319  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1959) took 2501 ms. Trace:
I20260709 12:27:36.969090  5949 rpcz_store.cc:276] 0709 12:27:34.439514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.981379 (+1541865us) service_pool.cc:224] Handling call
0709 12:27:35.981798 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.917685 (+935887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.940879 (+ 23194us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b5400674654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:36.941214 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2142086016}
I20260709 12:27:36.961331 12221 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:36.963969 12327 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.965358  6020 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:36.977133 11949 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:36.980409 11949 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:36.980693 12354 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:36.973939 12136 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.982125 12347 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:36.981966 12136 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:36.982483 11949 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:36.980436 12385 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:27:36.984472 12136 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:36.984836 12359 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:36.974838 12198 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:36.986753  6611 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:36.993245  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.974249 12092 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.975528 12278 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.995169 12307 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:36.976950 12327 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:36.997879 12199 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:27:36.997192  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 201816us
I20260709 12:27:37.042363  6768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:37.043397  6768 raft_consensus.cc:606] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:37.043574 12340 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:37.046034 12359 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:37.050071 12221 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:37.000089 12092 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:37.031868  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:37.068408  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:37.004854 12278 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:37.077584 12387 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:27:37.078234 12278 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:37.080824  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1506) took 1555 ms. Trace:
I20260709 12:27:37.081446  6023 rpcz_store.cc:276] 0709 12:27:35.529839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.495210 (+965371us) service_pool.cc:224] Handling call
0709 12:27:36.495581 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:36.496802 (+  1221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.496835 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:36.497339 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 12:27:36.824938 (+327599us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:36.855058 (+ 30120us) log.cc:844] Serialized 14 byte log entry
0709 12:27:36.855378 (+   320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.855492 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.046995 (+191503us) spinlock_profiling.cc:243] Waited 1.25 ms on lock 0x7b18000f4c00. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:27:37.080420 (+ 33425us) raft_consensus.cc:1685] finished
0709 12:27:37.080554 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.080742 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18947,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":1947,"raft.queue_time_us":2598,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":6455,"spinlock_wait_cycles":2863872,"thread_start_us":279091,"threads_started":1,"wal-append.queue_time_us":399571}
I20260709 12:27:37.083021 12092 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:37.074754 12202 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:37.074546  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:37.034511 12307 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:37.075507  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:37.084667  5941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1631) took 3970 ms (client timeout 1963 ms). Trace:
W20260709 12:27:37.093428  5941 rpcz_store.cc:269] 0709 12:27:33.113941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.114274 (+   333us) service_pool.cc:224] Handling call
0709 12:27:37.084379 (+3970105us) spinlock_profiling.cc:243] Waited 3.43 s on lock 0x7b5400672854. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:37.084553 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7886071552}
I20260709 12:27:37.094106 12223 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:37.081322 12359 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:37.097900 12359 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:37.072479  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.103017 12154 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:37.104621 12265 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:37.088224 12110 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:37.142062  5946 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:37.086916 12160 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:37.110780  6773 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1251) took 2054 ms (client timeout 1878 ms). Trace:
W20260709 12:27:37.109233  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:37.119899  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.117023  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.148744  6017 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:37.150362  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.123798  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:37.153412  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:37.154353  5941 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:37.155520  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:37.156505  5941 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:37.140589  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:37.157596  5949 raft_consensus.cc:2410] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:27:37.140761  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:37.159010  5947 raft_consensus.cc:2410] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:27:37.160408  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1197) took 5095 ms. Trace:
I20260709 12:27:37.160781  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:37.145005  6773 rpcz_store.cc:269] 0709 12:27:35.056130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.059785 (+  3655us) service_pool.cc:224] Handling call
0709 12:27:37.110497 (+2050712us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540004d8d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:37.110670 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2509966080}
I20260709 12:27:37.164562  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:37.161123  5945 rpcz_store.cc:276] 0709 12:27:32.045899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.055408 (+  9509us) service_pool.cc:224] Handling call
0709 12:27:32.508357 (+452949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.200556 (+1692199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.900744 (+700188us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b5400308ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:37.141029 (+2240285us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b5400308ed0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:37.141203 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4927455360}
I20260709 12:27:37.166285  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.166548  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.167078  6017 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:37.168265  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:37.168907  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:37.169806  5948 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:27:37.171701  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:37.172519  6022 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:37.167153  6018 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:27:37.180473  6021 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:37.181618  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:37.107043 12153 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:27:37.179960  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 106093us
W20260709 12:27:37.195910  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.107383 12243 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.072119 12340 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:37.207988  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 265136us
I20260709 12:27:37.209146  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:37.105577 12265 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.090s
I20260709 12:27:37.105804 12369 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 12:27:37.196691  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:37.197062  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.201606  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:37.205067 12390 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:27:37.218770  6018 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:37.209280  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:37.238795  6021 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 22: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:37.223124  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:37.224300  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.229347  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:37.231750  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:37.099226 12359 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:37.193156 12153 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:37.255280  6611 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:37.245900  5947 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:37.249178  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:37.245350  6017 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:37.255427  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:37.260615  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:37.244438  5990 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:37.192776 12310 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:27:37.215283 12227 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:37.298504  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:37.306469 12153 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:37.307775 12227 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:37.277391 12276 consensus_meta.cc:287] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.668s	user 0.009s	sys 0.005s
I20260709 12:27:37.302850  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:37.308707 12227 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:37.306028  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1304) took 3254 ms. Trace:
I20260709 12:27:37.309648  5873 rpcz_store.cc:276] 0709 12:27:34.051837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.052563 (+   726us) service_pool.cc:224] Handling call
0709 12:27:34.053090 (+   527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:34.294271 (+241181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:34.294322 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:35.083287 (+788965us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b480000cfd8. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d7bacaa1 00007f39d7ba3b37 00007f39d7ba39ca 00007f39d7c9b26f 00007f39d7c9af37 00007f39d7caf55c 00007f39d7caf508 00007f39d7caf488 00007f39d7caf443 00007f39d7cae6ec 00007f39d6bd8a96
0709 12:27:36.640276 (+1556989us) spinlock_profiling.cc:243] Waited 9.63 ms on lock 0x7b2c00198418. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:27:36.641031 (+   755us) log.cc:844] Serialized 26 byte log entry
0709 12:27:36.994978 (+353947us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:36.995020 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.005707 (+ 10687us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.305696 (+299989us) raft_consensus.cc:1685] finished
0709 12:27:37.305724 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.305923 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112199,"raft.run_cpu_time_us":3541,"raft.run_wall_time_us":906209,"spinlock_wait_cycles":45558784,"thread_start_us":593915,"threads_started":2,"wal-append.queue_time_us":654695}
I20260709 12:27:37.311848  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:37.312455  5873 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:37.290632 12347 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:37.215461 12340 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:37.313575 12347 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.313994 12347 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:37.315702 12340 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:37.309000  5946 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:37.277249 12221 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:37.317762 12221 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:37.317873  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:37.319633  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:37.319954 12221 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:37.321524 12311 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:37.324813 12327 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:37.290956 12369 consensus_queue.cc:1243] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:27:37.308612 12276 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:37.330428 12158 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.550s	user 0.015s	sys 0.000s
W20260709 12:27:37.330881  6016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1553) took 2248 ms (client timeout 1675 ms). Trace:
I20260709 12:27:37.207708 12243 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:37.358793  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:37.331211  6015 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 12:27:37.363967  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:37.365810  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:37.366248  5872 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:37.367237  5873 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:37.370067  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:37.370937  5873 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:37.331387  6016 rpcz_store.cc:269] 0709 12:27:35.083063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.095229 (+ 12166us) service_pool.cc:224] Handling call
0709 12:27:37.330580 (+2235351us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b5400661cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:37.330797 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4831192832}
I20260709 12:27:37.358853  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1956) took 3049 ms. Trace:
I20260709 12:27:37.336382 12310 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:37.377815 12310 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:37.378527  6773 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:37.379693  6778 raft_consensus.cc:2393] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 21: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:37.377245 12262 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:37.387569  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:37.387902  6020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 1501) took 1870 ms (client timeout 2078 ms). Trace:
I20260709 12:27:37.377820  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:37.388537  6020 rpcz_store.cc:269] 0709 12:27:35.517710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.150974 (+633264us) service_pool.cc:224] Handling call
0709 12:27:37.387664 (+1236690us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b54005659d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:37.387821 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1870433024}
I20260709 12:27:37.389158  6016 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:37.307909 12153 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:37.376502  5943 rpcz_store.cc:276] 0709 12:27:34.309125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.556466 (+2247341us) service_pool.cc:224] Handling call
0709 12:27:36.556926 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:36.557383 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.557416 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:36.558523 (+  1107us) log.cc:844] Serialized 26 byte log entry
0709 12:27:36.912915 (+354392us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:36.916151 (+  3236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.916266 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.327253 (+410987us) raft_consensus.cc:1685] finished
0709 12:27:37.327276 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.358768 (+ 31492us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22769,"raft.queue_time_us":1643,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":9146,"thread_start_us":236222,"threads_started":1,"wal-append.queue_time_us":420984}
I20260709 12:27:37.394682  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:37.402937  6022 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:37.403259  6773 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 18: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:37.395052  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:37.396473  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:37.403990  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:37.398548  5874 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.023s	sys 0.050s
I20260709 12:27:37.399354  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.330966 12158 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:37.415562  5874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.023s	sys 0.050s
I20260709 12:27:37.418711  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:37.419524  6020 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:27:37.421701  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1319) took 1185 ms. Trace:
W20260709 12:27:37.417405  6785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1669) took 2685 ms (client timeout 1963 ms). Trace:
I20260709 12:27:37.400485  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:37.429786  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:37.422180  5874 rpcz_store.cc:276] 0709 12:27:36.236739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.846206 (+609467us) service_pool.cc:224] Handling call
0709 12:27:36.846895 (+   689us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.847153 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:27:36.861986 (+ 14833us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.421186 (+559200us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.421263 (+    77us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:37.421341 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.421433 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.421601 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24663,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"thread_start_us":555873,"threads_started":1}
I20260709 12:27:37.432196  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.434356  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:37.436131  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:37.436791  6778 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:37.437904  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.442751  5871 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 21: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:37.417439  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:37.443949  5872 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
W20260709 12:27:37.425347  6785 rpcz_store.cc:269] 0709 12:27:34.705074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.726335 (+ 21261us) service_pool.cc:224] Handling call
0709 12:27:37.390471 (+2664136us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54002a17d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:37.390628 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294737664}
I20260709 12:27:37.362613 12077 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:37.271338 12071 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.128s
I20260709 12:27:37.468945  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:37.433007  6017 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:37.471138  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.471195  5873 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:37.459072  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.459942  5943 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:37.462085  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.473727  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:37.474756  5874 raft_consensus.cc:2393] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:37.473968  6022 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 18: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:37.476648  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.483420  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.485872  6018 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
I20260709 12:27:37.484947  6793 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:27:37.236322 12111 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:37.489075  6609 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:37.491078  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:37.491818  5872 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:37.492022  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:37.493669  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:37.497172  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:37.417133 12300 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.502678  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:37.416779 12294 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:37.503664  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:37.512198  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.505978  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:37.508643  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:37.428275 12070 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:37.510852 12310 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:37.527215 12310 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:37.468694 12298 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:37.534188 12298 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:37.539417 12158 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:37.541587 12111 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:37.545279  5943 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:37.518828  5874 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:37.546751  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2073) took 1115 ms. Trace:
I20260709 12:27:37.547079  5874 rpcz_store.cc:276] 0709 12:27:36.430858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.517763 (+1086905us) service_pool.cc:224] Handling call
0709 12:27:37.518347 (+   584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.546478 (+ 28131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.546695 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:37.548167  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:37.520426  6022 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 18: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:37.524480  6757 raft_consensus.cc:2393] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 20: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:37.550153  6778 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:37.467834 12311 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:37.503786 12202 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:37.560622  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:37.552865  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:37.585932  5874 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:37.586439  6793 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 18: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:27:37.561374  6608 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.023s	sys 0.018s
I20260709 12:27:37.561781  5872 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
W20260709 12:27:37.587181  6608 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.024s	sys 0.018s
I20260709 12:27:37.563874  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:37.470002 12221 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:37.580080  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:37.582504  6785 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:37.589483  6757 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:37.598802  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:37.599828  5949 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:37.602443  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.604756  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:37.605398  6778 raft_consensus.cc:2376] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18. Re-sending same reply.
I20260709 12:27:37.606544  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:37.621268  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.625278  5990 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:37.605115  5840 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:37.630438  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:37.633908  6020 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:37.638509  5941 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:27:37.591682  6016 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:37.578630  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.642644  6022 raft_consensus.cc:2410] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:27:37.647117  5945 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:27:37.700757  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.701555  6016 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:37.638628  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.707226  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:37.710175  5874 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:37.597781 12262 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:37.715335  5941 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:27:37.593693 12221 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:37.716910  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2061) took 1924 ms. Trace:
W20260709 12:27:37.717969  6024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1565) took 2078 ms (client timeout 1860 ms). Trace:
I20260709 12:27:37.720866  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:37.720358  6024 rpcz_store.cc:269] 0709 12:27:35.640032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.125900 (+485868us) service_pool.cc:224] Handling call
0709 12:27:37.717719 (+1591819us) spinlock_profiling.cc:243] Waited 42 ms on lock 0x7b54001716d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:37.717875 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96535936}
I20260709 12:27:37.722260  6016 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:37.722960  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:37.723706  5941 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:37.724838  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:37.717967  6020 raft_consensus.cc:2376] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18. Re-sending same reply.
W20260709 12:27:37.659756  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 54731.7us
I20260709 12:27:37.723748  6022 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:37.729974  6757 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:37.720101  5868 rpcz_store.cc:276] 0709 12:27:35.792803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.793099 (+   296us) service_pool.cc:224] Handling call
0709 12:27:35.793598 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.609019 (+815421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.716542 (+1107523us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b5400026f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:37.716817 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1873268992}
W20260709 12:27:37.639065 12334 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.018s	sys 0.014s
I20260709 12:27:37.732839  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:37.739130 12265 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:37.740353 12265 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:37.684118  5949 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 25.
W20260709 12:27:37.639429 12199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.021s	sys 0.012s
I20260709 12:27:37.648603 12202 raft_consensus.cc:697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:37.743631  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:37.747798  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:37.552330 12300 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:37.751411  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:37.753008  6024 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:37.707830 12295 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.708078 12263 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:27:37.760073  5869 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.016s	sys 0.049s
I20260709 12:27:37.763563  6018 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
W20260709 12:27:37.818610  5869 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.016s	sys 0.049s
I20260709 12:27:37.702049 12398 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:37.811652  5945 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:37.812393  5949 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:27:37.740607 12369 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
W20260709 12:27:37.812784  5866 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.056s	sys 0.093s
W20260709 12:27:37.747301 12199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.022s	sys 0.012s
I20260709 12:27:37.821691  5946 raft_consensus.cc:2410] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:27:37.754967 12154 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:37.607758 11835 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.201s
W20260709 12:27:37.755342 12305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.025s	sys 0.039s
W20260709 12:27:37.825592  5866 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.056s	sys 0.093s
I20260709 12:27:37.647379 12160 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:37.817391  5944 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.029s	sys 0.053s
W20260709 12:27:37.872273 12223 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.777s	user 0.013s	sys 0.001s
W20260709 12:27:37.874934  5944 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.030s	sys 0.053s
I20260709 12:27:37.875655  6017 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
W20260709 12:27:37.833446 12389 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.060s	user 0.000s	sys 0.003s
I20260709 12:27:37.826879 12154 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:37.809883 12316 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:37.735414 12262 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:37.942080  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:37.863170  5987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.042s	sys 0.053s
I20260709 12:27:37.876132  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:37.860900 12198 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.012s	sys 0.062s
I20260709 12:27:37.825248 12199 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:37.867302 12303 consensus_meta.cc:287] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.895s	user 0.012s	sys 0.001s
W20260709 12:27:37.904435  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.019s	sys 0.007s
W20260709 12:27:37.952857  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.020s	sys 0.008s
W20260709 12:27:37.953342  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:37.953617  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 730535us
I20260709 12:27:37.954710  5915 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:37.788873 12334 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.018s	sys 0.014s
I20260709 12:27:37.807250 12362 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:27:37.832310 12263 raft_consensus.cc:697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:37.854537 12305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.025s	sys 0.040s
W20260709 12:27:37.946607  5987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.043s	sys 0.053s
W20260709 12:27:38.024132  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 727053us
I20260709 12:27:37.871706 12398 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:27:38.022965  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68374.2us
I20260709 12:27:38.029742  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:38.033133  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.879690 12173 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.038028  5865 raft_consensus.cc:2410] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 12:27:38.040169  5947 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.773s	user 0.013s	sys 0.000s
I20260709 12:27:38.040796  5947 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
W20260709 12:27:37.948122 12198 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.012s	sys 0.062s
I20260709 12:27:37.822994 12295 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:38.050520  5915 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:38.058822  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:38.061909  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:38.068536 12154 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:38.068969 12263 consensus_queue.cc:237] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:38.070334 12399 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.438s	user 0.001s	sys 0.001s
I20260709 12:27:38.071099  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:38.071652 12397 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.341s	user 0.001s	sys 0.000s
I20260709 12:27:38.075804  6609 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:38.089586  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:37.935907 12160 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:38.079082 12334 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:38.111866  5874 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:38.079653  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:38.080955  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1510) took 2096 ms. Trace:
I20260709 12:27:38.114740  6015 rpcz_store.cc:276] 0709 12:27:35.984370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.609986 (+625616us) service_pool.cc:224] Handling call
0709 12:27:37.330814 (+720828us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:37.379932 (+ 49118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.080669 (+700737us) spinlock_profiling.cc:243] Waited 720 ms on lock 0x7b5400661cd0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:38.080873 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1656814464}
I20260709 12:27:38.023928 12243 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:38.133517  5874 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21. Ops: [18.10-20.11]
I20260709 12:27:38.134855  6023 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:38.050815 12405 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.034948 12278 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.038815 12117 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:38.061348 12403 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 12:27:38.043187 12315 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:38.086445  6793 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:38.071198 12396 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 12:27:38.103745 11949 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:38.148360  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:37.942803 12202 consensus_queue.cc:237] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:38.087021  5915 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:38.108631  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:38.060544 12395 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:38.134730  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:38.104749 12259 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:38.134677  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:38.049742 12173 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:38.145533  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1577) took 1023 ms. Trace:
I20260709 12:27:38.052371 12400 consensus_queue.cc:1059] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:27:38.146445  6018 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:38.085744  5947 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:38.135722  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1318) took 1899 ms. Trace:
I20260709 12:27:38.173847  5867 rpcz_store.cc:276] 0709 12:27:36.236460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.761276 (+524816us) service_pool.cc:224] Handling call
0709 12:27:36.761903 (+   627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.762295 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:36.762818 (+   523us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.762528 (+999710us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b180006d5a0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:27:38.135286 (+372758us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:38.135358 (+    72us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:38.135429 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.135515 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.135656 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23646,"prepare.queue_time_us":185,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":23767,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":13123,"spinlock_wait_cycles":26802944,"thread_start_us":348805,"threads_started":1}
I20260709 12:27:38.151769  5874 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:38.176805  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:38.105124 12311 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:38.160928  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:38.162410 12280 raft_consensus.cc:3055] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:38.170220  6023 rpcz_store.cc:276] 0709 12:27:37.122417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.079334 (+956917us) service_pool.cc:224] Handling call
0709 12:27:38.145457 (+ 66123us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:38.171852  5871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.029s	sys 0.079s
W20260709 12:27:38.147496 12404 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260709 12:27:38.175526  6766 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.023s	sys 0.074s
I20260709 12:27:38.177148  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:38.157747  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 70818.4us
I20260709 12:27:38.143599 12154 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:38.179061  6019 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.034s	sys 0.064s
W20260709 12:27:38.186203  5871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.029s	sys 0.079s
W20260709 12:27:38.199499  6019 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.035s	sys 0.064s
I20260709 12:27:38.156960 12221 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:38.187130  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1579) took 1045 ms. Trace:
I20260709 12:27:38.178571 11949 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:38.188931  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:38.192775  6766 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.024s	sys 0.074s
I20260709 12:27:38.200806  6018 rpcz_store.cc:276] 0709 12:27:37.142203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.096918 (+954715us) service_pool.cc:224] Handling call
0709 12:27:38.146250 (+ 49332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.186747 (+ 40497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.187058 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:38.192588  5915 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:38.190425 12411 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:27:38.206202 12160 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:38.207074  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1326) took 1931 ms. Trace:
I20260709 12:27:38.207544  5866 rpcz_store.cc:276] 0709 12:27:36.275947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.091448 (+815501us) service_pool.cc:224] Handling call
0709 12:27:37.091923 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:37.145450 (+ 53527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.145490 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:37.160417 (+ 14927us) log.cc:844] Serialized 26 byte log entry
0709 12:27:37.854966 (+694549us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:37.855033 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.855139 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:38.206854 (+351715us) raft_consensus.cc:1685] finished
0709 12:27:38.206875 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.207010 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63804,"raft.queue_time_us":53189,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":17131,"thread_start_us":663307,"threads_started":1,"wal-append.queue_time_us":869654}
I20260709 12:27:38.208344 11949 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:38.205214 12409 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:27:38.205500  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:38.200098  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1329) took 1907 ms. Trace:
I20260709 12:27:38.231185  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1325) took 1959 ms. Trace:
I20260709 12:27:38.232106 12414 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:27:38.232949 12347 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.918s	user 0.011s	sys 0.000s
W20260709 12:27:38.189136 12406 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
W20260709 12:27:38.187314 11846 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.246s	user 0.017s	sys 0.005s
I20260709 12:27:38.210811  6018 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:38.209344 12315 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:38.149176 12405 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:38.234493 11846 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:38.236858 12405 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:38.194615  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.233289 12414 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:27:38.231470  5871 rpcz_store.cc:276] 0709 12:27:36.293050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.499525 (+1206475us) service_pool.cc:224] Handling call
0709 12:27:37.506581 (+  7056us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.506831 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:37.507119 (+   288us) log.cc:844] Serialized 14 byte log entry
0709 12:27:38.199716 (+692597us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:38.199781 (+    65us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:38.199828 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.199905 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.200038 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":597,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":30975,"raft.queue_time_us":988,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":106,"thread_start_us":692213,"threads_started":1}
I20260709 12:27:38.233415  5872 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:38.172931 12243 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:38.246280 12243 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.224407  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2058) took 3356 ms. Trace:
I20260709 12:27:38.231895  5869 rpcz_store.cc:276] 0709 12:27:36.272097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.051636 (+779539us) service_pool.cc:224] Handling call
0709 12:27:37.052077 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:37.060143 (+  8066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.060186 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:37.065896 (+  5710us) log.cc:844] Serialized 26 byte log entry
0709 12:27:37.819253 (+753357us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:37.819321 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.819429 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:38.230832 (+411403us) raft_consensus.cc:1685] finished
0709 12:27:38.230869 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.231096 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24340,"raft.queue_time_us":7343,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":20326,"thread_start_us":752911,"threads_started":1,"wal-append.queue_time_us":957609}
I20260709 12:27:38.249720  5870 rpcz_store.cc:276] 0709 12:27:34.868353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.868548 (+   195us) service_pool.cc:224] Handling call
0709 12:27:34.889455 (+ 20907us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:34.893769 (+  4314us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:27:36.635831 (+1742062us) spinlock_profiling.cc:243] Waited 7.08 ms on lock 0x7b18003f01c0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d667b3c7 00007f39d2c8d9a7 00007f39d2c9d0bf 00007f39d2c9d038
0709 12:27:36.908195 (+272364us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.461063 (+552868us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:37.461799 (+   736us) log.cc:844] Serialized 26 byte log entry
0709 12:27:37.461974 (+   175us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:37.462007 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.462088 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:38.223946 (+761858us) raft_consensus.cc:1685] finished
0709 12:27:38.224084 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.224300 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":647008,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":53,"raft.queue_time_us":10020,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":56589,"spinlock_wait_cycles":16278784,"thread_start_us":763452,"threads_started":2,"wal-append.queue_time_us":791457}
I20260709 12:27:38.179291 12311 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.259869  6024 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:38.268471  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1583) took 1048 ms. Trace:
I20260709 12:27:38.269030  6024 rpcz_store.cc:276] 0709 12:27:37.220496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.724672 (+504176us) service_pool.cc:224] Handling call
0709 12:27:38.268363 (+543691us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:38.186151 12259 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:38.148667 12117 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:38.186539 12307 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:38.151268 12278 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:38.185429 12280 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:38.265486 12077 consensus_meta.cc:287] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.783s	user 0.012s	sys 0.000s
I20260709 12:27:38.148290 12199 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:38.246954 12243 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:38.271479  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:38.271684 12311 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:38.282558  5949 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:38.284600  5947 raft_consensus.cc:2376] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 25. Re-sending same reply.
I20260709 12:27:38.287222 12262 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:38.289166  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.289145  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:38.289510  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:38.302908  6785 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:38.272982 12300 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:38.308519  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:38.306129  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:38.313033  6775 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 1442) took 1986 ms (client timeout 2535 ms). Trace:
W20260709 12:27:38.316725  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 287896us
I20260709 12:27:38.319694  5869 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:27:38.313720  6775 rpcz_store.cc:269] 0709 12:27:36.326451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.326731 (+   280us) service_pool.cc:224] Handling call
0709 12:27:38.312716 (+1985985us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b540015efd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:38.312938 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236127488}
I20260709 12:27:38.254673  5942 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:38.233379 12347 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:38.296737 12295 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:38.327085  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:38.304572 12391 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.296010 12278 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:38.331004  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:38.350234  5840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.032s	sys 0.099s
W20260709 12:27:38.350648  5840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.032s	sys 0.099s
I20260709 12:27:38.351063  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:38.351270  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 603567us
I20260709 12:27:38.356469  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:38.361517  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:38.366406  6773 raft_consensus.cc:2368] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22.
I20260709 12:27:38.367231  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1228) took 1239 ms. Trace:
I20260709 12:27:38.370952  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:38.372444  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:38.257895 12398 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:38.380028  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:27:38.358135 12153 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.942s	user 0.012s	sys 0.004s
I20260709 12:27:38.349881 12413 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:38.377759 11977 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:38.381696 12223 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:38.421679  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:38.381942 12305 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:38.385485 12071 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:38.481979  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:38.423596  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:38.426828  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.366447 12262 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:38.384583 12307 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:27:38.492411 12423 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 12:27:38.387621 12422 raft_consensus.cc:2764] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:27:38.482331 12413 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:38.331763 12259 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:38.342303 12391 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:38.494570  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55542 (request call id 53) took 1456 ms. Trace:
I20260709 12:27:38.496274  6768 rpcz_store.cc:276] 0709 12:27:37.029828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.030030 (+   202us) service_pool.cc:224] Handling call
0709 12:27:38.486183 (+1456153us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b54001a07a0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c00520 00007f39d2c7d652 00007f39d7c4b60a 00007f39d11d17d4 00007f39d11d1762 00007f39d11d1691 00007f39d11d160c 00007f39d11d08c2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:38.486500 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34545536}
W20260709 12:27:38.490908  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 67430us
I20260709 12:27:38.499136  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:38.486661 12153 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:38.386708  6785 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:38.482331 12071 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:38.496107 12310 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.968s	user 0.017s	sys 0.000s
W20260709 12:27:38.382547 12416 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:27:38.511089  6793 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:38.483741 12137 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:38.514101  5940 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.000s	sys 0.043s
I20260709 12:27:38.516175  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:38.328414 12280 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:38.554388  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 323217us
I20260709 12:27:38.521184  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:38.555610  6785 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:38.555858  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:38.530109  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.488312 12262 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:38.495023 12290 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:38.537271  5947 raft_consensus.cc:1217] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Deduplicated request from leader. Original: 12.7->[15.8-17.9]   Dedup: 15.8->[17.9-17.9]
I20260709 12:27:38.538698  5946 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:27:38.559608  5947 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 12:27:38.554519  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:38.553150  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1243) took 1009 ms. Trace:
W20260709 12:27:38.533593 12417 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:27:38.540299  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:38.561748  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:38.562091  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:38.562211  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:38.536499 12071 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:38.587430  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:38.559268 12262 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:38.588419  6785 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 18: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:38.590349  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:38.565307 12153 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:38.559950  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 295055us
I20260709 12:27:38.554916 11854 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:38.535718 12305 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:38.591295  6768 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:27:38.568523 12369 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:38.600683  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:38.607760  6785 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
I20260709 12:27:38.604336  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:38.608883  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:38.611133  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:38.558969 12137 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:38.613348  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:38.613436  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:38.614368  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:38.579481 12280 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:38.615079  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:38.633651  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1699) took 1012 ms. Trace:
I20260709 12:27:38.634246  6762 rpcz_store.cc:276] 0709 12:27:37.620857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.589772 (+968915us) service_pool.cc:224] Handling call
0709 12:27:38.633526 (+ 43754us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:38.625820  5945 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:38.618650  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1272) took 1223 ms. Trace:
I20260709 12:27:38.635741  5867 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:38.636273  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:38.637848  5945 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:38.635603  6766 rpcz_store.cc:276] 0709 12:27:37.395068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.395534 (+   466us) service_pool.cc:224] Handling call
0709 12:27:37.395964 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:37.396415 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.396443 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:37.397085 (+   642us) log.cc:844] Serialized 26 byte log entry
0709 12:27:38.201568 (+804483us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:38.201661 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.201745 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:38.618443 (+416698us) raft_consensus.cc:1685] finished
0709 12:27:38.618463 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.618589 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2251,"raft.queue_time_us":926,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":29078,"thread_start_us":691339,"threads_started":1,"wal-append.queue_time_us":821812}
I20260709 12:27:38.640518  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:38.601070 12153 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:38.607421 12369 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.644687  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:38.614035 12290 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:38.578291 12414 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:27:38.644639  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:38.651585  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 279233us
I20260709 12:27:38.645783  5941 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 19: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:38.659189  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:38.699234  6020 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 18: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:38.613932 12421 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:38.658474  6609 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:38.672866  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:38.643085 12305 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:38.707909  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1696) took 1088 ms. Trace:
I20260709 12:27:38.700171  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:38.713187  6768 rpcz_store.cc:276] 0709 12:27:37.619430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.613912 (+994482us) service_pool.cc:224] Handling call
0709 12:27:38.707790 (+ 93878us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:38.656008 12307 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:38.653486 12153 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:38.700369  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 254321us
I20260709 12:27:38.725016  6020 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:27:38.714198  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:38.620231 12303 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:27:38.646214 12347 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:38.656834 12369 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:38.714203  6024 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
W20260709 12:27:38.717139  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 58818.3us
I20260709 12:27:38.726727  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:38.790522  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.877311  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:38.888278  6018 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:27:38.888514  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:38.711539 12421 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:38.889858 12421 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:38.718106 12172 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:38.891260  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:38.891945  6020 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:38.892057 12137 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:38.876348 12223 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:38.892832  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:38.893437  6020 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:38.893365 12223 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:38.895632 12223 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:38.896574 12305 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:38.898339  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:38.899662  6609 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:38.901165 12278 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:38.902055 12278 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:38.902376 12278 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.902691 12278 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:38.889546  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:38.906661 12424 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
W20260709 12:27:38.908082 12430 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 12:27:38.890241  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:38.911216  6024 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:38.914568 12278 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:38.917074 12278 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:38.889590  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:38.922158 12415 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.368s	user 0.000s	sys 0.001s
I20260709 12:27:38.923651  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:38.881649  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1585) took 1660 ms. Trace:
I20260709 12:27:38.925172  6021 rpcz_store.cc:276] 0709 12:27:37.221361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.171794 (+950433us) service_pool.cc:224] Handling call
0709 12:27:38.201217 (+ 29423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.201431 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:27:38.578365 (+376934us) log.cc:844] Serialized 14 byte log entry
0709 12:27:38.880545 (+302180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:38.880623 (+    78us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:38.880963 (+   340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.881312 (+   349us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.881547 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76657,"prepare.queue_time_us":441445,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":13434,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":140,"thread_start_us":599473,"threads_started":2}
I20260709 12:27:38.928179  6021 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:38.718436 12431 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.931914 12431 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:38.935250 12431 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:38.941646 12413 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:38.942955 12117 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:38.701330 12202 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.180s
W20260709 12:27:38.964838  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.922188  6020 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:39.018056  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:39.023922  6020 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:38.878341 12300 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:27:39.043169  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 329009us
I20260709 12:27:38.881268 12400 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:39.063150  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:39.025689 12439 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 12:27:39.058806  5990 leader_election.cc:336] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.698s (SENT)
I20260709 12:27:38.951354 12422 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:39.057966  6015 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.039s	sys 0.065s
W20260709 12:27:39.118907  6015 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.039s	sys 0.066s
I20260709 12:27:39.052968  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:39.117794  5990 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:39.076575  6024 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
W20260709 12:27:39.089596  5915 connection.cc:256] RPC call timeout handler was delayed by -1.2025307670000984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
I20260709 12:27:38.879410 12307 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:39.050573 12117 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.099s
W20260709 12:27:39.157032  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 98250.2us
I20260709 12:27:39.065559 12223 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
W20260709 12:27:39.152413  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 99562.8us
I20260709 12:27:38.922194 12421 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:39.151167  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1582) took 1944 ms. Trace:
I20260709 12:27:39.165472  6015 rpcz_store.cc:276] 0709 12:27:37.207089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.138286 (+931197us) service_pool.cc:224] Handling call
0709 12:27:38.138748 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.139032 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:38.312820 (+173788us) log.cc:844] Serialized 14 byte log entry
0709 12:27:39.150362 (+837542us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:39.150476 (+   114us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:39.150559 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.150792 (+   233us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.151054 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21508,"prepare.queue_time_us":395229,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":36621,"spinlock_wait_cycles":8147200,"thread_start_us":979007,"threads_started":2,"wal-append.queue_time_us":837478}
I20260709 12:27:39.161500  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:39.048847 12300 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:39.056291 12441 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:39.151813 12446 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.176481 12117 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:39.178638 12223 raft_consensus.cc:2764] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:27:39.163707 12413 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:39.163419 12445 raft_consensus.cc:2764] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:27:38.368105  5944 rpcz_store.cc:276] 0709 12:27:37.127832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.127973 (+   141us) service_pool.cc:224] Handling call
0709 12:27:37.160399 (+ 32426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.160654 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:37.875631 (+714977us) log.cc:844] Serialized 14 byte log entry
0709 12:27:38.366829 (+491198us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:38.366890 (+    61us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:38.366940 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.367017 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.367161 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":894879,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":8654,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":129,"thread_start_us":1164843,"threads_started":2}
I20260709 12:27:39.060818 12290 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.103s
I20260709 12:27:39.068352 12303 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 12:27:39.139374 12422 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:39.266570 12447 consensus_queue.cc:1059] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:39.207315 12307 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:39.276398  6016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.072s	sys 0.137s
W20260709 12:27:39.277328  6016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.072s	sys 0.137s
W20260709 12:27:39.279167  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 379534us
I20260709 12:27:39.280594  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:39.284785  6785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.007s	sys 0.039s
W20260709 12:27:39.285240  6785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.007s	sys 0.039s
I20260709 12:27:39.287621  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:39.279199 12137 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:39.287564 12243 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.976s	user 0.009s	sys 0.005s
I20260709 12:27:39.295526  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:39.285244 12447 consensus_queue.cc:1243] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:39.255290 12446 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:39.263484 12300 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:39.288019 12307 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.293995 12199 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:39.304670 12154 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.101s	user 0.012s	sys 0.004s
I20260709 12:27:39.297819 12259 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:39.315555 12438 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 12:27:39.311291 12448 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 12:27:39.317399 12444 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:27:39.318368 12421 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:39.319048 12354 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:39.319633 12354 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:39.320380 12276 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:39.320506 12422 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:39.320587 12354 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:39.323494 12354 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:39.324173 12369 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:39.309695 12300 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:39.334558 12369 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:39.310320 12307 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:39.336457 12446 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:39.391561  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 318717us
W20260709 12:27:39.395349  5865 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1311) took 4163 ms (client timeout 2382 ms). Trace:
W20260709 12:27:39.396137  5865 rpcz_store.cc:269] 0709 12:27:35.232111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.232333 (+   222us) service_pool.cc:224] Handling call
0709 12:27:39.394458 (+4162125us) spinlock_profiling.cc:243] Waited 2.79 s on lock 0x7b5400026a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:39.394886 (+   428us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6405676928}
I20260709 12:27:39.396808  5838 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:39.401388  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2077) took 1267 ms. Trace:
I20260709 12:27:39.402145  5867 rpcz_store.cc:276] 0709 12:27:38.133283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.176398 (+ 43115us) service_pool.cc:224] Handling call
0709 12:27:39.400700 (+1224302us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b54001be754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:39.401133 (+   433us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":862004608}
I20260709 12:27:39.141474  5915 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:39.411515  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1704) took 1109 ms. Trace:
I20260709 12:27:39.339265 12307 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:39.331444 12262 consensus_meta.cc:287] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.724s	user 0.010s	sys 0.002s
I20260709 12:27:39.300396 12243 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:39.299827 12117 raft_consensus.cc:3055] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:27:39.334935 12311 consensus_meta.cc:287] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.020s	user 0.008s	sys 0.008s
W20260709 12:27:39.412246  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 270667us
I20260709 12:27:39.428373  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1550) took 2170 ms. Trace:
W20260709 12:27:39.429942  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 134323us
I20260709 12:27:39.431219  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1242) took 1926 ms. Trace:
W20260709 12:27:39.422163  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:39.415096  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:39.426209  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 264619us
I20260709 12:27:39.429129  5872 rpcz_store.cc:276] 0709 12:27:37.258144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.588632 (+330488us) service_pool.cc:224] Handling call
0709 12:27:38.233095 (+644463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.245823 (+ 12728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.427524 (+1181701us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b5400118ad0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:39.427946 (+   422us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481339136}
I20260709 12:27:39.425402 12243 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:39.392870  5696 catalog_manager.cc:5697] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:39.477890  5943 rpcz_store.cc:276] 0709 12:27:37.504563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.504794 (+   231us) service_pool.cc:224] Handling call
0709 12:27:37.505122 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.545838 (+ 40716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.430262 (+1884424us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b5400530f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:39.430810 (+   548us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91849088}
W20260709 12:27:39.496563  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 93262.5us
I20260709 12:27:39.499958  5915 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:39.415804 12307 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:39.508378  6756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1271) took 2113 ms (client timeout 1560 ms). Trace:
I20260709 12:27:39.511142  5838 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:27:39.513250  6773 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1698) took 1892 ms (client timeout 1704 ms). Trace:
W20260709 12:27:39.514003  6773 rpcz_store.cc:269] 0709 12:27:37.620540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.288773 (+668233us) service_pool.cc:224] Handling call
0709 12:27:39.512529 (+1223756us) spinlock_profiling.cc:243] Waited 65.4 ms on lock 0x7b5400006c54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:39.512923 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150392192}
I20260709 12:27:39.514673 12455 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:39.515856 12455 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:39.518496  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:39.519393  6793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1695) took 1900 ms (client timeout 1506 ms). Trace:
W20260709 12:27:39.509166  6756 rpcz_store.cc:269] 0709 12:27:37.394633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.394811 (+   178us) service_pool.cc:224] Handling call
0709 12:27:39.507433 (+2112622us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b5400004e54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:39.507901 (+   468us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4317156224}
W20260709 12:27:39.520001  6793 rpcz_store.cc:269] 0709 12:27:37.618801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.702703 (+ 83902us) service_pool.cc:224] Handling call
0709 12:27:39.518934 (+1816231us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b5400005d54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:39.519178 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":780036352}
I20260709 12:27:39.430663 12347 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:39.422588 12077 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:39.528390 12315 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:39.529289 12315 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.529923 12315 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:39.532030 12446 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:39.422900 12117 raft_consensus.cc:740] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:39.539119 12440 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:27:39.546537  5942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 1963) took 1983 ms (client timeout 2039 ms). Trace:
I20260709 12:27:39.547304 12173 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:39.547462  5942 rpcz_store.cc:269] 0709 12:27:37.562927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.565040 (+  2113us) service_pool.cc:224] Handling call
0709 12:27:39.545799 (+1980759us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b54004682d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:39.546153 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390939904}
W20260709 12:27:39.548657 12398 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.024s	sys 0.034s
I20260709 12:27:39.427145 12311 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:39.417078 12262 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:39.535317 12077 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:27:39.556910 12398 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.024s	sys 0.036s
I20260709 12:27:39.412319  6785 rpcz_store.cc:276] 0709 12:27:38.302174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.617280 (+315106us) service_pool.cc:224] Handling call
0709 12:27:38.633520 (+ 16240us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:38.634044 (+   524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:38.634094 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:38.642170 (+  8076us) log.cc:844] Serialized 52 byte log entry
0709 12:27:39.285930 (+643760us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:39.285984 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.286066 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:39.410351 (+124285us) raft_consensus.cc:1685] finished
0709 12:27:39.410460 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.411059 (+   599us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":717,"raft.queue_time_us":117,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":5248,"thread_start_us":575125,"threads_started":1,"wal-append.queue_time_us":689561}
I20260709 12:27:39.579763  5838 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:39.588240 12307 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:39.589913  5915 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:39.591070  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 47505.9us
I20260709 12:27:39.560680 12077 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:39.597021  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44905.9us
W20260709 12:27:39.572403 12280 consensus_meta.cc:287] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.929s	user 0.015s	sys 0.002s
I20260709 12:27:39.602465 12092 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:39.589617 12298 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:39.606526  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.609995  5840 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:39.612047 12456 consensus_queue.cc:1059] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:39.613798  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:39.609341 12391 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:39.592547 12117 consensus_queue.cc:260] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:39.622506 12334 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:38.575966  5940 rpcz_store.cc:276] 0709 12:27:37.543802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.547196 (+  3394us) service_pool.cc:224] Handling call
0709 12:27:37.583365 (+ 36169us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.583664 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:37.950054 (+366390us) log.cc:844] Serialized 14 byte log entry
0709 12:27:38.552679 (+602625us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:38.552755 (+    76us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:38.552811 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.552904 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.553063 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7557,"prepare.queue_time_us":595262,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":4985,"raft.queue_time_us":118,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":6890,"thread_start_us":968009,"threads_started":2}
I20260709 12:27:39.626616  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:39.628118 12262 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:39.630788 12311 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:39.642283  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1479) took 4394 ms. Trace:
W20260709 12:27:39.646879  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 120320us
I20260709 12:27:39.555596 12455 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:39.662937 12446 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.002s
W20260709 12:27:39.676362  6017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1491) took 2305 ms (client timeout 2565 ms). Trace:
I20260709 12:27:39.677501  6022 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:39.679046 12334 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:39.683497 12092 raft_consensus.cc:697] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:39.694178 12334 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:39.693766 12092 consensus_queue.cc:237] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:39.681486  6019 rpcz_store.cc:276] 0709 12:27:35.247860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.391672 (+1143812us) service_pool.cc:224] Handling call
0709 12:27:37.391038 (+999366us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:37.391629 (+   591us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:37.391671 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:37.418280 (+ 26609us) log.cc:844] Serialized 78 byte log entry
0709 12:27:38.200017 (+781737us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:38.578691 (+378674us) log.cc:844] Serialized 14 byte log entry
0709 12:27:38.579391 (+   700us) log.cc:844] Serialized 14 byte log entry
0709 12:27:38.579605 (+   214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.579716 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:38.886421 (+306705us) raft_consensus.cc:1685] finished
0709 12:27:38.886440 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.641611 (+755171us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b54005659d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:39.642022 (+   411us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73844,"prepare.queue_time_us":553295,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":106,"raft.queue_time_us":24099,"raft.run_cpu_time_us":525,"raft.run_wall_time_us":49166,"spinlock_wait_cycles":2294635264,"thread_start_us":990323,"threads_started":2,"wal-append.queue_time_us":847554}
W20260709 12:27:39.677505  6778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1690) took 2052 ms (client timeout 1698 ms). Trace:
W20260709 12:27:39.696877  6778 rpcz_store.cc:269] 0709 12:27:37.604888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.606286 (+  1398us) service_pool.cc:224] Handling call
0709 12:27:39.657091 (+2050805us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b5400008a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:39.657511 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2054000896}
W20260709 12:27:39.683974  6017 rpcz_store.cc:269] 0709 12:27:37.370725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.470874 (+100149us) service_pool.cc:224] Handling call
0709 12:27:39.672549 (+2201675us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b5400764b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:39.676108 (+  3559us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":928699776}
I20260709 12:27:39.689558  5873 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:27:39.690259  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:39.691341  5867 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:39.698711  5873 pending_rounds.cc:85] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 8
I20260709 12:27:39.704659  5872 raft_consensus.cc:2410] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 12:27:39.705192  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 126786us
I20260709 12:27:39.708029 12347 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:39.709092 12347 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.702271  5873 pending_rounds.cc:107] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:27:39.710620  5873 raft_consensus.cc:2889] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:39.718713  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:39.715227  6015 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:39.721071 12200 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.729822  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:39.730718  6019 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:39.734745  6017 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:39.735459  5943 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 12:27:39.736820  6017 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:27:39.737473  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:39.729549 12200 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:39.727403 12137 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:39.740466 12137 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:39.741588 12227 raft_consensus.cc:3055] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:39.742267 12200 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:39.726739 12203 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:39.742434 12227 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:39.742990 12137 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:39.743283 12203 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:39.745831  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:39.746312  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 1972) took 1918 ms. Trace:
I20260709 12:27:39.717288  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:39.746747  5949 rpcz_store.cc:276] 0709 12:27:37.827515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.828869 (+  1354us) service_pool.cc:224] Handling call
0709 12:27:37.829268 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.299279 (+470011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.745708 (+1446429us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b5400632254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:39.746072 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1041180032}
I20260709 12:27:39.714020  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:39.750463  6016 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 19: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:39.750878  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:39.751847  5942 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
W20260709 12:27:39.753973  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 159539us
I20260709 12:27:39.755271  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:39.757300 12154 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:39.757997 12154 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:39.758651 12154 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:39.760499 12070 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.762049 12203 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:39.762327 12223 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.762967 12223 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:39.716503  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:39.720940  6611 connection.cc:256] RPC call timeout handler was delayed by -1.1614907820003282s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
W20260709 12:27:39.769572  6611 connection.cc:256] RPC call timeout handler was delayed by -1.0942577710002297s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
I20260709 12:27:39.773669  6024 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:39.715864  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:39.727391  5865 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
W20260709 12:27:39.774827  6611 connection.cc:256] RPC call timeout handler was delayed by -1.0202305380003054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
I20260709 12:27:39.709609 12347 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:39.771579  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 72383.5us
I20260709 12:27:39.777117 12398 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:39.780885  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:39.781777 12398 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:39.781996  5941 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:39.782562  5872 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:39.761466 12070 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:39.781055  5696 catalog_manager.cc:5697] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 15 to 18, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 18 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:39.784775 12263 raft_consensus.cc:3055] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:39.785288 12263 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:39.785741  5940 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 12:27:39.786916  6021 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:27:39.786564 12263 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:39.784504 12398 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:39.792156 12263 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:39.781903  6018 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:39.774223 12456 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:39.801831 12223 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:39.804301  5696 catalog_manager.cc:5697] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 13 to 18, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 18 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:27:39.806843  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:39.808477  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:39.806862  5696 catalog_manager.cc:5697] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 14 to 19, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 19 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:27:39.805884  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:39.811623  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1575) took 2677 ms. Trace:
I20260709 12:27:39.814141  6022 rpcz_store.cc:276] 0709 12:27:37.121772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.079406 (+957634us) service_pool.cc:224] Handling call
0709 12:27:39.676425 (+1597019us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:39.684256 (+  7831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.789487 (+105231us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b5400764b50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:39.798750 (+  9263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3668520704}
I20260709 12:27:39.811079  5696 catalog_manager.cc:5697] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:39.745214 12227 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:39.815801  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:39.827297  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:39.824615  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 49498.3us
W20260709 12:27:39.825999  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:39.829523  5913 connection.cc:256] RPC call timeout handler was delayed by -1.5638011500004723s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
W20260709 12:27:39.830410  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 66155.4us
I20260709 12:27:39.802223 12070 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:39.833942  5913 connection.cc:256] RPC call timeout handler was delayed by -1.3946011500001987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2006s for a response
I20260709 12:27:39.841905  6018 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:39.842372  6021 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:39.852308  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:39.857257  6016 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:39.860987  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:39.865015 12310 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.866217 12310 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:39.869165 12137 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:39.869619  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:39.870045 12137 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:39.872751 12137 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:39.870458  5940 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:39.873605  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:39.865290  5942 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:39.872783  6793 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:27:39.856266 12070 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:39.891422  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.858448 12456 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:39.887802  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:39.887038 12117 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:39.863415 12153 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.933s	user 0.014s	sys 0.016s
I20260709 12:27:39.901983  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:39.869222 12310 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:39.909221  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:39.913264  6016 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
I20260709 12:27:39.912879  6021 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:39.897571 12334 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.926133 12305 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.926944 12305 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:39.930377  5940 raft_consensus.cc:3055] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:39.931133  5940 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:39.914242 12227 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:39.925310  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:39.940449  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:39.941360  6022 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 14.
I20260709 12:27:39.942418  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:39.944458 12174 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:39.945747 12300 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.555s	user 0.009s	sys 0.016s
I20260709 12:27:39.932217 12070 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:39.946060  5696 catalog_manager.cc:5697] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 19 to 23, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 23 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:39.948997 12070 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:39.949905 12120 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:39.950788 12120 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:39.951447  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.953293  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:39.936793  6793 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
W20260709 12:27:39.953799  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 102859us
I20260709 12:27:39.958209  6019 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:39.958997  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:39.948716  5696 catalog_manager.cc:5697] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:27:39.966796  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:39.970116  6016 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:39.919623 12153 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:39.977656  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 122671us
I20260709 12:27:39.977975 12262 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:39.924373 12456 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:39.980641  6024 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 18: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:39.953236 12120 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:39.975553  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:39.982759  5941 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:39.936880 12334 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:39.977618 12462 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:27:39.973975 12305 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:40.001330  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:40.003350  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:40.015555  6793 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:27:40.011123  5940 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:40.011539 12071 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.302s	user 0.021s	sys 0.009s
I20260709 12:27:40.016146  5838 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:40.014925  5696 catalog_manager.cc:5697] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 11 to 16, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 16 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:40.023625  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:40.019205  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:39.984364 12077 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:40.030741  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:40.031900  5872 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 23: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:40.040647 12203 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:40.042157  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:40.043355  5871 raft_consensus.cc:2410] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
W20260709 12:27:40.045312 12464 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:27:40.023623 12398 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:40.049458  5915 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:40.059402 12154 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:40.057423  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:40.070129  5874 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:40.070813  5865 raft_consensus.cc:2393] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:40.060045 12311 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:40.071519 12311 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.071957 12311 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:40.074085  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:40.075208 12465 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.079s	user 0.003s	sys 0.000s
I20260709 12:27:40.075934  5940 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:40.050237  6016 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 10. (term mismatch)
I20260709 12:27:40.060487 12153 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:40.050012 11977 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.061141  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:40.050160 12300 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:40.027765 12307 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:27:40.029928 12071 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:40.063825  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:40.064081  5873 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 15.
I20260709 12:27:40.050370  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:40.096629  6021 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:40.051214  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1229) took 2844 ms. Trace:
W20260709 12:27:40.051272  5945 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.342s	user 0.017s	sys 0.014s
I20260709 12:27:40.053592 12398 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:40.054805  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:40.053957  6785 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:40.055152  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:40.055953  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:40.050619  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:40.066802  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:40.065237  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 83655.9us
I20260709 12:27:40.078251  6016 pending_rounds.cc:85] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 9
I20260709 12:27:40.027712 12092 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:40.065817 12456 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:40.102698  6017 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:40.101992  6016 pending_rounds.cc:107] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:27:40.103440  6016 raft_consensus.cc:2889] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:40.106451  5869 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:40.109104  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:40.109282  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:40.056715  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:40.113723  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 52835.3us
I20260709 12:27:40.054870 12334 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:40.051854 12077 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:40.117592  5990 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:40.117445  5945 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:40.117302  5870 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 14.
I20260709 12:27:40.124819  6021 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:40.103439 12398 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:40.127350  5696 catalog_manager.cc:5697] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 16 to 21, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 21 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:40.139111  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:40.141829 12120 consensus_queue.cc:1059] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:40.143327 12120 raft_consensus.cc:3055] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:40.143754 12120 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:40.145004 12120 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:40.147296 12120 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:40.097548  5948 rpcz_store.cc:276] 0709 12:27:37.128197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.170632 (+ 42435us) service_pool.cc:224] Handling call
0709 12:27:37.175418 (+  4786us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:37.461050 (+285632us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:27:39.540206 (+2079156us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b1800225880. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d667b3c7 00007f39d2c8d9a7 00007f39d2c9d0bf 00007f39d2c9d038
0709 12:27:39.586141 (+ 45935us) log.cc:844] Serialized 14 byte log entry
0709 12:27:39.770135 (+183994us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:39.862184 (+ 92049us) log.cc:844] Serialized 52 byte log entry
0709 12:27:39.862419 (+   235us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:39.901576 (+ 39157us) log.cc:844] Serialized 14 byte log entry
0709 12:27:39.901996 (+   420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.937182 (+ 35186us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b540058f554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:39.937254 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:39.954656 (+ 17402us) raft_consensus.cc:1685] finished
0709 12:27:39.954682 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.972116 (+ 17434us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61986,"prepare.queue_time_us":679454,"prepare.run_cpu_time_us":157,"prepare.run_wall_time_us":9932,"raft.queue_time_us":86708,"raft.run_cpu_time_us":547,"raft.run_wall_time_us":65126,"spinlock_wait_cycles":563448960,"thread_start_us":155237,"threads_started":2,"wal-append.queue_time_us":208213}
I20260709 12:27:40.153939  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:40.154636  6021 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:40.122239 12154 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:40.153191  6016 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:40.128746 12446 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:40.151678  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:40.087584 11977 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:40.163184  5945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 1251) took 1547 ms (client timeout 2023 ms). Trace:
W20260709 12:27:40.164826  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 57375.2us
W20260709 12:27:40.163882  5945 rpcz_store.cc:269] 0709 12:27:38.615729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.635846 (+ 20117us) service_pool.cc:224] Handling call
0709 12:27:40.157198 (+1521352us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:40.168938  5696 catalog_manager.cc:5697] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 16 to 20, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 20 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:40.172564  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 99306.5us
I20260709 12:27:40.150972 12334 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:40.179798  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 197916us
I20260709 12:27:40.183357  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1588) took 2866 ms. Trace:
I20260709 12:27:40.183969  6016 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:40.184182  6020 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: []
W20260709 12:27:40.184993  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:40.185561  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:40.185889  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.159579 12446 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:40.190348  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 149978us
I20260709 12:27:40.132917 12092 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:40.130452 12300 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:40.197780  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 134152us
I20260709 12:27:40.196410  5696 catalog_manager.cc:5697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:40.162782 12153 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:40.181178 12070 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:40.168200 12391 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:40.208307  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 91197.3us
I20260709 12:27:40.209466 11977 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:40.202378  5915 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:40.212740  6018 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:27:40.195623 12446 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:40.194793 12077 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.210244 12305 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:40.197412 12280 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:40.216840  5696 catalog_manager.cc:5697] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 9 to 12, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 12 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:27:40.172178 12347 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:40.183929  6023 rpcz_store.cc:276] 0709 12:27:37.222531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.186403 (+963872us) service_pool.cc:224] Handling call
0709 12:27:38.186843 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.187175 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:39.786691 (+1599516us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b1800430e40. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39d282b4bc 00007f39cfb7948b 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d667b3c7 00007f39d2c8d9a7 00007f39d2c9d0bf
0709 12:27:39.787173 (+   482us) log.cc:844] Serialized 14 byte log entry
0709 12:27:40.004226 (+217053us) log.cc:844] Serialized 14 byte log entry
0709 12:27:40.008793 (+  4567us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:40.012340 (+  3547us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:40.026416 (+ 14076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.034835 (+  8419us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:40.088434 (+ 53599us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37799,"prepare.queue_time_us":527085,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":103,"raft.queue_time_us":17457,"raft.run_cpu_time_us":574,"raft.run_wall_time_us":3602,"spinlock_wait_cycles":5961472,"thread_start_us":457650,"threads_started":3,"wal-append.queue_time_us":137801}
I20260709 12:27:40.221318  5838 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:40.248797  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 58067.1us
I20260709 12:27:40.220412 11977 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:40.227658 12137 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:40.248490  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:40.271509  5913 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:40.272533 12280 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:40.281759 12280 consensus_queue.cc:1243] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:27:40.255149 12315 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.724s	user 0.008s	sys 0.012s
I20260709 12:27:40.258972 11977 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.258551  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:40.273672  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 76104.9us
I20260709 12:27:40.280220  6021 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:40.274899  5696 catalog_manager.cc:5697] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 13 to 21, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 21 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:27:40.258077 12298 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:40.282723 12280 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:40.293139 12158 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:40.239058 12456 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
W20260709 12:27:40.295377  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 76969.4us
I20260709 12:27:40.230464 12334 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:40.239760 12153 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:40.251286 12077 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:40.290302 11977 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:40.312317  5697 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40818 (request call id 149) took 69153 ms (client timeout 15000 ms). Trace:
I20260709 12:27:40.291704 12298 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:40.317919  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 46547.8us
I20260709 12:27:40.318755 12480 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:40.319043  5913 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:40.320005  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1591) took 1596 ms. Trace:
I20260709 12:27:40.324277 12347 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:40.241564 12391 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:40.320684  6020 rpcz_store.cc:276] 0709 12:27:38.723835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.056766 (+332931us) service_pool.cc:224] Handling call
0709 12:27:39.057303 (+   537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:40.184661 (+1127358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.319471 (+134810us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400364cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:40.319894 (+   423us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2460491136}
I20260709 12:27:40.326880 12391 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:40.327750 12391 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:40.328428 12077 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:40.326774  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.319262 12480 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.291266 12262 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:27:40.331084 12334 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:40.336618  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 48219.2us
I20260709 12:27:40.295060 12280 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:40.333511 12154 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:40.318015 12298 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:40.289609 12315 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:40.344249  5696 catalog_manager.cc:5697] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:40.329180 12077 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:27:40.360445  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 60216.7us
I20260709 12:27:40.338948 12480 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:40.347679 12071 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:40.367738  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 138190us
I20260709 12:27:40.368834  6611 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:40.370048 12473 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:27:40.370496 12280 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:40.372248 12280 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:40.373266 12479 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:40.374363 12092 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:40.353366 12158 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:40.368911 12477 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:27:40.366070  5840 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:40.349337 12154 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:40.399220  5940 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:27:40.414499  5949 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: [21.10-21.10]
W20260709 12:27:40.421639  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 60072.6us
I20260709 12:27:40.430050  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:40.398540 12154 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:40.438789  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:40.449084 11835 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.434180 12455 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:40.449934 11835 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:40.450028 12455 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:40.450480 12398 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:40.452028 12136 raft_consensus.cc:3055] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:40.451778 12455 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:40.452502 12136 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:40.452965 12455 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:40.458101 12482 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:40.458837 12482 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.459216 12482 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:40.462086  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:40.463140  6756 raft_consensus.cc:2376] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19. Re-sending same reply.
I20260709 12:27:40.464246 12310 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:40.465009 12310 raft_consensus.cc:697] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:40.465276 12300 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:40.466727  6019 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 24. Current term is 25. Ops: [24.14-24.14]
W20260709 12:27:40.450074  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 131179us
W20260709 12:27:40.478302  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:40.478672  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 109990us
I20260709 12:27:40.428540 12158 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:40.465363  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:40.312851  5697 rpcz_store.cc:269] 0709 12:26:31.159136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.165889 (+  6753us) service_pool.cc:224] Handling call
0709 12:27:40.311965 (+69146076us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:30.102292 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:30.103030 (+   738us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:30.103092 (+    62us) write_op.cc:435] Acquired schema lock
0709 12:27:30.103113 (+    21us) tablet.cc:662] Decoding operations
0709 12:27:30.103582 (+   469us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:30.103678 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:30.103704 (+    26us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:27:30.104141 (+   437us) tablet.cc:701] Row locks acquired
0709 12:27:30.104198 (+    57us) write_op.cc:260] PREPARE: finished
0709 12:27:30.104438 (+   240us) write_op.cc:270] Start()
0709 12:27:30.104609 (+   171us) write_op.cc:276] Timestamp: P: 1783600050104348 usec, L: 0
0709 12:27:30.104646 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 12:27:30.150697 (+ 46051us) log.cc:844] Serialized 916 byte log entry
0709 12:27:30.698596 (+547899us) op_driver.cc:464] REPLICATION: finished
0709 12:27:30.857762 (+159166us) write_op.cc:301] APPLY: starting
0709 12:27:30.857904 (+   142us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:30.860527 (+  2623us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:30.860662 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:30.961167 (+100505us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:30.987293 (+ 26126us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:27:30.987357 (+    64us) write_op.cc:312] APPLY: finished
0709 12:27:30.987683 (+   326us) log.cc:844] Serialized 32 byte log entry
0709 12:27:31.011240 (+ 23557us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:31.011387 (+   147us) write_op.cc:454] Released schema lock
0709 12:27:31.011469 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:31.012011 (+   542us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:31.012062 (+    51us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":531936,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":127146,"apply.run_cpu_time_us":7065,"apply.run_wall_time_us":173953,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12495,"num_ops":2,"prepare.queue_time_us":477542,"prepare.run_cpu_time_us":31584,"prepare.run_wall_time_us":300930,"raft.queue_time_us":149493,"raft.run_cpu_time_us":21077,"raft.run_wall_time_us":82539,"replication_time_us":593754,"thread_start_us":653727,"threads_started":4,"wal-append.queue_time_us":271088,"wal-append.run_cpu_time_us":45183,"wal-append.run_wall_time_us":1672967}]]}
W20260709 12:27:40.486213  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 107969us
I20260709 12:27:40.446552  5696 catalog_manager.cc:5697] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
W20260709 12:27:40.493302  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 69846.3us
I20260709 12:27:40.490155  5696 catalog_manager.cc:5697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 15 to 20, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 20 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:40.483546 12136 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:40.495653 12136 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:40.497617 12305 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:27:40.498487 12263 consensus_meta.cc:287] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.705s	user 0.012s	sys 0.004s
I20260709 12:27:40.499787 12315 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:40.499989 12476 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:40.500958  6793 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:40.502013 12203 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:40.502653 12203 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.503072 12203 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:40.512126  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:40.512709 12298 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:40.498572 11835 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:40.499688  6611 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:40.481343 12310 consensus_queue.cc:237] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:40.517671 12347 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:40.518208 12347 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.518606 12347 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:40.520560 12391 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:40.521296 12391 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:40.522656  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:40.523030 12158 raft_consensus.cc:993] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: : Instructing follower 7a0204b931394fc48ffc503716e0d111 to start an election
I20260709 12:27:40.488832 12153 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:40.524791  6773 raft_consensus.cc:2368] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22.
I20260709 12:27:40.523243  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:40.498332 12305 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:40.530460  5696 catalog_manager.cc:5697] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:40.532678  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:40.498232 12398 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:40.536747  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:40.526356  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:40.542160  6785 raft_consensus.cc:2368] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22.
I20260709 12:27:40.547257 12200 raft_consensus.cc:1081] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Signalling peer 7a0204b931394fc48ffc503716e0d111 to start an election
I20260709 12:27:40.547463 11977 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:40.537254 12263 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.086s
W20260709 12:27:40.569108  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.605397  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:40.614787 12391 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:40.624347 12315 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:40.625000 11977 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:40.627239 12493 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:27:40.544593 12153 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:40.623787  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:40.566275 12488 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:40.639218  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 92596.1us
I20260709 12:27:40.651484 12200 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:40.659231  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:40.670044 12203 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:40.662068  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 139610us
I20260709 12:27:40.680831 12154 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:40.666503 12120 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.518s	user 0.014s	sys 0.000s
W20260709 12:27:40.640363 12311 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.568s	user 0.008s	sys 0.012s
W20260709 12:27:40.653999  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 161989us
I20260709 12:27:40.697875  5945 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:40.696684  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:40.699821 12347 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:40.709578  6611 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:40.634562  5696 catalog_manager.cc:5697] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 13 to 14, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 14 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:27:40.654014  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:40.716619  6019 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 22: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:27:40.549052  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 261909us
I20260709 12:27:40.687556 12280 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.706524  5949 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:40.630569 12479 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:40.720829  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:40.626405 12263 consensus_queue.cc:1243] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:40.687079 12488 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:40.730120  5696 catalog_manager.cc:5697] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 16 to 19, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 19 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:27:40.667404 12492 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:40.632594 12354 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:27:40.728317  5990 leader_election.cc:336] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.506s (SENT)
I20260709 12:27:40.640590 12153 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.709687 12311 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:40.751330 12347 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:40.752380 12203 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:40.709857 12200 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
W20260709 12:27:40.753685  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44200.6us
I20260709 12:27:40.753952 12200 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:40.754385 12200 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:40.742486  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:40.756657 12137 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:40.729091 12136 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:40.757126 12137 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:40.737208 12398 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:40.757665 12137 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:40.758306 12398 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.758903 12398 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:40.732671 12305 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:40.765182 12305 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:40.765367  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:40.765919 12153 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:40.766343  5945 raft_consensus.cc:2376] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22. Re-sending same reply.
I20260709 12:27:40.765652  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:40.767113  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:40.767894 12487 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:40.768338 12487 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:40.768748 12487 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:40.768659  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:40.770038  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:40.770378 12137 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:40.771853  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:40.775346  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.777285  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:40.691846 12482 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:40.729334 12280 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:40.787995  5943 raft_consensus.cc:2410] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:27:40.640960 12446 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:40.565896 12300 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.226s
I20260709 12:27:40.792243 12300 consensus_queue.cc:1243] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:40.679509 12480 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:40.800050  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:40.803601  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 85622.7us
W20260709 12:27:40.805162  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 47102.9us
I20260709 12:27:40.802214  5944 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:40.733286 12298 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:40.802512  6778 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
W20260709 12:27:40.810391  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.799649  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:40.729707 12154 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:40.800661  5947 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:40.803866 12481 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:40.798921 12482 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:40.825132 12481 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:40.826788  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:40.828233 12481 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:40.811717  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:40.830035  5946 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:40.813567  5941 raft_consensus.cc:2368] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21.
I20260709 12:27:40.815284 12137 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:40.819496 12070 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.834538 12070 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:40.836620  5943 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:27:40.838830 12354 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:40.822933  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:40.837193 12398 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:40.823464  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:40.840309 12354 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:40.815752  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:40.866272  5949 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:40.819732  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.822250  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:40.821420  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:40.810370  5838 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:40.820757  6611 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:40.824067  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:40.843803  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:40.844923 12200 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:40.872850  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:40.873677  5866 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:40.849974  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:40.850142  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:40.819489  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:40.837615 12070 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:40.821547 12347 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:27:40.857275  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:40.858211  5696 catalog_manager.cc:5697] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 13 to 18, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 18 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:27:40.860417  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:40.871201  6019 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:40.876066  5872 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:40.872608  6785 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:40.878383  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:40.878764  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:40.879401  6021 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:40.879527  6023 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:40.879854  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:40.880533  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:40.880470  6017 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:40.882475  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:40.883015  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:40.883196  5867 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:27:40.883767  6768 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:40.885064  6024 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:40.885789  6775 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 23: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:40.886358  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:40.887058  5870 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:40.880632  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:40.887928 12137 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:40.888396 12137 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.888779 12137 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:40.890967 12200 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:40.891927 12481 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.892663 12481 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:40.894349  6015 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:40.894748  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:40.895526  6016 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:40.895596  5942 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 24: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:40.891815  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:40.901877  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 91522.8us
I20260709 12:27:40.896032 12482 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.906708  6756 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:40.907164  6019 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:40.909701 12482 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:40.910313  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 89661.9us
I20260709 12:27:40.911697  6611 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:40.912132 12482 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:40.913264 12136 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:40.913599  6611 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:40.905290  5866 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:40.914026  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:40.914572  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:40.918936  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:40.914085  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:40.914822  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:40.913280  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:40.931752  5941 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:40.914347  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:40.933882  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:40.934106  6024 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:40.935101  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:40.914654  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:40.937261  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:40.939673  6019 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:40.945187  5944 raft_consensus.cc:2393] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:40.913864 12479 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:40.947669  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:40.944676  5946 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
I20260709 12:27:40.953331  5945 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:40.914222 12136 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:27:40.955727  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:40.956823  6016 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:40.932211 12263 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:27:40.959899 12398 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:40.938793 12154 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:40.961309 12154 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:40.961808  5948 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:40.967748 12487 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:40.969069  6018 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:40.970026  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:40.970470  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:40.971627  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:40.971962  6762 raft_consensus.cc:2410] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 12:27:40.973043  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:40.975400 12481 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:40.976107  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:40.978363 12484 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:40.979175 12484 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:40.979305  6024 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: [16.9-18.10]
I20260709 12:27:40.961828 12154 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:40.981334 12153 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:40.970907  6775 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 25: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:40.985543  6785 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 20: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:40.987035  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:40.950564 12479 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.967818  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:40.989953  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:40.993691  5868 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:40.994364  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:41.004081  6018 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:41.005992  6757 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 18: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:41.008047  6762 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:41.013749  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:41.016386  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:41.018340  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:41.020938  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:41.021960  5941 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:41.023425  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:40.987803 12203 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:41.025041  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:41.025933  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 59501.1us
I20260709 12:27:41.025051  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:41.025324  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:41.029294  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:40.999313 12479 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:41.018483  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:41.035228  5872 raft_consensus.cc:2410] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:27:41.030694  5945 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 21: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:40.941859 12310 consensus_queue.cc:1059] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 12:27:41.031884  6019 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:27:41.032866  5946 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:41.039826  6022 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: [19.9-21.10]
I20260709 12:27:41.040870  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:41.045382  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 150220us
I20260709 12:27:41.044224  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.046172  5867 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:41.047231  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:41.028858 11835 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.033363 12203 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:40.980320 12484 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:41.013604 12137 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:41.064028  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:41.050275  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 322292us
I20260709 12:27:41.064505  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:41.066597  5949 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:41.083401  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:41.082149  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:41.085319  5947 raft_consensus.cc:2410] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 23 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:27:41.087649  5940 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 21: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:41.048830 12263 raft_consensus.cc:3055] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:41.090214  6768 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 12:27:41.082062  5840 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.801s (SENT) [suppressed 44 similar messages]
I20260709 12:27:41.089887  5915 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:41.082551  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:41.051702 12315 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:41.095577  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:41.096648  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:41.092885  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:41.097276  6017 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:41.098943  6019 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:41.093601  5944 raft_consensus.cc:2376] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19. Re-sending same reply.
I20260709 12:27:41.104619  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:41.106158  6024 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:41.066184 12446 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:41.107542  6023 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:41.065734 12481 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.083490 12310 raft_consensus.cc:3055] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:41.112933  5942 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:27:41.112424  6015 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:41.135959  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:41.122884  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:41.146579  5940 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 21: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:41.130482  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:41.132007  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:41.114228 12311 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:41.121129  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:41.123847 12488 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:41.177580  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:41.092939 12263 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:41.121022  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:41.396535  6018 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 25: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:41.126127 12280 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:41.122028 12203 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:41.121833 12499 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:27:41.066628 11835 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:41.205317  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 178128us
I20260709 12:27:41.409847  6757 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21. Ops: [18.10-20.11]
I20260709 12:27:41.410254  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:41.218132  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.411288  6768 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:41.152431 12310 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:41.130483 12487 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:41.114058 12315 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:41.337432  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 247603us
W20260709 12:27:41.338781  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 256736us
I20260709 12:27:41.121177 12092 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:27:41.378058  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 226695us
I20260709 12:27:41.123049 12503 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.132s
I20260709 12:27:41.425397 12503 consensus_queue.cc:1243] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:27:41.379467  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.427314 12487 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.427855  6022 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:41.427986 12487 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:41.416543 12263 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:41.430991 12263 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:41.380168  5949 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 15: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:41.434346 12310 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:41.436535 12310 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:41.395612 12486 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:41.438210 12486 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:41.440934 12486 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:41.441538  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:41.443744  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:41.379967  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:41.225732 12300 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:27:41.378119  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.446144  6756 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:41.150813  6793 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:27:41.069365 12158 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:27:41.200721  5868 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 21: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:41.389969  5948 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:41.391417  5873 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:41.383627  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.394052  6016 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:41.399749  6785 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:27:41.407047  6017 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:41.401937  5869 pending_rounds.cc:85] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 8
I20260709 12:27:41.453524  5869 pending_rounds.cc:107] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:27:41.453858  5869 raft_consensus.cc:2889] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:41.416957  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:41.392473 12070 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:41.422281 12137 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:41.422777 12153 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:41.470775  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 60579.1us
I20260709 12:27:41.472347  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:41.474022  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:41.475462  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.475708 12300 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:41.476145  6762 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:41.476475 12300 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.477021 12300 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:41.478122 12510 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.478734 12510 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:41.479357 12347 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:41.480103 12347 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:41.423395  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:41.480679 12315 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:41.481019  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 57708.6us
I20260709 12:27:41.442925 12475 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.434842  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:41.393059 12504 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:27:41.458632  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:41.458977 12512 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:41.397897 12481 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:41.459750  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:41.453503 12120 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.462674  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.477200  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:41.462942 12200 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.571s	user 0.011s	sys 0.008s
I20260709 12:27:41.465314 11835 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:41.454741 12092 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:41.481843  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:41.492954  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 49294.2us
W20260709 12:27:41.482396 12507 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:27:41.498238 11835 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:41.498803 11835 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.499142 11835 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:41.481009 12510 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:41.483167  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.528409  6757 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
W20260709 12:27:41.483170  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 59361.8us
I20260709 12:27:41.485571  6015 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:41.503860  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:41.487103  6762 raft_consensus.cc:2410] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:27:41.487177  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:41.484130 12491 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:27:41.485966  5865 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 14.
I20260709 12:27:41.482293 12475 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:41.491153  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:41.537045  6785 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:41.489046 12092 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:41.487422 12481 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:41.506882  6773 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:27:41.507408 12487 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:41.510587  5873 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:41.511101  5872 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:41.511349  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:41.511547  5874 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:41.514760  5868 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:41.484277  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.534965  5915 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:41.552644  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.552412  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:41.576043  6611 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:41.605494  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:41.605693  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 70870.4us
I20260709 12:27:41.605284  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:41.606669  5915 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:41.490464 12154 consensus_meta.cc:287] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.509s	user 0.014s	sys 0.002s
I20260709 12:27:41.608309  5940 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 14.
W20260709 12:27:41.617424  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 145245us
I20260709 12:27:41.481595 12315 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.110s
I20260709 12:27:41.612620  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:41.625784  5946 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:27:41.625898 12154 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:41.628189  5870 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:41.482985 12455 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:41.630081 12455 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:41.629848 12154 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:41.504104 12311 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:41.618434  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.611265  5947 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 12:27:41.633862 12311 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:41.619431  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 483550us
I20260709 12:27:41.577924 12503 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:41.615756  6793 raft_consensus.cc:2376] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21. Re-sending same reply.
I20260709 12:27:41.623131  5944 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:41.619357  5942 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:41.630836 12455 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:41.485574 12512 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:41.541469 12480 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:41.610594 12490 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:41.636421 12311 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:41.643069  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:41.643311  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 139539us
I20260709 12:27:41.655454  5871 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:41.643843  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.645190  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:41.575114 12446 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 12:27:41.646210 12503 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:41.639572  5838 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:41.617560 12488 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:41.611008 12510 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:41.632444 12391 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:41.646253  5873 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:41.654099 11977 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:27:41.642400  6756 raft_consensus.cc:2410] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 24 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
W20260709 12:27:41.655366  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.636916 12486 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.679829  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.486274 12120 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:41.590674 12203 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:27:41.649467 12315 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.589156 12158 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:27:41.600229 12347 consensus_queue.cc:1059] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:27:41.674225  5987 leader_election.cc:400] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21.
I20260709 12:27:41.668952 12490 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:41.657436 12354 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:27:41.658811  6785 raft_consensus.cc:2410] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:27:41.681224  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.666760 12487 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:41.642796  5874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e"
dest_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:54876
W20260709 12:27:41.688531  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 70000.5us
I20260709 12:27:41.689675  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.701359  6756 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:41.654752 12512 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:41.680063  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.703125  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.672441 12510 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:41.681597 11977 consensus_queue.cc:1243] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:41.698076  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.682991 12488 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:41.705483  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 300902us
I20260709 12:27:41.688139 12480 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:27:41.713164  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 53161.6us
W20260709 12:27:41.700546 12479 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.657s	user 0.009s	sys 0.006s
I20260709 12:27:41.709857  6785 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:41.713871  6756 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:41.683612 12263 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:27:41.730396  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:41.730746  5990 leader_election.cc:336] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.704s (SENT)
I20260709 12:27:41.731665  5990 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:41.703029 12487 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:41.708973 12510 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:41.686472 12486 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:41.694017 12315 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:41.741163  6785 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:41.711478  6793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:41.751217  6793 raft_consensus.cc:606] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:41.751912  6793 raft_consensus.cc:614] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:41.753896  6757 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 14. (term mismatch)
I20260709 12:27:41.713471 12480 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:41.754452  6757 pending_rounds.cc:85] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 13
I20260709 12:27:41.749768 12504 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:41.738896 12515 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.768261  6757 pending_rounds.cc:107] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 12:27:41.740521 12487 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:41.772483 12490 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:41.774595  6757 raft_consensus.cc:2889] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:41.772112  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 71395us
I20260709 12:27:41.748378 12347 raft_consensus.cc:3055] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:41.797834 12347 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:41.798483 12136 raft_consensus.cc:3055] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:41.798908 12347 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:41.800350 12347 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:41.800979 12120 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:41.783970 12514 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:27:41.746587 12516 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:27:41.788586 12523 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.803934 12523 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:41.806023  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:41.808193  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:41.777383 12512 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:41.809878 12492 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:41.810449 12512 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:41.733016 12479 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:41.796880  5946 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
W20260709 12:27:41.795289 12513 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.089s	user 0.002s	sys 0.000s
I20260709 12:27:41.813089  6766 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:41.803220 12475 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:41.809348 12158 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:41.745775 12153 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:41.793476  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:41.816011  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.799539 12488 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:41.818416  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:41.815726  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:41.783634 12486 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:41.767566 12481 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:41.829108  6609 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:41.813958 12479 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:41.833139  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:41.817920 12153 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:41.876652  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.821790 12136 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:41.878749  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:41.888890  6017 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 21: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:41.888608  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:41.883682  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:41.825995 12137 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:41.879496  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:41.843380  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:41.892282  5870 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:41.895545  6024 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 22: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:41.838232 12203 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:41.841370 12504 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:41.914986  6015 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 15: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:41.897231  6020 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
W20260709 12:27:41.913980  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 84985.8us
I20260709 12:27:41.816540 12515 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:41.929905  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 209792us
I20260709 12:27:41.883138 12484 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:41.935014  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.843146 11835 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:41.896849 12137 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:41.913035 12070 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:41.957324  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 124235us
I20260709 12:27:41.917331 12504 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.918996 12203 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.913095 12516 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:41.913929 12479 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:41.922276 12481 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.996618  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.886279 12480 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:41.931449 12446 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:41.971351  5990 leader_election.cc:336] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.779s (SENT)
I20260709 12:27:41.995040  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:41.995925  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:42.010733  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:42.012092  5940 raft_consensus.cc:2410] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
W20260709 12:27:42.012493  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 353686us
I20260709 12:27:42.009176  6778 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:27:41.984972 12504 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:41.968678 12137 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:42.012708  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 340227us
I20260709 12:27:42.013708  5990 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:41.966756 12503 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:42.000020  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:41.962663  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1260) took 1105 ms. Trace:
I20260709 12:27:42.015301  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:41.983255 12203 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:42.004993 12523 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:42.019222  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:42.023597  5838 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:41.994206 12517 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.973415 12391 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:42.037853  5943 rpcz_store.cc:276] 0709 12:27:40.857382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.876596 (+ 19214us) service_pool.cc:224] Handling call
0709 12:27:40.906662 (+ 30066us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:40.931274 (+ 24612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:40.931309 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:40.943789 (+ 12480us) log.cc:844] Serialized 26 byte log entry
0709 12:27:41.402996 (+459207us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:41.961661 (+558665us) log.cc:844] Serialized 14 byte log entry
0709 12:27:41.962154 (+   493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.962274 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:41.962337 (+    63us) raft_consensus.cc:1685] finished
0709 12:27:41.962355 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:41.962566 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11293,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":23218,"thread_start_us":889672,"threads_started":2,"wal-append.queue_time_us":551890}
I20260709 12:27:42.045586  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:42.091480  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:42.099826  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 54322.2us
I20260709 12:27:42.101291  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:42.005322 12481 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:41.997114 12070 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:42.041236 12354 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.967013 12153 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:42.127765  5945 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:41.983950 12136 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:42.045928 12479 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:42.014430 12446 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:42.141983  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:42.139305 12310 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.702s	user 0.010s	sys 0.006s
I20260709 12:27:42.130492 12315 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:42.099752 12480 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:42.106678 12486 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:42.105822 12523 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:27:42.170301  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 186754us
W20260709 12:27:42.177811  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.102628 12517 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:42.195199  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.197088  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1611) took 1119 ms. Trace:
W20260709 12:27:42.187914  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 142926us
I20260709 12:27:42.194065  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:42.194169 12505 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.009s	sys 0.023s
I20260709 12:27:42.194622  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:42.194330  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:42.132016 12455 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:42.197741  6021 rpcz_store.cc:276] 0709 12:27:41.077335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.135922 (+ 58587us) service_pool.cc:224] Handling call
0709 12:27:41.383216 (+247294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:41.396602 (+ 13386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:41.396634 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:41.411532 (+ 14898us) log.cc:844] Serialized 26 byte log entry
0709 12:27:41.715051 (+303519us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:41.977536 (+262485us) log.cc:844] Serialized 14 byte log entry
0709 12:27:42.196661 (+219125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.196779 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:42.196838 (+    59us) raft_consensus.cc:1685] finished
0709 12:27:42.196854 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.197018 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":462653,"raft.queue_time_us":110987,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":30394,"spinlock_wait_cycles":18626944,"thread_start_us":718740,"threads_started":3,"wal-append.queue_time_us":341118}
W20260709 12:27:42.205412  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.240190  6775 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:27:42.247051  6793 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 20: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:27:42.247846  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.206568  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2002) took 1143 ms. Trace:
I20260709 12:27:42.241770  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:42.263247  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 162075us
I20260709 12:27:42.268451  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:42.268697  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.186192 12153 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:42.263811  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:42.240372 12505 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.009s	sys 0.027s
I20260709 12:27:42.167179 12354 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:42.187842 12516 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:42.199015 12310 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:42.286049  6768 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 22: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:42.288120  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:42.288856  5874 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:27:42.289160  5913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.012s	sys 0.003s
I20260709 12:27:42.289356  5874 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:42.289541  5913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.012s	sys 0.003s
W20260709 12:27:42.291872  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 600537us
W20260709 12:27:42.306933  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.252982  5941 rpcz_store.cc:276] 0709 12:27:41.062886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.063014 (+   128us) service_pool.cc:224] Handling call
0709 12:27:41.101975 (+ 38961us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:41.599524 (+497549us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:41.599567 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:41.638104 (+ 38537us) log.cc:844] Serialized 26 byte log entry
0709 12:27:41.966568 (+328464us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:41.966628 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.966707 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:42.206233 (+239526us) raft_consensus.cc:1685] finished
0709 12:27:42.206267 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.206483 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92154,"raft.run_cpu_time_us":4309,"raft.run_wall_time_us":188897,"thread_start_us":746019,"threads_started":2,"wal-append.queue_time_us":499501}
I20260709 12:27:42.322158  5866 pending_rounds.cc:85] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 8
I20260709 12:27:42.322654  5866 pending_rounds.cc:107] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:27:42.322999  5866 raft_consensus.cc:2889] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:42.318778 12315 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.324153  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:27:42.324736  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 368890us
W20260709 12:27:42.325614  5990 leader_election.cc:336] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.795s (SENT)
W20260709 12:27:42.326615  5990 leader_election.cc:336] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.834s (SENT)
I20260709 12:27:42.206159 12479 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:42.290474 12310 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:42.245306 12391 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:42.331207  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:42.324630 12315 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:42.332247  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:42.335335  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 85674us
I20260709 12:27:42.327898  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:27:42.359227  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.343178  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:42.328637 12300 consensus_meta.cc:287] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.832s	user 0.010s	sys 0.002s
I20260709 12:27:42.397218  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:42.397665  5948 pending_rounds.cc:85] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 9
I20260709 12:27:42.398134  5948 pending_rounds.cc:107] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 19.10
W20260709 12:27:42.328826 12525 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.360s	user 0.001s	sys 0.001s
I20260709 12:27:42.377072  5940 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
W20260709 12:27:42.370306 12347 consensus_meta.cc:287] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.558s	user 0.012s	sys 0.000s
I20260709 12:27:42.334090 12310 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:42.341348 12512 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.530s	user 0.007s	sys 0.004s
I20260709 12:27:42.398408  5948 raft_consensus.cc:2889] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:42.358347 12487 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.554s	user 0.005s	sys 0.007s
W20260709 12:27:42.342653 12539 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:27:42.398149  5941 raft_consensus.cc:2368] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21.
I20260709 12:27:42.401129  5944 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 12:27:42.450728  5944 pending_rounds.cc:85] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 9
I20260709 12:27:42.451040  5944 pending_rounds.cc:107] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:27:42.451274  5944 raft_consensus.cc:2889] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:42.379056 12544 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:27:42.251098 12486 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.107s
I20260709 12:27:42.410007  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:42.452200  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 58422.2us
I20260709 12:27:42.335350 12446 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:42.414789  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:42.415136  6017 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:42.335541 12092 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:42.343350 11835 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:42.377163 12523 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:27:42.431609  6022 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:42.331880 12565 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:42.342881 12531 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:42.433660  5946 raft_consensus.cc:2410] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:27:42.344023 12541 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:42.343446 12515 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:42.334036 12517 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:42.454713  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:42.397356 12154 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:42.462268  5947 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:27:42.463508  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 195105us
I20260709 12:27:42.483736  5987 leader_election.cc:400] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21.
I20260709 12:27:42.545186  5987 leader_election.cc:403] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:27:42.493150  5865 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:27:42.407537 12300 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:42.499405  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:42.463012 12311 consensus_meta.cc:287] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.787s	user 0.015s	sys 0.000s
I20260709 12:27:42.552716  5840 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:42.466153 12488 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.009s	sys 0.017s
W20260709 12:27:42.546233  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 82193.5us
W20260709 12:27:42.466862 12510 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.706s	user 0.010s	sys 0.003s
W20260709 12:27:42.548091  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 64840.2us
I20260709 12:27:42.550205  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:42.546967 11835 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:42.502097 12455 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:42.593387  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.627146  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1612) took 1538 ms. Trace:
W20260709 12:27:42.621411  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 77465.6us
I20260709 12:27:42.330541 12516 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:42.453158 12487 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:42.627751  6019 rpcz_store.cc:276] 0709 12:27:41.089377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.396208 (+306831us) service_pool.cc:224] Handling call
0709 12:27:41.410176 (+ 13968us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:41.410409 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:41.410435 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:41.418590 (+  8155us) log.cc:844] Serialized 26 byte log entry
0709 12:27:41.770625 (+352035us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:41.770677 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.770769 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:42.626834 (+856065us) raft_consensus.cc:1685] finished
0709 12:27:42.626861 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.627066 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":351641,"threads_started":1,"wal-append.queue_time_us":497723}
W20260709 12:27:42.635074  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 82459.5us
I20260709 12:27:42.636353  5990 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:42.498453 12512 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.123s
W20260709 12:27:42.628677 12488 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.016s	sys 0.019s
I20260709 12:27:42.704803  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1613) took 1591 ms. Trace:
I20260709 12:27:42.547039 12565 raft_consensus.cc:697] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:42.708709  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44929.3us
I20260709 12:27:42.638994 12541 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:42.634282 11835 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:42.557529 12311 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:42.738030 12528 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.277s
I20260709 12:27:42.738547 12551 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21.
I20260709 12:27:42.740566 12551 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:42.740767 12567 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.182s	user 0.000s	sys 0.000s
I20260709 12:27:42.552165 12531 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:42.741222 12551 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:42.743582 12531 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:42.744040 12551 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:42.745051 12551 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:42.745565 12551 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:42.745131 12565 consensus_queue.cc:237] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:42.746191 12551 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:42.705350  6018 rpcz_store.cc:276] 0709 12:27:41.113385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.403736 (+290351us) service_pool.cc:224] Handling call
0709 12:27:41.416048 (+ 12312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:41.661201 (+245153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:41.661245 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:41.676361 (+ 15116us) log.cc:844] Serialized 26 byte log entry
0709 12:27:42.000444 (+324083us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:42.000486 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.000554 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:42.704508 (+703954us) raft_consensus.cc:1685] finished
0709 12:27:42.704552 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.704726 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":53840,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"thread_start_us":540621,"threads_started":2,"wal-append.queue_time_us":491256}
I20260709 12:27:42.720434 12481 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:42.720247 12070 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:42.752027  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:42.771250  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:42.775903  5915 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:42.658211 12510 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:42.773240  5987 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:42.627985  5941 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
W20260709 12:27:42.787094  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:42.789091  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.793300  5942 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:42.795253  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:42.802182  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:42.803923  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:42.806020  5915 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:42.810148  5940 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
W20260709 12:27:42.809957  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 462607us
I20260709 12:27:42.821484  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:42.826463  5915 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:42.773487 12455 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.776504 12523 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:42.856217  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.769337 12531 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:42.800800 12512 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:42.810149 12070 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:42.886555 12510 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:42.876168 12528 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:42.889513 12528 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:42.890036 12528 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:42.894131  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:42.796046 12541 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:42.856731  5868 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:42.815562 12515 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:42.813041 12354 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:42.825160 12488 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:42.868860  5840 connection.cc:256] RPC call timeout handler was delayed by -1.5739933559998462s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
W20260709 12:27:42.869063  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 48027.6us
I20260709 12:27:42.871049  5869 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
W20260709 12:27:42.869700  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 487469us
I20260709 12:27:42.877679  5867 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:42.881585  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:42.862931 12481 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:42.884971 12512 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:42.903051  5871 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:27:42.909852  5840 connection.cc:256] RPC call timeout handler was delayed by -1.5712933559998419s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1946s for a response
W20260709 12:27:42.915514  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:42.916496  5840 connection.cc:256] RPC call timeout handler was delayed by -1.3576925709999159s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1779s for a response
W20260709 12:27:42.916973  5840 connection.cc:256] RPC call timeout handler was delayed by -1.2774842419999004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
I20260709 12:27:42.917945  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:42.865183 12455 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:42.918828  5945 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:42.873220 12523 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:42.925787  6018 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:42.894161 12560 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:42.924145  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:42.928139  5869 raft_consensus.cc:2410] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 22 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:27:42.924423  5941 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
I20260709 12:27:42.910244 12488 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:42.902556 12070 raft_consensus.cc:697] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:42.905299 12503 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:42.974128  5874 consensus_meta.cc:287] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.651s	user 0.015s	sys 0.000s
I20260709 12:27:42.938155  6019 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:42.910696 12481 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:42.974717  5874 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:42.964241  5945 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:42.920431 12354 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:43.000478 12354 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:43.002954 12154 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:43.003666 12154 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:42.929934 12523 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:42.896927 12531 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:42.988126  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 59055.5us
I20260709 12:27:43.011977 12531 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:42.990283 12481 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:42.991981 12455 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:42.992875 12070 consensus_queue.cc:237] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:42.922762 12558 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
W20260709 12:27:42.995111 12564 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:27:42.995635 12577 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:42.997529 12492 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:27:42.998570 12504 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.915s	user 0.009s	sys 0.005s
I20260709 12:27:43.073802 12542 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.881182 11835 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:42.998703  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:43.081492  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:43.082054  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:43.082827 12488 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:43.083264 12488 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.083629 12488 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:43.070683 12481 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:42.997648 12581 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:42.924774 12300 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:42.978472 12541 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:43.085978  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:42.919826 12490 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 12:27:43.086012 12581 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:43.067414  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 56919.5us
I20260709 12:27:43.089015 12581 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:43.075476  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.073971 12504 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:43.077900 12492 consensus_queue.cc:1243] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 23.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 12:27:43.074061 12579 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 12:27:43.082206  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 83632.4us
I20260709 12:27:42.983240 12503 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:43.092123 12504 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:43.093169 12503 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:42.930871 12560 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:43.093891 12560 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:43.093943 12529 log.cc:927] Time spent T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Append to log took a long time: real 0.192s	user 0.002s	sys 0.001s
W20260709 12:27:43.061740 12580 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:27:43.059561 12487 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:43.141495  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:43.149796  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 61727.7us
I20260709 12:27:43.018170 12120 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:27:43.154781  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 518493us
I20260709 12:27:43.154985  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:43.158267  6793 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:43.158914  5990 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:43.173295  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:43.181277  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 346055us
W20260709 12:27:43.183216  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 250343us
I20260709 12:27:42.973704 12557 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.416s
W20260709 12:27:43.156085 12310 consensus_meta.cc:287] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.701s	user 0.013s	sys 0.000s
I20260709 12:27:43.194985  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:43.100680 12311 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:43.099031 12455 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:43.195173  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:43.148319 12503 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:43.238454  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 43607.8us
W20260709 12:27:43.171461 12564 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:27:43.194736  5990 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:43.169087 12504 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:27:43.229426  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 78577.7us
I20260709 12:27:43.246476  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:43.246625  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:43.232532  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:43.237097  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.240149  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:43.248715  6766 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
W20260709 12:27:43.233515  5866 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.008s	sys 0.024s
W20260709 12:27:43.244881  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 49808.7us
I20260709 12:27:43.242100  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:43.249456  5866 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.008s	sys 0.024s
I20260709 12:27:43.252614  5944 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 19: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:43.170804 12120 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:43.263072  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.142709 12570 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.145s
I20260709 12:27:43.226785 12573 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:43.269905  5840 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:43.285559  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 45507.1us
I20260709 12:27:43.251945 12557 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:43.246155 12300 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:43.242355 12484 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:43.237874 12577 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:27:43.303925 12484 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:43.306965 12484 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:43.245889 12504 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:43.331199 12504 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:43.332437 12153 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.047s	user 0.005s	sys 0.006s
I20260709 12:27:43.333201 12153 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:43.272449 11835 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:43.141108 12542 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:43.245466 12503 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:43.336382 12542 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:43.338109  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:43.256283 12491 raft_consensus.cc:2764] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:27:43.295339  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:43.295674 12571 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.295849 12576 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:43.291079 12203 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.186s	user 0.012s	sys 0.004s
I20260709 12:27:43.345844 12154 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:43.346128 12571 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:43.346701 12203 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:43.347288  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1269) took 1350 ms. Trace:
I20260709 12:27:43.347764  5942 rpcz_store.cc:276] 0709 12:27:41.997363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.997872 (+   509us) service_pool.cc:224] Handling call
0709 12:27:42.013846 (+ 15974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.810015 (+796169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.346977 (+536962us) spinlock_profiling.cc:243] Waited 763 ms on lock 0x7b54006725d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:43.347201 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1754525952}
I20260709 12:27:43.348755 12203 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:43.302672 12558 consensus_queue.cc:1059] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:43.308686  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:43.350328 12203 raft_consensus.cc:3055] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:43.350903 12576 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:43.351357 12576 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.351686 12576 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:43.351742  5915 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:43.308203 12569 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:27:43.352826 12590 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:43.312580  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1621) took 1014 ms. Trace:
I20260709 12:27:43.353690  6021 rpcz_store.cc:276] 0709 12:27:42.298090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.341846 (+ 43756us) service_pool.cc:224] Handling call
0709 12:27:42.342423 (+   577us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:42.354111 (+ 11688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.354177 (+    66us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:42.425842 (+ 71665us) log.cc:844] Serialized 52 byte log entry
0709 12:27:42.786878 (+361036us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:42.809326 (+ 22448us) log.cc:844] Serialized 14 byte log entry
0709 12:27:42.809619 (+   293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.809733 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.312278 (+502545us) raft_consensus.cc:1685] finished
0709 12:27:43.312308 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.312502 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14126,"prepare.queue_time_us":21782,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":17692,"raft.run_cpu_time_us":411,"raft.run_wall_time_us":35917,"thread_start_us":288822,"threads_started":1,"wal-append.queue_time_us":734841}
I20260709 12:27:43.211670 12310 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:43.357016  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:43.242236 12455 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:43.350685 12203 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:43.319731 12120 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:43.371702  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:43.323385 12488 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:43.325304  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1358) took 1813 ms. Trace:
I20260709 12:27:43.327364  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:43.328789 12589 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.379380 12589 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:43.377946  5872 rpcz_store.cc:276] 0709 12:27:41.512192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.615526 (+103334us) service_pool.cc:224] Handling call
0709 12:27:41.637982 (+ 22456us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:41.668867 (+ 30885us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:42.041136 (+372269us) log.cc:844] Serialized 14 byte log entry
0709 12:27:42.463501 (+422365us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:42.544542 (+ 81041us) log.cc:844] Serialized 52 byte log entry
0709 12:27:42.544761 (+   219us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:42.547106 (+  2345us) log.cc:844] Serialized 14 byte log entry
0709 12:27:42.547324 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.547414 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.324990 (+777576us) raft_consensus.cc:1685] finished
0709 12:27:43.325019 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.325222 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20908,"prepare.queue_time_us":116286,"prepare.run_cpu_time_us":1223,"prepare.run_wall_time_us":15834,"raft.queue_time_us":711347,"raft.run_cpu_time_us":551,"raft.run_wall_time_us":28845,"thread_start_us":683248,"threads_started":3,"wal-append.queue_time_us":671457}
W20260709 12:27:43.330349  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 60573us
I20260709 12:27:43.381451  6775 raft_consensus.cc:2376] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21. Re-sending same reply.
I20260709 12:27:43.309748 12311 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:43.361739  6793 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:43.361701  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:43.376458  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.348748 12571 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:43.348667 12154 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:43.257085 12490 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.115s
I20260709 12:27:43.387292  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:43.390623  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:43.387334  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:43.388077  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 193465us
I20260709 12:27:43.389040  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:43.390831  5942 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:43.391901  6793 raft_consensus.cc:2410] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 12:27:43.387116  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:43.392413  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:43.393360  6766 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:43.394140  5838 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:43.392990  5943 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:43.391816  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:43.399016  6768 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:27:43.336491 12503 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:43.319886 12354 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:43.365164 12153 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:43.357547 12310 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:43.407058  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:43.400965 12491 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.383292 12577 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:27:43.392606 12120 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:27:43.440203  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 171159us
I20260709 12:27:43.410881  5838 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:43.403208 12578 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:27:43.451577  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 44612us
I20260709 12:27:43.452356  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:43.409112 12153 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:43.457069 12480 consensus_meta.cc:287] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.269s	user 0.014s	sys 0.001s
I20260709 12:27:43.457921 12154 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:43.459292 12523 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:43.454052 12592 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:27:43.480278 12523 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:43.480829  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2110) took 1159 ms. Trace:
I20260709 12:27:43.481222  5866 rpcz_store.cc:276] 0709 12:27:42.321016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.321516 (+   500us) service_pool.cc:224] Handling call
0709 12:27:42.321933 (+   417us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:42.545942 (+224009us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.545986 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:42.546613 (+   627us) log.cc:844] Serialized 52 byte log entry
0709 12:27:43.255030 (+708417us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:43.255093 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.255181 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.480601 (+225420us) raft_consensus.cc:1685] finished
0709 12:27:43.480623 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.480769 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21999,"raft.queue_time_us":321814,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":5853,"thread_start_us":692307,"threads_started":2,"wal-append.queue_time_us":777100}
I20260709 12:27:43.457975 12311 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.483132 12311 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:43.483667 12487 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:43.484396 12487 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:43.439791 12503 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.494179  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1360) took 1981 ms. Trace:
W20260709 12:27:43.461836  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 51062.4us
I20260709 12:27:43.494663  5868 rpcz_store.cc:276] 0709 12:27:41.512950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.616930 (+103980us) service_pool.cc:224] Handling call
0709 12:27:41.617352 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.896233 (+1278881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.493895 (+597662us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400118ad4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:43.494109 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2786394880}
I20260709 12:27:43.463367 12541 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:27:43.505215  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 62777.1us
I20260709 12:27:43.446602 12491 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:43.446955 12120 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:43.509665  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:43.470829 12488 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:43.471748  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1619) took 1220 ms. Trace:
I20260709 12:27:43.442327 12203 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:43.475672 12137 consensus_meta.cc:287] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.431s	user 0.014s	sys 0.002s
I20260709 12:27:43.514043  5990 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:43.447105 12481 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
W20260709 12:27:43.487130  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 231051us
I20260709 12:27:43.454906 12578 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.486707  5840 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:43.504719 12541 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.397816 12528 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:43.449246  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:43.486356 12577 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:43.544366 12120 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:43.492030  5942 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:43.515064  6024 rpcz_store.cc:276] 0709 12:27:42.251846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.252162 (+   316us) service_pool.cc:224] Handling call
0709 12:27:42.290437 (+ 38275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:42.826283 (+535846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.826540 (+   257us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:42.862907 (+ 36367us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.097285 (+234378us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:43.097330 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.097415 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.471475 (+374060us) raft_consensus.cc:1685] finished
0709 12:27:43.471500 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.471678 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37141,"raft.queue_time_us":189637,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":5062,"thread_start_us":624964,"threads_started":2,"wal-append.queue_time_us":523335}
W20260709 12:27:43.530117  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:43.507221 12551 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.688s	user 0.007s	sys 0.007s
W20260709 12:27:43.549362  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 62735.6us
I20260709 12:27:43.519182 12516 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:43.563759  5840 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:43.552186  6609 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:43.564941  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 1266) took 1588 ms. Trace:
I20260709 12:27:43.473412 12504 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:43.504396 12503 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:43.575759  5946 rpcz_store.cc:276] 0709 12:27:41.976885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.993156 (+ 16271us) service_pool.cc:224] Handling call
0709 12:27:43.564701 (+1571545us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b5400632754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:43.564873 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":905110784}
W20260709 12:27:43.602001  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 152837us
I20260709 12:27:43.602929  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:43.479190 12576 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:43.550256 12541 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:43.547515 12578 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:43.561455 12354 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:43.612915  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:43.614429  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 100471us
I20260709 12:27:43.560846 12203 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:43.617951  5946 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:43.548139 12137 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:43.626546  5915 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:43.620057  5990 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:43.606002 12070 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:27:43.633333  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 69634us
I20260709 12:27:43.608876 12488 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:43.660919  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:43.545501 12490 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 12:27:43.610138 12455 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:43.663587  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:43.621773 12154 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:43.673346  6015 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:43.603729 12551 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:43.675434  5840 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:43.686491 12538 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:43.663683 12590 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:43.664193 12120 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:43.689780  5915 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:43.670418 12577 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:43.695983  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 143882us
I20260709 12:27:43.677871 12515 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:43.678128 12516 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:43.670550 12200 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:27:43.673918 12310 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:27:43.689018 12589 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:43.708264 12588 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 12:27:43.709089 12607 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:43.700932 12608 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
W20260709 12:27:43.737291  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 47637.7us
W20260709 12:27:43.722146  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 102153us
W20260709 12:27:43.714838  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.694916 12557 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:43.743698 12560 consensus_meta.cc:287] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.649s	user 0.012s	sys 0.002s
I20260709 12:27:43.741093  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:43.730049 12516 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:27:43.710830 12604 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 12:27:43.710206 12605 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:27:43.705178 12515 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:43.709579 12594 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:43.729704 12555 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:43.759164  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 54958.9us
I20260709 12:27:43.702087 12576 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:43.725003 12590 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:43.764176  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 96515.9us
I20260709 12:27:43.749956 12491 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:43.770473  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 95121.3us
I20260709 12:27:43.771915 12503 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:43.771466 12516 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:43.772636 12528 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:43.773813 12455 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.774161 12551 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:43.774101 12578 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:43.778307 12504 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:43.780620 12315 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:43.790650  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:43.797015  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:43.762636 12070 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:43.764539 12515 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:43.797986  5942 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 21: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:43.801261  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:43.800328  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:43.799108  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:43.802506  5944 raft_consensus.cc:2393] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 19: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:43.712353 12487 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:43.760107 12560 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:43.817795  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:43.818236  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:43.819285  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:43.822125  5868 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:43.795148 12607 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.824070  5872 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:43.824878  5865 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:43.825914  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:43.826663  5870 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:43.827018  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:43.820308 12311 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:43.802192 12070 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.829427  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:43.830597  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:43.831470  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:43.832834  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:43.845523  5866 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:43.845981  6778 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:43.833391  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:43.846979  6773 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:43.834697  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:43.835354  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:43.848990  6785 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:43.837939 12613 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.839272  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:43.840039  5874 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:43.840699  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:43.848404  5869 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:43.793434 12455 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:43.852104  6766 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:43.852977  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:43.854334  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:43.855495  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:54876 (request call id 1359) took 2339 ms. Trace:
I20260709 12:27:43.824573 12263 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:43.856396  6609 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:43.851933  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:43.858274  5865 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:43.858294  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:43.859351  5866 raft_consensus.cc:2410] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 21 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:27:43.856110  5874 rpcz_store.cc:276] 0709 12:27:41.512607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.617175 (+104568us) service_pool.cc:224] Handling call
0709 12:27:43.838649 (+2221474us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b5400027454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c7aa3d 00007f39d7c4acd0 00007f39d11cf254 00007f39d11cf1e2 00007f39d11cf111 00007f39d11cf08c 00007f39d11ce342 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:43.851765 (+ 13116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1353896320}
I20260709 12:27:43.861173  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:43.826689 12578 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:43.867098  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.846807 12070 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:43.879796 12263 raft_consensus.cc:697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:43.881111 12263 consensus_queue.cc:237] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:43.882642  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1617) took 1689 ms. Trace:
I20260709 12:27:43.832932 12607 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:43.883111  6015 rpcz_store.cc:276] 0709 12:27:42.193382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.193687 (+   305us) service_pool.cc:224] Handling call
0709 12:27:42.245836 (+ 52149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.687573 (+1441737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.882339 (+194766us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54001612d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:43.882566 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3282302208}
I20260709 12:27:43.884253  6775 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:43.856256 12158 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.823196 12154 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:43.868538  6773 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:27:43.843796 12491 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:27:43.869068 12528 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:43.874174 12531 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:43.866606 12588 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:43.930701  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:43.885433 12607 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:43.888645  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:43.899164 12603 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:43.910307  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:43.899333 12491 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:43.889747 12158 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:43.856841 12613 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:43.907809  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:43.915547 12455 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:43.915597 12560 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:43.929383  5869 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:43.894071 12548 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.660s	user 0.000s	sys 0.004s
I20260709 12:27:43.943701  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:43.886916 12487 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:43.964179  5874 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:43.970285  5868 raft_consensus.cc:2410] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:27:43.980386  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:43.988991  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:43.981877  5865 raft_consensus.cc:2368] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23.
I20260709 12:27:43.906498 12531 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:43.963541 12603 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:43.996420  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:43.996531  6757 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:43.995959  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:43.999224  5868 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
W20260709 12:27:43.994656  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.976856 12504 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:44.003767  5865 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:44.008097  6778 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:27:44.008621  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:44.012100  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:43.990159 12455 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.015865  6775 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:43.991405 12516 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:44.019416 12531 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:44.020551 12531 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:44.017488 12614 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:44.027196  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:43.994972 12557 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:27:44.034996  6768 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:44.033083  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2106) took 2491 ms. Trace:
I20260709 12:27:44.028621  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:44.037927  5946 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:44.052243  5915 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:44.054746  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:44.015560 12153 consensus_meta.cc:287] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.005s
I20260709 12:27:44.007717 12504 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.055790  6778 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:44.055598  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:44.058493  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.058652  5874 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:27:44.047842  5873 rpcz_store.cc:276] 0709 12:27:41.542119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.688016 (+145897us) service_pool.cc:224] Handling call
0709 12:27:41.699604 (+ 11588us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:41.704809 (+  5205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:41.704841 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:41.734504 (+ 29663us) log.cc:844] Serialized 26 byte log entry
0709 12:27:42.087417 (+352913us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:42.087474 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.087584 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.881119 (+1793535us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:44.032741 (+151622us) raft_consensus.cc:1685] finished
0709 12:27:44.032763 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.032987 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3911,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":3798,"spinlock_wait_cycles":6587648,"thread_start_us":273303,"threads_started":1,"wal-append.queue_time_us":1337134}
I20260709 12:27:44.059132  5915 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:44.059765 11977 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:44.038000 12154 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:44.057842 12391 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:44.038959 12603 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:44.038715 12594 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:44.057986 12516 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:44.071657 12391 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.073472 12391 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:44.073869 12603 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:44.073731 12516 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:44.075646 12613 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:44.076221 12311 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:44.065402 12153 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:44.076840 12311 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:44.077888  5871 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:44.077816 12311 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:44.080359  5871 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:44.080904 12608 raft_consensus.cc:3055] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:44.081418 12608 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:44.082363 12153 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:44.082576 12608 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:44.083902 12608 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:44.086920  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:44.013470 12560 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:27:44.089407  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:44.071578 12120 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:44.090464 12120 raft_consensus.cc:697] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:44.091287 12158 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:44.091429 12120 consensus_queue.cc:237] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:44.092726 12137 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:44.094831 12623 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:27:44.065827  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:44.066352 12092 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:44.059898 12504 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:44.067528  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:44.068198  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:44.068934  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:44.099370  5865 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:27:44.108067  6778 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:44.109030  6762 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:44.111990 12092 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:44.112389  5915 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:44.114555 12315 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:44.115198 12315 raft_consensus.cc:697] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:44.115672 12609 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.429s	user 0.006s	sys 0.000s
I20260709 12:27:44.120297  6773 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:27:44.122031  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:44.123874  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:44.124780  6785 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:44.033335 12515 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:44.127612 12572 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.071887 11977 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:44.130163  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: true dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:44.133966  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:44.136441  6785 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:44.136502  6773 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:44.137022  6785 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:44.042578 12455 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:44.146210  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:44.144830  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:44.141948 11977 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:44.116394 12315 consensus_queue.cc:237] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:44.116744 12560 consensus_queue.cc:1059] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:27:44.116976 12280 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:44.158104  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:44.149221  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:44.152405  5945 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:44.156231  5869 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 19: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:44.154783 12594 raft_consensus.cc:3055] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:44.186218 12594 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:44.187729  6793 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:44.196030 12594 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:44.198367 12594 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:44.146368 12572 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:44.205487 12572 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:44.206424 12280 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:44.209645  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:44.232571  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.213515  6793 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:27:44.225558  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:44.236191  5874 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:44.239060  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:44.240170  6757 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:44.240262  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:44.245913 12573 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:44.246747 12573 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:44.247985 12573 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:44.251031 12560 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:44.252300  5913 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:44.260190 12590 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:44.260911 12590 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.261471 12590 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:44.265585 12492 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:44.266326 12492 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:44.267539 12492 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:44.260993 12576 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:27:44.274060 12576 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:44.278199  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:44.282801 12492 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:44.229552  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 80477.2us
W20260709 12:27:44.295116  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68368.4us
I20260709 12:27:44.224802  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:44.101275 12541 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:44.149951 12575 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:44.183984  6785 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:44.350203  6785 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:27:44.310030 12614 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:27:44.312397  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.312980 12535 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:27:44.314913 12627 log.cc:927] Time spent T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 12:27:44.324023  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:44.321870  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 203982us
W20260709 12:27:44.376636  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.326633  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:44.377414  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 51687.9us
I20260709 12:27:44.331425 12070 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:44.343494 12092 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:44.346501  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.151743 12515 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:44.353268  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.364670  6773 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 25.
I20260709 12:27:44.224701 12503 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.225095 12517 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:44.379521  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:44.409014  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.227175 12391 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:44.418526  5872 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:44.421676  5868 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 23. Ops: []
I20260709 12:27:44.416026  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:44.468093  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 52163.1us
I20260709 12:27:44.425220 12503 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:44.426033 12582 consensus_queue.cc:1175] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6add331326ca47038308a447bf798fa7" responder_term: 21 status { last_received { term: 21 index: 14 } last_committed_idx: 14 last_received_current_leader { term: 21 index: 14 } }
I20260709 12:27:44.426911  5870 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:44.427256 12596 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:44.428444 12619 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.421s	user 0.000s	sys 0.002s
I20260709 12:27:44.486924  6793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
W20260709 12:27:44.429764 12628 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:27:44.431097 12537 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.129s
I20260709 12:27:44.432518 12136 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:44.440970 12515 raft_consensus.cc:3055] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:44.446898  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:44.398947 12568 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:44.449723 12541 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:44.383594 12614 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:44.452441  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:44.461724  5946 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:44.421880  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:44.464848  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:44.564406  5945 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:44.473403  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:44.369765 12535 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:44.482528  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:44.488780  6793 raft_consensus.cc:606] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:44.572091  6793 raft_consensus.cc:614] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:44.348171 12578 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.116s
W20260709 12:27:44.491346  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.495273  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:44.497813  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:44.415652 12310 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.517277  6773 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
W20260709 12:27:44.517235  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 257765us
I20260709 12:27:44.488307 12596 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:44.522433  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:44.529467  5941 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 12:27:44.578339  5944 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:44.485772 12137 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.484165 12572 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:44.598950  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:44.561012  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:44.600271  6773 raft_consensus.cc:2393] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 19: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:44.600353  5943 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:44.601884  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:44.602325  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:44.583271  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:44.603681  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:44.604473  6775 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 19: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:44.604899  5872 raft_consensus.cc:2393] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:44.516855 12515 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:44.584663  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:44.585237  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:44.555801  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:44.610313  5940 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:44.472633 12354 consensus_queue.cc:1059] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 12:27:44.592278  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:44.612327  5948 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:44.595810  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:44.473242 12517 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:44.553562 12568 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.605263  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:44.584307  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:44.614590  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:44.618427  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.623553  5942 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:44.618068  6785 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:27:44.625319  6762 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:44.626211  5867 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:44.591548 12596 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:44.646044  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:44.648092  6793 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:44.650451  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:44.632700  5945 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:44.630633  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:44.596658 12541 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:44.656528  5948 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 19: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:44.669714  6793 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 21: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:44.683705  5947 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:44.687970  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:44.585604 12590 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:44.621085 12568 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:44.617604 12572 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:44.704186  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 381330us
I20260709 12:27:44.705780  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:44.707791  5867 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:44.707991  6793 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:44.714735  6793 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 12:27:44.597920 12310 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:44.711444  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:44.720876  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:44.731575  6775 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: [18.10-20.11]
I20260709 12:27:44.670975 12541 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:44.735301  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:44.606966 12137 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:44.736177  5940 raft_consensus.cc:2410] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:27:44.732465 12516 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:44.778301  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:44.779289  5867 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:44.787900  5947 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 12:27:44.815891  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.683306 12535 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:44.766899 12280 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:44.819531 12480 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.126s	user 0.100s	sys 0.199s
I20260709 12:27:44.838583  6773 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:44.741241 12635 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:44.786640 12070 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:44.835994  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 285096us
I20260709 12:27:44.731060 12515 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:44.829456 12391 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:44.882660  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:44.869727 12480 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.174s	user 0.101s	sys 0.199s
I20260709 12:27:44.845471 12516 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:44.889277  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:44.894558  5947 raft_consensus.cc:2410] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:27:44.886137 12070 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:44.886646 12487 consensus_meta.cc:287] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.896s	user 0.008s	sys 0.006s
I20260709 12:27:44.877661 12503 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:44.880561 12635 raft_consensus.cc:697] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:44.864275 12280 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:27:44.880674 12517 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:44.889853 12535 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:44.889189 12570 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:27:44.912533 12637 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:27:44.912124 12635 consensus_queue.cc:237] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:44.907306 12503 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:44.907341 12636 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:44.917100 12636 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:44.917533 12636 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:44.918919 12570 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:44.919498  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:44.920447  5948 raft_consensus.cc:2368] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20.
I20260709 12:27:44.920109 12570 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:44.897589 12516 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:44.854388 12541 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:44.837046 12572 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:44.929730  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 507926us
I20260709 12:27:44.936110  5943 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:27:44.928056  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:44.942548  5865 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 16. Current term is 17. Ops: []
W20260709 12:27:44.942552  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 355440us
W20260709 12:27:44.937289 12573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.019s	sys 0.037s
I20260709 12:27:44.950198  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:44.951997  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.923815 12614 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:44.955510  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:44.956580  6015 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
W20260709 12:27:44.957791  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.959219  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:44.960052  5943 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:44.902112 12070 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:44.947100 12632 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.215s	user 0.000s	sys 0.005s
W20260709 12:27:44.952414 12641 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.050s	user 0.002s	sys 0.004s
I20260709 12:27:44.956156 12120 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:44.948156 12455 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:45.013139  5943 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 25.
I20260709 12:27:44.966776 12092 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.971699  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:44.974315  5949 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:44.974048  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:44.974691  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:44.976176  5867 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 15.
I20260709 12:27:44.976219 12516 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:44.988912  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:45.030095  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:45.030455  6024 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:45.030460  5838 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:44.994376 12643 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:27:45.031320  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:45.030045 12092 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:45.032660  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:44.989689 12614 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:45.034317 12092 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:45.034544 12492 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:45.035145 12492 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:45.035537 12492 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:45.034382 12614 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:45.002220 12310 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:45.003310 12391 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.037189  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:45.012106  6762 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
W20260709 12:27:44.953446 12573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.019s	sys 0.038s
W20260709 12:27:45.020910 12603 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.947s	user 0.004s	sys 0.010s
I20260709 12:27:45.031193  6022 raft_consensus.cc:2368] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19.
I20260709 12:27:45.038053  6021 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:45.038632  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:45.039120  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.078907  6018 raft_consensus.cc:2410] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
W20260709 12:27:44.957698 12633 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.107s	user 0.005s	sys 0.000s
I20260709 12:27:45.025954  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.080052  6019 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:45.005940 12590 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:45.036662 12614 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:27:45.022466 12531 consensus_meta.cc:287] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.002s	user 0.009s	sys 0.005s
I20260709 12:27:45.048501  5838 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:45.037658  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:45.096300  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 62703.7us
I20260709 12:27:45.050626  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.088961  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:45.097815  6017 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:45.096531  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:45.098634  5949 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:45.094993  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:45.100123  6024 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:45.051155 11835 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:45.071790 12158 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:45.101693  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 53327.4us
I20260709 12:27:45.021528 12120 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:45.103783  6766 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:27:45.106468  5838 leader_election.cc:400] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20.
I20260709 12:27:45.106818  5838 leader_election.cc:403] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:45.104080  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:45.090797 11977 consensus_meta.cc:287] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.940s	user 0.020s	sys 0.000s
I20260709 12:27:45.094669 12614 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:45.114161  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:45.115121  6018 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:45.123526  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.124810  6022 raft_consensus.cc:2410] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:27:45.071431 12391 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:45.105727 12310 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:45.105357 12603 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:45.136888  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.105383 12158 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:45.138808  6019 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
W20260709 12:27:45.128641 12504 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.004s	user 0.014s	sys 0.001s
I20260709 12:27:45.150079  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.113745 12354 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:45.144979  6017 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:45.158102  6016 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 19: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:27:45.159875  5988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.000s	sys 0.028s
W20260709 12:27:45.160498  5988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.000s	sys 0.028s
I20260709 12:27:45.160640  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:45.121999 12581 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:45.161664  6022 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:27:45.147176  6021 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:45.107189 12479 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
W20260709 12:27:45.170876  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 450029us
I20260709 12:27:45.114290 11977 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:45.167490  5949 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:45.149885 12310 raft_consensus.cc:697] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:45.150367 12158 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:45.137084 12603 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.181355  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:45.191691  6019 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:45.195686  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:45.130353 12531 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:27:45.171411  6018 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:45.203006  5987 leader_election.cc:400] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23.
I20260709 12:27:45.208240  5987 leader_election.cc:403] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:45.208544  6021 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:45.206032  5949 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:45.156845 12504 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:45.208596  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: true dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:45.214780  6015 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:45.216063 12504 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:45.221506  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:45.181190 12647 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.231935  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:45.200752 12603 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:45.239464  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.240453  6023 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:45.237495  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:45.237084  5949 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:45.207530 12516 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:27:45.231199 12631 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:27:45.252453  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.253357  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:45.250859  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 144504us
I20260709 12:27:45.271148  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:45.277110  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:45.248354 12070 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:45.245419 12642 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:45.293821  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1320) took 1214 ms. Trace:
I20260709 12:27:45.295895  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:45.299110  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:45.299757  6768 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:45.296772  5872 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:45.236774 12568 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:45.294512  6756 rpcz_store.cc:276] 0709 12:27:44.067278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.069975 (+  2697us) service_pool.cc:224] Handling call
0709 12:27:44.071615 (+  1640us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:44.109838 (+ 38223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:44.109886 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:44.147144 (+ 37258us) log.cc:844] Serialized 52 byte log entry
0709 12:27:44.466183 (+319039us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:44.487379 (+ 21196us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.493865 (+  6486us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.507967 (+ 14102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.508099 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.281337 (+773238us) raft_consensus.cc:1685] finished
0709 12:27:45.281372 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.281612 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26078,"prepare.queue_time_us":119586,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":116,"raft.queue_time_us":105136,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":8531,"spinlock_wait_cycles":43965696,"thread_start_us":310470,"threads_started":1,"wal-append.queue_time_us":576638}
W20260709 12:27:45.302757  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 179413us
W20260709 12:27:45.305538  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 345257us
I20260709 12:27:45.313848  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:45.271842 12516 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:45.319483  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:45.319612  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:45.315239  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:45.292397 12136 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.776s	user 0.011s	sys 0.004s
I20260709 12:27:45.272099 12653 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.322108  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.331625  6023 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
W20260709 12:27:45.306777 12640 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:27:45.242653 12647 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:45.301457 12638 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:45.335623  5865 raft_consensus.cc:2393] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:45.285422 12655 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:45.324954  5987 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:45.346561  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:45.320899 12503 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:45.358505  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.358347 12503 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:45.361029 12503 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:45.316838 12570 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:27:45.353112  5947 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:27:45.364341 12568 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:45.366112  6016 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:27:45.367355 12647 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:45.268105 12659 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 12:27:45.373533  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:45.332798 12642 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:45.356904  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:45.355482  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.381994  6024 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:27:45.333253 12120 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:45.266505 12310 consensus_queue.cc:237] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:45.388345  6023 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:45.347074 12655 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:45.350944 12517 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:45.351794 12136 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.354280 12475 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.393285  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:45.325609 12504 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.061s
W20260709 12:27:45.403294  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 122539us
I20260709 12:27:45.404706  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.374949 12391 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:45.417326  5949 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:45.410506  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.419160  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:45.392747  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 85163.8us
I20260709 12:27:45.420984  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:45.402143 12667 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:45.425525  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.426738 12531 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:45.427744 12531 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:27:45.428236 12636 consensus_meta.cc:287] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.510s	user 0.015s	sys 0.000s
I20260709 12:27:45.428910 12636 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:45.430689  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:45.432068 12603 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:45.433516 12570 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:45.434250 12649 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
I20260709 12:27:45.434569  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:45.431815 12636 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:45.452105  6022 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:45.346580 12581 raft_consensus.cc:697] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:45.455440  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:45.490425  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 55973.9us
I20260709 12:27:45.491252  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.444571  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:45.404822 12487 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:45.406138 12655 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:45.502981  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:45.504002  6024 raft_consensus.cc:2393] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:45.388538 12568 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:27:45.489532  5990 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:45.491974  6022 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:27:45.415715 12517 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:45.453482 11977 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:45.502354  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 57876.1us
I20260709 12:27:45.380795 12479 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 12:27:45.340904 12653 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:45.458671 12667 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:45.505177  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:45.519558  5840 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:45.507843  5990 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:45.415086 12136 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:45.440454 12475 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:45.531502  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:45.532321  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.532572  6023 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 23: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:45.532356  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:45.535084  5867 raft_consensus.cc:2410] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 20 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 12:27:45.535054  6022 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:45.539161  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:45.542580  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:45.515121 12310 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:45.514323 12642 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:45.505183 12504 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.530285 12570 raft_consensus.cc:3055] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:45.528666 12531 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:45.552302  5990 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:45.533491 12535 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:45.548260 12651 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.279s	user 0.000s	sys 0.002s
I20260709 12:27:45.528887 12667 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:45.554792  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.560470  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.547940 12666 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:45.548177 12657 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 12:27:45.620637  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 69648.1us
I20260709 12:27:45.529085 12568 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:27:45.622148  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 69902.8us
I20260709 12:27:45.723155  5990 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:45.562898  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.562340 12504 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:45.626170 12535 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:45.623147 12659 raft_consensus.cc:2764] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:27:45.633543 12676 consensus_queue.cc:1059] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
W20260709 12:27:45.634275 12674 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:27:45.624171 12636 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.634792 12581 consensus_queue.cc:237] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:45.635226  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:45.636610 12664 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
W20260709 12:27:45.636852 12650 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.350s	user 0.000s	sys 0.002s
I20260709 12:27:45.642637 12154 raft_consensus.cc:3055] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:45.644248 12455 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:45.645193 12653 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:45.645839 12671 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:27:45.548353 12673 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:45.651163  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:45.553898 12570 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:45.792039 12479 raft_consensus.cc:3055] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:45.792598 12479 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:45.792423 12570 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:45.755558 12503 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:45.794649  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.729530  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:45.730881  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 51562.4us
I20260709 12:27:45.628190 12667 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:45.722285 12568 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.798756 12568 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:45.800863 12603 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:45.801115  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:45.763437 12670 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:27:45.804282  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:45.806504  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1381) took 1095 ms. Trace:
I20260709 12:27:45.793532 12479 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:45.807814  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:45.806911  5866 rpcz_store.cc:276] 0709 12:27:44.711692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.711972 (+   280us) service_pool.cc:224] Handling call
0709 12:27:44.712406 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:44.725823 (+ 13417us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:45.115951 (+390128us) log.cc:844] Serialized 14 byte log entry
0709 12:27:45.312900 (+196949us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:45.322938 (+ 10038us) log.cc:844] Serialized 26 byte log entry
0709 12:27:45.334378 (+ 11440us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:45.335482 (+  1104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.335686 (+   204us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.806243 (+470557us) raft_consensus.cc:1685] finished
0709 12:27:45.806267 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.806432 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":505309,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":65112,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":28857,"thread_start_us":515006,"threads_started":2,"wal-append.queue_time_us":400621}
I20260709 12:27:45.766238 12535 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:45.808398 12479 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:45.809231 12679 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:27:45.727871 12680 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:45.786764 12681 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 12:27:45.776309 12596 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.075s	user 0.006s	sys 0.011s
I20260709 12:27:45.810671  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1704) took 1672 ms. Trace:
I20260709 12:27:45.810881 12596 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:45.811134  5941 rpcz_store.cc:276] 0709 12:27:44.137886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.138041 (+   155us) service_pool.cc:224] Handling call
0709 12:27:44.182621 (+ 44580us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:44.596849 (+414228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.810353 (+1213504us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b540055ffd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:45.810605 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":731115904}
I20260709 12:27:45.813434  6020 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 19: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:27:45.780295 12660 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.391s	user 0.001s	sys 0.000s
W20260709 12:27:45.813907 12354 consensus_meta.cc:287] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.648s	user 0.010s	sys 0.003s
I20260709 12:27:45.815340  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:45.816179  6020 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:45.816737  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:45.814427 12354 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20.
I20260709 12:27:45.817440  5942 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:45.773407 12136 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:45.739463 12557 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:45.734925 12517 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:45.786728  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 135703us
I20260709 12:27:45.788900 12558 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:45.822738 12555 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.835750 12555 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:45.797196  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 67794.1us
I20260709 12:27:45.793875 12570 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:45.770956 12154 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:45.760111 12636 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:45.763628 12665 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 12:27:45.817543  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:45.817920  6017 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:45.777045  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1376) took 1210 ms. Trace:
I20260709 12:27:45.757664  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:45.820271  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1374) took 1305 ms. Trace:
W20260709 12:27:45.775430 12492 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.739s	user 0.016s	sys 0.000s
I20260709 12:27:45.821063  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:45.862752  5942 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:45.722919 12666 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:45.821380  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:45.817544  6020 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:45.808651  5945 raft_consensus.cc:2368] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20.
I20260709 12:27:45.759732 12310 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:45.867094  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 45816.8us
W20260709 12:27:45.854350 12158 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.650s	user 0.012s	sys 0.002s
I20260709 12:27:45.868109 12158 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:45.868897  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:45.869915 12647 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:45.828439  6019 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:45.870553 12647 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.871054 12647 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:45.871634 12158 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:45.872136  6019 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:27:45.873204  6019 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:45.874171  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:45.829125  6022 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:45.830034  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1749) took 1308 ms. Trace:
I20260709 12:27:45.875753  6773 rpcz_store.cc:276] 0709 12:27:44.521194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.603324 (+ 82130us) service_pool.cc:224] Handling call
0709 12:27:45.829828 (+1226504us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b54002a0154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:45.829971 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":392685952}
I20260709 12:27:45.776791 12455 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:45.877243 12659 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:45.877786 12659 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.878304 12659 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:45.880056 12455 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:45.833714 12070 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:45.882495 12070 raft_consensus.cc:3055] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:45.882982 12070 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:45.884262 12070 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:45.887770  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.834022 12596 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:45.842994  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:45.851933  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:45.823226 12642 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:45.856139  5869 rpcz_store.cc:276] 0709 12:27:44.567087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.567312 (+   225us) service_pool.cc:224] Handling call
0709 12:27:44.595878 (+ 28566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:44.596445 (+   567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:44.596490 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:44.631532 (+ 35042us) log.cc:844] Serialized 26 byte log entry
0709 12:27:45.076002 (+444470us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:45.076369 (+   367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.076482 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.776709 (+700227us) raft_consensus.cc:1685] finished
0709 12:27:45.776743 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.776946 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19575,"raft.queue_time_us":292,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":11314,"thread_start_us":336228,"threads_started":1,"wal-append.queue_time_us":490397}
I20260709 12:27:45.831380 12503 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:45.926867 12487 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23.
I20260709 12:27:45.927361 12666 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:45.858426  5870 rpcz_store.cc:276] 0709 12:27:44.515376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.515585 (+   209us) service_pool.cc:224] Handling call
0709 12:27:44.580522 (+ 64937us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:44.593300 (+ 12778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:44.593373 (+    73us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:44.615353 (+ 21980us) log.cc:844] Serialized 26 byte log entry
0709 12:27:45.105280 (+489927us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:45.105343 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.105452 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.820002 (+714550us) raft_consensus.cc:1685] finished
0709 12:27:45.820048 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.820209 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61822,"raft.queue_time_us":13052,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":11340,"thread_start_us":417683,"threads_started":1,"wal-append.queue_time_us":581506}
I20260709 12:27:45.929448 12642 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:45.930061 12642 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:45.930953 12475 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:45.911700  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:45.921164 12557 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.934811  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:45.935037 12655 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:45.936075  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:45.936861  5945 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:45.937568  6022 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:45.938242 12608 raft_consensus.cc:3055] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:45.938678 12608 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:45.939626 12608 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:45.940419  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:45.941015 12608 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:45.913169 12310 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:45.942245 12555 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:45.941036  6022 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:45.943347  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:45.944203  5873 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:45.914800  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:45.915089  5987 leader_election.cc:400] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20.
I20260709 12:27:45.950462  5987 leader_election.cc:403] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:45.950485  5941 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:45.923777  6024 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 12:27:45.919600  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 76737.8us
I20260709 12:27:45.922595 12455 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:45.916781 12492 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:45.923897 12614 consensus_meta.cc:287] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.792s	user 0.007s	sys 0.007s
I20260709 12:27:45.954856 12614 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:45.955096 12686 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:45.955967 12653 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20.
I20260709 12:27:45.956264 12492 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:45.959515  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:45.957614 12311 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.611s	user 0.017s	sys 0.047s
I20260709 12:27:45.963711  6022 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:45.975785  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.976824  6022 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 25.
W20260709 12:27:45.969334 12311 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.623s	user 0.018s	sys 0.047s
I20260709 12:27:45.956774 12686 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 12:27:45.961524 12576 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.984660  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:45.990963  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:45.987102  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:27:45.988087  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.966890 12070 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:45.988888 12647 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:45.982510  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:45.993897  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:45.985059  6609 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:45.998854  6020 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:45.977391 12555 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:27:45.996542 11977 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:46.007376 11977 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.996587 12576 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:46.007906 11977 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:46.008591  5947 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:46.008886 12537 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:46.009449 12537 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.009989 12614 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:46.010013 12537 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:46.011404  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:45.934828 12557 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:46.006517 12555 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:46.016495  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:46.014000 12680 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:46.025465 12555 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:45.857973 12154 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:46.031917  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:45.990373 11835 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:46.024034  5915 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:46.044219  5941 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:27:46.036553  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.045183  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.046718  6022 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:46.050706  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:46.015369 12515 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:46.017910 12455 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:46.053975  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:46.056138  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:46.056943  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.057344 12653 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:46.057967  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:46.027863 12614 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.027740 12311 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:46.063367 12667 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:46.066444  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.067225  6020 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:46.068250  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:46.068625 12667 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:46.068985  6020 raft_consensus.cc:2368] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 22. Current term is 23.
I20260709 12:27:46.069753 12680 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:46.054041 12391 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:46.070926 12638 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.703s	user 0.013s	sys 0.000s
I20260709 12:27:46.071635 12638 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:46.063733 12653 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.072342 12653 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:46.075508 12541 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:46.064584  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:46.079563  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.088624 12503 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:46.090598  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:46.091650  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 2046) took 1075 ms. Trace:
I20260709 12:27:46.092303  5949 rpcz_store.cc:276] 0709 12:27:45.016525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.241726 (+225201us) service_pool.cc:224] Handling call
0709 12:27:46.091354 (+849628us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400673ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:46.091568 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374711552}
I20260709 12:27:46.096423  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.098485 12680 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:46.105007  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1317) took 2038 ms. Trace:
W20260709 12:27:46.105186  5838 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.195:43329 timed out after 1.859s (SENT) [suppressed 25 similar messages]
I20260709 12:27:46.105515  6766 rpcz_store.cc:276] 0709 12:27:44.066409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.067465 (+  1056us) service_pool.cc:224] Handling call
0709 12:27:44.067783 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:45.106393 (+1038610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.104603 (+998210us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002a0dd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:46.104929 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2376911744}
I20260709 12:27:46.106328 12636 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:46.049355 12154 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:46.100972  5944 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:46.110551  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:46.112392  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:46.116351  6022 raft_consensus.cc:2376] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20. Re-sending same reply.
I20260709 12:27:46.112416  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.091755  5941 raft_consensus.cc:2410] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:27:46.060561 12596 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:46.122826  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2032) took 1229 ms. Trace:
I20260709 12:27:46.155370  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.156776  6016 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:46.159466  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:46.159497  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.161661  6022 raft_consensus.cc:2376] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20. Re-sending same reply.
I20260709 12:27:46.163444  5949 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:46.112221 12503 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:46.165479  5944 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:46.168063  6020 raft_consensus.cc:2368] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19.
I20260709 12:27:46.161780  6023 raft_consensus.cc:2376] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18. Re-sending same reply.
I20260709 12:27:46.053812 11835 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:46.182977  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:46.184276  6016 raft_consensus.cc:2376] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19. Re-sending same reply.
I20260709 12:27:46.155519 12608 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.188259  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:46.124943 12653 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:46.156895 12673 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:46.189092  6016 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:46.200707  6016 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:46.189975 12315 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.765s	user 0.019s	sys 0.013s
I20260709 12:27:46.191244  6018 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:46.113430 12614 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:46.214363  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.215649  6020 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:46.128108 12642 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:46.113672 12555 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:46.221061  5871 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:46.206559 12315 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.792s	user 0.020s	sys 0.013s
I20260709 12:27:46.199584 12581 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.236719  5948 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:46.239715  6020 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:46.244755  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 193347us
I20260709 12:27:46.201362 12608 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:46.246759  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:46.256105  5840 leader_election.cc:400] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 22. Current term is 23.
I20260709 12:27:46.256619  5840 leader_election.cc:403] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:27:46.259656  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:46.270345  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.302443  5840 leader_election.cc:400] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19.
I20260709 12:27:46.302977  5840 leader_election.cc:403] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 12:27:46.306100 12689 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:27:46.306667  6019 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
I20260709 12:27:46.307214  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:46.307485 12608 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:46.308035  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:46.308727  6019 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:46.310994 12614 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:46.311691 12565 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19.
I20260709 12:27:46.312522 12565 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:46.312882 12565 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.313186 12565 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:46.238620 12581 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:46.315984 12581 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:46.317200 12687 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:27:46.319059 11835 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:46.319264 12647 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:46.320997 12653 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:46.322115 12537 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:46.323611 12644 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 1.147s	user 0.000s	sys 0.007s
I20260709 12:27:46.324847 12537 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:46.328322 12557 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:46.330475 12642 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:46.331231 12690 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:27:46.336380  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.254304 12555 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:46.261762 12572 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Leader election lost for term 22. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 22. Current term is 23.
I20260709 12:27:46.267923 12667 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:46.343152  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 229024us
I20260709 12:27:46.362258  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:46.302045 12153 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:46.333105 12670 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:27:46.261929 12578 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.090s
I20260709 12:27:46.268189 12542 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.084s
I20260709 12:27:46.368016  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:46.341948 12581 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.342320 12537 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.343801 12535 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:46.377305  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:46.387905  6611 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:46.338428 11835 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:27:46.393275  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:46.262038 12659 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:46.366637 12589 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.925s	user 0.012s	sys 0.025s
I20260709 12:27:46.366011 12667 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:46.429965  6019 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:27:46.374873  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.385121  5915 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:46.369062 12153 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.436760  6016 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:27:46.439992  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:46.441331  6016 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:46.441761  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56734.2us
I20260709 12:27:46.382779 12311 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:27:46.406806 12602 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.971s	user 0.014s	sys 0.000s
W20260709 12:27:46.432458 12589 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.991s	user 0.012s	sys 0.026s
I20260709 12:27:46.463198  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.433583 12608 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:46.369463 12565 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:46.398626 12666 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:46.451426  5915 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:46.468742  5990 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:46.432224 12315 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:46.433470 12673 raft_consensus.cc:2764] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
W20260709 12:27:46.476752  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.389604 12537 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:46.402119 12535 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:46.485805 12695 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:27:46.491631 12565 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:46.441763 12492 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:27:46.451046 12515 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:46.533510  5949 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:46.474433 12613 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
W20260709 12:27:46.521411 12551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.013s	sys 0.015s
W20260709 12:27:46.565661 12551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.013s	sys 0.015s
I20260709 12:27:46.566977 12504 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:46.568301 12537 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:46.510541 12391 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:46.568470 12551 raft_consensus.cc:3055] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:46.513280 12666 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.401073 12570 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.079s
I20260709 12:27:46.547677 12475 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:46.561796 12455 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.572031 12475 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:46.572916  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1388) took 1606 ms. Trace:
I20260709 12:27:46.572734 12455 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:46.380121 12614 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 12:27:46.573318  5865 rpcz_store.cc:276] 0709 12:27:44.966552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.973777 (+  7225us) service_pool.cc:224] Handling call
0709 12:27:46.572673 (+1598896us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54005d6bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:46.572845 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":695844224}
I20260709 12:27:46.574647 12475 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:46.574779  5867 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 12:27:46.574998 12568 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.776s	user 0.011s	sys 0.002s
I20260709 12:27:46.569548 12391 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:46.575553 12568 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:46.575953 12455 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:46.576380  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1597) took 1046 ms. Trace:
I20260709 12:27:46.576742  5867 rpcz_store.cc:276] 0709 12:27:45.530294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.538476 (+  8182us) service_pool.cc:224] Handling call
0709 12:27:46.574532 (+1036056us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:46.575337 (+   805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.576186 (+   849us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54005d6bd0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:46.576321 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2351187200}
W20260709 12:27:46.513952  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 224283us
I20260709 12:27:46.577338 12391 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:46.569801 12515 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.578156 12515 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:46.578873 12614 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:46.579341 12614 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:46.515548  5946 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
W20260709 12:27:46.580240  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:46.581272  5946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 2042) took 1608 ms (client timeout 1859 ms). Trace:
I20260709 12:27:46.450363 12655 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:46.549819 12691 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:27:46.519448  6016 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
W20260709 12:27:46.582487  6768 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1316) took 2516 ms (client timeout 1856 ms). Trace:
I20260709 12:27:46.438575 12136 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:27:46.583248  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.584188 12576 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:46.584714  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.584964 12136 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.566026 12613 consensus_queue.cc:1243] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:27:46.585858  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:46.586735  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.586813  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2047) took 1559 ms. Trace:
I20260709 12:27:46.545068 12480 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:46.587427 12492 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:46.587960  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1629) took 1415 ms. Trace:
I20260709 12:27:46.588564  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2041) took 1616 ms. Trace:
I20260709 12:27:46.588276 12492 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:46.589010  5948 rpcz_store.cc:276] 0709 12:27:44.972632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.973789 (+  1157us) service_pool.cc:224] Handling call
0709 12:27:44.974198 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:46.237152 (+1262954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.588350 (+351198us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54008b06d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:46.588508 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2902931712}
I20260709 12:27:46.585541 12535 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:46.590775 12492 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:46.588002 12480 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.591684 12480 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:46.594786 12666 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:46.596730 12683 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.790s	user 0.001s	sys 0.001s
I20260709 12:27:46.587180  5947 rpcz_store.cc:276] 0709 12:27:45.027837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.364983 (+337146us) service_pool.cc:224] Handling call
0709 12:27:45.365408 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:46.009103 (+643695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.586568 (+577465us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b5400633654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:46.586749 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1478482816}
W20260709 12:27:46.521764  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 181565us
W20260709 12:27:46.510811 11977 consensus_meta.cc:287] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.502s	user 0.014s	sys 0.000s
W20260709 12:27:46.462654 12602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.027s	user 0.018s	sys 0.000s
I20260709 12:27:46.450601 12479 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 12:27:46.601379  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:46.604892  5947 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:46.475648 12311 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:46.617395 12475 raft_consensus.cc:3055] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:46.617657  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.618084 12475 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:46.618379 12311 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:46.619215 12475 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:46.620582 12475 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:46.620890 12479 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:46.621345 12479 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:46.549497  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2049) took 1129 ms. Trace:
I20260709 12:27:46.386999 12581 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:46.622248 12479 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:46.623486 12479 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:46.623965 12581 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:46.511507  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 68935.3us
I20260709 12:27:46.542722  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1538) took 1526 ms. Trace:
I20260709 12:27:46.626047  6015 rpcz_store.cc:276] 0709 12:27:45.016749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.016898 (+   149us) service_pool.cc:224] Handling call
0709 12:27:45.025323 (+  8425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:45.215345 (+190022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.542375 (+1327030us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400562a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:46.542626 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434989312}
I20260709 12:27:46.626804  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1628) took 1455 ms. Trace:
I20260709 12:27:46.627226  6021 rpcz_store.cc:276] 0709 12:27:45.171936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.178621 (+  6685us) service_pool.cc:224] Handling call
0709 12:27:45.208324 (+ 29703us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:45.209148 (+   824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.626575 (+1417427us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b5400764b50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:46.626738 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67282176}
I20260709 12:27:46.627935  6015 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:46.628636  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1564) took 1191 ms. Trace:
I20260709 12:27:46.628795 11977 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:46.628952  6015 rpcz_store.cc:276] 0709 12:27:45.437410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.627317 (+1189907us) service_pool.cc:224] Handling call
0709 12:27:46.627738 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:46.628376 (+   638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.628582 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:46.546348  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 1562) took 1109 ms. Trace:
I20260709 12:27:46.438118 12667 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:46.555044 12659 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:46.519421 12503 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:27:46.514036  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 45382.1us
I20260709 12:27:46.585417  6016 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
W20260709 12:27:46.581729  5946 rpcz_store.cc:269] 0709 12:27:44.973064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.974410 (+  1346us) service_pool.cc:224] Handling call
0709 12:27:46.581070 (+1606660us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400580554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:46.581203 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3240285440}
I20260709 12:27:46.645087  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:46.569546 12682 log.cc:927] Time spent T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Append to log took a long time: real 0.791s	user 0.002s	sys 0.000s
I20260709 12:27:46.548553 12137 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:46.582976  6768 rpcz_store.cc:269] 0709 12:27:44.066101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.089071 (+ 22970us) service_pool.cc:224] Handling call
0709 12:27:46.582287 (+2493216us) spinlock_profiling.cc:243] Waited 986 ms on lock 0x7b54002a12d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:46.582405 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2268573568}
I20260709 12:27:46.569809 12551 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:46.631923 11977 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:46.442787 12153 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:46.605657  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:46.606251  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:46.653277  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 71661.4us
I20260709 12:27:46.608224  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:46.653559  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:46.654042  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:46.128553  5940 rpcz_store.cc:276] 0709 12:27:44.893118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.893334 (+   216us) service_pool.cc:224] Handling call
0709 12:27:44.900202 (+  6868us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.900463 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:45.175714 (+275251us) log.cc:844] Serialized 14 byte log entry
0709 12:27:46.121648 (+945934us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b18001ff1e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39d282b4bc 00007f39cfb7948b 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a
0709 12:27:46.122171 (+   523us) log.cc:844] Serialized 14 byte log entry
0709 12:27:46.122395 (+   224us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:46.122447 (+    52us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:46.122510 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.122595 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:46.122746 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":389149,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":5401,"spinlock_wait_cycles":99906304,"thread_start_us":416302,"threads_started":2,"wal-append.queue_time_us":621410}
I20260709 12:27:46.484138 12608 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.628826  6021 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:27:46.656848  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:46.657223  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1566) took 1186 ms. Trace:
I20260709 12:27:46.633159  5941 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:27:46.632727  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:46.621863  5945 rpcz_store.cc:276] 0709 12:27:45.419649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.077735 (+658086us) service_pool.cc:224] Handling call
0709 12:27:46.078164 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:46.078581 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:46.078611 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:46.079135 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 12:27:46.248466 (+169331us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:46.248526 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.248627 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:46.549125 (+300498us) raft_consensus.cc:1685] finished
0709 12:27:46.549168 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:46.549418 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1229,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":95,"thread_start_us":168886,"threads_started":1,"wal-append.queue_time_us":348908}
W20260709 12:27:46.635622  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:46.638398  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 198411us
W20260709 12:27:46.647704 12672 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.846s	user 0.004s	sys 0.000s
W20260709 12:27:46.597856 12070 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.002s
I20260709 12:27:46.588451  6017 rpcz_store.cc:276] 0709 12:27:45.172377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.178377 (+  6000us) service_pool.cc:224] Handling call
0709 12:27:45.817737 (+639360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:45.855465 (+ 37728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.587777 (+732312us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b5400160650. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:46.587904 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1469099776}
I20260709 12:27:46.645188  5990 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:46.645167  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:46.665494  5872 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 26.
I20260709 12:27:46.570727  5838 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:46.667321  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:46.648011  5866 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:27:46.648097  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:46.668928  5948 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:46.648713  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:46.649101  6773 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:46.562202  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:46.650842  6768 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:46.546031 12565 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:46.652268  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:46.655467  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:46.656934  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:46.657933  5949 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:46.636792  6018 rpcz_store.cc:276] 0709 12:27:45.436882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.634890 (+198008us) service_pool.cc:224] Handling call
0709 12:27:46.546146 (+911256us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b5400464454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:46.546279 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":686593792}
I20260709 12:27:46.654417  6793 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 25.
I20260709 12:27:46.658228  6021 rpcz_store.cc:276] 0709 12:27:45.472214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.628183 (+1155969us) service_pool.cc:224] Handling call
0709 12:27:46.628478 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:46.656933 (+ 28455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.657166 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:46.661201  6611 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:46.687597  6766 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 23: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:46.661342  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:46.499102  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.659981  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:46.663431  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:46.641825 12568 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:46.702960  6778 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:27:46.701848  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 131158us
I20260709 12:27:46.705193  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:46.695969  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 50865.6us
I20260709 12:27:46.672730  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:46.674572  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:46.678036  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:46.678824  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:46.683967  6017 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:27:46.710192  6773 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 26: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:46.684898  6020 raft_consensus.cc:2410] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:27:46.647334 12315 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.662170  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:46.714500  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 52430.7us
I20260709 12:27:46.687474  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:46.691208  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:46.716228  5871 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:46.690469  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:46.694764  6019 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:46.585569 12136 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:46.720238  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:46.651108 12602 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.651239 12572 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:46.707520  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:46.692924  6611 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:46.631992 12092 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:27:46.663743 12608 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:46.647397 12535 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:46.651149 12354 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:46.721436  5870 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:46.697897  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:46.721716  5990 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:46.724238  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:46.712844  6015 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:27:46.663690 12581 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:46.736898  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:46.762394  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:46.762050  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 55625.2us
I20260709 12:27:46.770349  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:46.734660  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:46.771884  6756 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:46.772502  6019 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:46.721721 11835 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:46.725075 12487 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.721455 12596 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:46.580260 12614 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:46.724721 12557 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:46.793757  6019 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:27:46.698798 12576 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.711977 12565 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:46.714829 12541 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:46.759490 12647 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:46.806113  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:46.834604  6785 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 25.
I20260709 12:27:46.702099 12503 consensus_queue.cc:1059] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 12:27:46.821439  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:46.838409  6015 raft_consensus.cc:2376] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19. Re-sending same reply.
I20260709 12:27:46.768062 12581 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:46.778443 12572 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:46.721593 12659 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.108s
I20260709 12:27:46.784999 12653 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:46.763341 12092 consensus_queue.cc:1243] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:46.836592 12541 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:46.831552 12672 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:27:46.767553 12535 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:46.725386 12315 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:46.723137 12504 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.105s
I20260709 12:27:46.788254 12492 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:46.867296  6756 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:46.868549  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:46.898116  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:46.898402  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:46.899161  6756 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
W20260709 12:27:46.871526  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 156037us
I20260709 12:27:46.900677  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:46.857407 12565 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:46.761974 12280 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.078s
W20260709 12:27:46.859967 12154 consensus_meta.cc:287] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.732s	user 0.015s	sys 0.000s
I20260709 12:27:46.832243 12070 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:46.889837 12555 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.001s
I20260709 12:27:46.828243 12712 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:46.824843 11835 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:46.834177 12647 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.787998 12568 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 12:27:46.899300  6785 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:46.895985 12713 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:46.947772  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:46.948796  6785 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:46.774725 12354 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:46.860242 12581 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:46.871214  6768 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:46.914319 12653 raft_consensus.cc:3055] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:46.956210  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:46.957065  6768 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 23.
I20260709 12:27:46.919457 12516 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:46.825137 12455 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:27:46.824479 12487 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:46.915807  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 194193us
I20260709 12:27:46.922863 12158 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:46.919473  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 226619us
I20260709 12:27:46.826246 12557 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:46.772038 12551 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:46.921581  5872 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 23.
I20260709 12:27:46.837045 12576 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:46.778080 12602 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:46.936542  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 165230us
I20260709 12:27:46.950580  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:46.969766  5990 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:46.917057 12712 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.958252  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:46.913558  6756 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 23: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:46.919860 12589 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:46.951145 12647 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:46.983546  5990 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:46.958977 12516 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:27:46.993350  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.995877  6768 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:46.997068  6024 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:47.002494  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:47.005725 12589 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:47.011701 12589 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:46.974155 12655 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:27:47.018970  6019 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:27:46.988413  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:47.026134  6785 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:46.960525 12653 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:46.995826 12492 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.995262 12712 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:47.037093  6023 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 22. Current term is 23. Ops: [19.9-22.10]
I20260709 12:27:47.045138  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:47.046403  6785 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:47.048275  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:47.049726  6785 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 26: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:27:47.051097  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.996044 12503 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:47.043298  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:47.067971  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:47.071663  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:46.971508 12158 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:47.078495  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 76061.2us
I20260709 12:27:47.080448  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:47.085466  6785 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 26.
I20260709 12:27:47.000864 12596 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 12:27:46.967080 12541 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.10, Last appended by leader: 9, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:47.003023 12708 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 12:27:47.043916 12504 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:47.096681  6768 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:47.087800  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:47.070214 12092 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:47.021870 12137 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 12:27:47.106531  6768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:47.092288  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:47.067670 12136 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:47.109154  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 192264us
I20260709 12:27:47.098341 12713 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:47.112612  6785 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:47.095299 12659 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:27:47.052479 12492 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:47.087598 12721 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:47.113118  6768 raft_consensus.cc:606] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:27:47.165119  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:47.166785 12492 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:47.106091 12653 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:47.166862  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.168486  6768 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:47.168766 12653 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:47.022526 11835 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:47.124552  5871 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:47.125110  6019 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:27:47.171264  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:47.172128  6019 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 18: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:47.114619 12092 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:47.140601 12153 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:47.177857  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.153411 12602 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:47.146881 12158 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:47.148473 12557 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:47.148749 12535 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:47.151840 12572 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:47.150313 12608 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:47.153618 12576 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:47.212321 12608 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:47.152419 12487 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:47.152628  6785 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
W20260709 12:27:47.157593 12666 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.562s	user 0.013s	sys 0.000s
W20260709 12:27:47.158546 12479 consensus_meta.cc:287] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.535s	user 0.012s	sys 0.003s
I20260709 12:27:47.159349 12315 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:47.120580 12137 consensus_queue.cc:1243] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:47.116281 12503 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:27:47.162122 12714 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 12:27:47.173532  6768 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:27:47.127802  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:47.214783  5874 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:47.245097  6015 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:47.236019  6775 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:47.246209  6785 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:47.247228  6775 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:47.247619  6785 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:47.231024  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1397) took 1171 ms. Trace:
I20260709 12:27:47.179162  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:47.183146  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:47.196583 12475 consensus_meta.cc:287] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.576s	user 0.012s	sys 0.002s
I20260709 12:27:47.199709 12092 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:47.213241  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 90707us
I20260709 12:27:47.213428  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.250795  6019 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:47.258370  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:47.260319  6019 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:47.248644  5873 rpcz_store.cc:276] 0709 12:27:46.059834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.086208 (+ 26374us) service_pool.cc:224] Handling call
0709 12:27:46.086673 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:46.198635 (+111962us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:46.198671 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:46.198755 (+    84us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:46.198784 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.230683 (+1031899us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54002f94d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:47.230753 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:47.230938 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256979712}
I20260709 12:27:47.249940  6023 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:27:47.262997  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:47.263923  6023 raft_consensus.cc:2410] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 24 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 10.
W20260709 12:27:47.254629  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:47.148531 12515 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.570s	user 0.010s	sys 0.000s
I20260709 12:27:47.265859  6762 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:27:47.266541  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.268021  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:47.268574  5873 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:47.239218 12480 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.647s	user 0.012s	sys 0.002s
I20260709 12:27:47.270063 12480 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:47.273203 12480 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:47.273337 12647 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:47.275970 12667 consensus_meta.cc:287] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.626s	user 0.011s	sys 0.001s
I20260709 12:27:47.276372 12515 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:47.270069 12713 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:47.282644  5869 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 15: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:47.282953  6785 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:47.222644 12703 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.606s	user 0.001s	sys 0.002s
I20260709 12:27:47.290709  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:47.254347 12503 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.342092 12503 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:47.253831 12666 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:47.310142  6757 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:47.216012 12153 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:47.346191 12666 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:47.348220 12515 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:47.330011  6023 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:47.349764  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:47.351075  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:47.351696  6785 raft_consensus.cc:2376] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19. Re-sending same reply.
I20260709 12:27:47.352644  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:47.297292 12608 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.353596  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1553) took 1919 ms. Trace:
I20260709 12:27:47.354096  6022 rpcz_store.cc:276] 0709 12:27:45.434559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.194970 (+760411us) service_pool.cc:224] Handling call
0709 12:27:46.209762 (+ 14792us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:46.210401 (+   639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:46.210443 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:46.228953 (+ 18510us) log.cc:844] Serialized 52 byte log entry
0709 12:27:46.522287 (+293334us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:46.522381 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.522495 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:47.221919 (+699424us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:47.353249 (+131330us) raft_consensus.cc:1685] finished
0709 12:27:47.353277 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:47.353532 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":158,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":3379,"spinlock_wait_cycles":39903616,"thread_start_us":269277,"threads_started":1,"wal-append.queue_time_us":374896}
I20260709 12:27:47.356294  6022 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 12:27:47.356516  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1649) took 1016 ms. Trace:
I20260709 12:27:47.356894  6020 rpcz_store.cc:276] 0709 12:27:46.339846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.617296 (+277450us) service_pool.cc:224] Handling call
0709 12:27:47.356295 (+738999us) spinlock_profiling.cc:243] Waited 66 ms on lock 0x7b54001720d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:47.356460 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151785856}
I20260709 12:27:47.353739 12608 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:47.359547 12608 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:47.359622 12516 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:47.360275  6768 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:47.360994 12516 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:47.336860 12667 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:47.362493  6785 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:47.363359  6768 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:47.363868 12667 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:47.364665  5865 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:47.364678  6768 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:47.368283 12537 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:47.368894 12537 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:47.370209  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.370077 12537 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:47.371287 12537 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:47.371515  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.372783  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.331636 12726 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.333408 12647 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:47.341027 12712 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:47.374380 12726 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:47.376634 12712 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:47.378962  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:47.379786 12354 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:47.382548  6793 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:47.382654  5873 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:27:47.383950  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.384264  6793 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:47.386055  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.387032  5869 raft_consensus.cc:2393] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:47.387984  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.390278  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.390395  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.385959 12503 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:47.393460  5873 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:47.398623  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:47.379870  6022 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:47.408756  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:47.409098  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 118541us
I20260709 12:27:47.410923  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:47.413813  6019 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:47.413651 12712 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:47.416724  5869 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:47.376801 12726 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:47.425681  5870 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 22. Current term is 23. Ops: [22.10-22.10]
I20260709 12:27:47.429899  6015 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:47.381258 12581 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:47.419087 12673 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:27:47.444203  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.453594  6778 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:47.457449  6022 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
I20260709 12:27:47.433678 12503 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:47.460727  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:47.461697  6762 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:47.425851 12712 consensus_queue.cc:1059] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:27:47.470727  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1292) took 1624 ms. Trace:
I20260709 12:27:47.471311  5941 rpcz_store.cc:276] 0709 12:27:45.846708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.122846 (+276138us) service_pool.cc:224] Handling call
0709 12:27:46.151593 (+ 28747us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:46.659225 (+507632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.470395 (+811170us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b5400632c54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:47.470653 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1106801408}
I20260709 12:27:47.472632  6762 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 26.
I20260709 12:27:47.477751  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:47.489507 12503 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:47.507447 12551 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:47.491670 12613 raft_consensus.cc:3055] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:47.510001 12613 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:47.522569  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 45994.2us
I20260709 12:27:47.525446 12613 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:47.527137 12613 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:47.531457 12581 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:47.534345  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:47.535905  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.539127  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:47.541061 12515 raft_consensus.cc:2764] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:27:47.542814 12311 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:47.543329 12311 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:47.544685  5941 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c"
dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:48330
I20260709 12:27:47.545394  5941 raft_consensus.cc:606] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:47.559257  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:47.560101  6020 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 22: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:47.547005  5946 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:47.551059  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1571) took 1508 ms. Trace:
I20260709 12:27:47.552155  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:47.553396  6762 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:27:47.553408 12708 consensus_queue.cc:1243] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 22.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:47.554369  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:47.555420  5944 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:27:47.563195 12570 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.557598  6019 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:47.563575  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:47.541476 12666 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:27:47.558118  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:47.564780  6022 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 23: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:47.564822  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:47.565601  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:47.558398 12736 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:47.561268  6020 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:27:47.558550 12667 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:47.563751 12570 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:47.561276  6018 rpcz_store.cc:276] 0709 12:27:46.042617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.715727 (+673110us) service_pool.cc:224] Handling call
0709 12:27:46.717182 (+  1455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:46.721676 (+  4494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:46.721714 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:46.759036 (+ 37322us) log.cc:844] Serialized 26 byte log entry
0709 12:27:46.929169 (+170133us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:46.929230 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.929316 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:47.151448 (+222132us) raft_consensus.cc:1685] finished
0709 12:27:47.151482 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:47.550725 (+399243us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b54006b9ed0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:47.550986 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3902,"raft.queue_time_us":5679,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":34276,"spinlock_wait_cycles":2467200,"thread_start_us":101640,"threads_started":1,"wal-append.queue_time_us":304003}
I20260709 12:27:47.567056  6609 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:47.565511  6762 raft_consensus.cc:2376] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21. Re-sending same reply.
I20260709 12:27:47.577111  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:47.571676  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.577983 12712 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:47.578092  6762 raft_consensus.cc:2376] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21. Re-sending same reply.
I20260709 12:27:47.567826  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.578637  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:47.578759  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:47.579432  5872 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:47.579674  6762 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:47.579932  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.581032  6762 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:27:47.581207  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:47.573187  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.582970  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:47.572049  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.584474  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:47.585028  5944 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:47.577325  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:47.586575  6017 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:47.587996  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.588728  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:47.590076  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:47.590618  6768 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:47.595788  5866 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:47.596894  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:47.597818  5873 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:47.599234  5866 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:47.600472  5872 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:47.569772 12070 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:47.606122  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:47.617321  5840 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:47.619375  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:47.620738  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:47.586863 11835 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:47.623814  6611 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:47.623977  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:47.624888  5946 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 26: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:47.626040  6611 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:47.628520  6775 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:47.632563  6611 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:47.622699 12565 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:47.595503 12712 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:47.644177  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:47.645563  5942 raft_consensus.cc:2410] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:27:47.648579  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:47.649737  5947 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:47.654045  6611 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:47.665971  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:47.664147  5949 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:47.666925  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1561) took 1633 ms. Trace:
I20260709 12:27:47.654017  5868 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:47.667598  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:47.668444  5868 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:47.667563  6024 rpcz_store.cc:276] 0709 12:27:46.033469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.716540 (+683071us) service_pool.cc:224] Handling call
0709 12:27:46.764895 (+ 48355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:47.009386 (+244491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.666582 (+657196us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54006b8fd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:47.666814 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":532939008}
I20260709 12:27:47.672791  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:47.673765  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:47.674448  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 73829.3us
I20260709 12:27:47.674799  5941 raft_consensus.cc:2393] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 19: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:47.674721  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2125) took 1030 ms. Trace:
I20260709 12:27:47.675477  5867 rpcz_store.cc:276] 0709 12:27:46.643878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.644698 (+   820us) service_pool.cc:224] Handling call
0709 12:27:46.658055 (+ 13357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:46.975939 (+317884us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:47.000339 (+ 24400us) log.cc:844] Serialized 14 byte log entry
0709 12:27:47.552288 (+551949us) spinlock_profiling.cc:243] Waited 4.4 ms on lock 0x7b180024fe20. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb10c0a 00007f39d9250219 00007f39cfb98a40 00007f39cfb934e4 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287 00007f39d2c9e208
0709 12:27:47.552465 (+   177us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:47.552516 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:47.552562 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.674441 (+121879us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b5400210a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:47.674510 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:47.674650 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30200,"prepare.queue_time_us":30008,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":7754,"raft.queue_time_us":500775,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":740434688,"thread_start_us":51422,"threads_started":1,"wal-append.queue_time_us":113066}
I20260709 12:27:47.588142 12667 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:47.594966 12578 consensus_queue.cc:1059] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:27:47.673538  5940 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
I20260709 12:27:47.667595  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:47.636020 12713 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:47.602728 12581 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:27:47.640350 12565 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.621714 12310 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:27:47.641287 12535 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:47.682183  6611 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:47.687512  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:47.688571  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1572) took 1645 ms. Trace:
I20260709 12:27:47.619092 12542 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 12:27:47.605294 12516 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:27:47.698025  6021 rpcz_store.cc:276] 0709 12:27:46.042911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.714063 (+671152us) service_pool.cc:224] Handling call
0709 12:27:46.982874 (+268811us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:46.983229 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:46.983255 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:46.983810 (+   555us) log.cc:844] Serialized 52 byte log entry
0709 12:27:47.116084 (+132274us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:47.116143 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.116225 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:47.336180 (+219955us) raft_consensus.cc:1685] finished
0709 12:27:47.336203 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:47.688229 (+352026us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54003638d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:47.688489 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16069,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":14889,"spinlock_wait_cycles":506300160,"thread_start_us":115660,"threads_started":1,"wal-append.queue_time_us":338546}
I20260709 12:27:47.669380  5868 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:47.633549 12572 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:47.636312 12708 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:47.682499  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:47.692157 12713 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.710625  5941 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:47.642696 12719 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:47.636261 12589 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:47.694638  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 66498.7us
I20260709 12:27:47.696838  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:47.699512  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:47.673568 12136 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:47.673133 12515 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.740525  5940 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 26.
I20260709 12:27:47.679831 12092 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:47.746205 12092 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:47.748814 12092 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:47.751428  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1576) took 1703 ms. Trace:
I20260709 12:27:47.751891  6017 rpcz_store.cc:276] 0709 12:27:46.048090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.711248 (+663158us) service_pool.cc:224] Handling call
0709 12:27:46.756623 (+ 45375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:47.587114 (+830491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.751138 (+164024us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b5400661cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:47.751359 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1907765632}
W20260709 12:27:47.753093  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.631155 11835 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:47.755028 12744 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:27:47.755853 11835 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:47.691891 12137 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.787690  6018 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:47.787827 12137 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:47.755143 12614 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:47.691028 12727 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:47.791394 12727 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.791813 12727 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:47.793677  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:47.794651  6015 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 26: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:47.796413 12570 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:47.725780 12713 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:47.705646 12572 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:47.800751 12572 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:47.723656  6611 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:47.653388 12608 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:27:47.700585  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:47.712180  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:47.696991 12565 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:47.683202  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:47.742008  5945 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:47.744131 12515 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:47.703871 12535 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:47.690582  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:47.758247  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:47.817375  6773 raft_consensus.cc:2410] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 26 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 12:27:47.759104  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:47.818285  6756 raft_consensus.cc:2393] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 20: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:47.818439  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:47.760397  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:47.762251  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:47.819308  6017 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:47.819676  6019 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 22: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:47.756544 12719 raft_consensus.cc:697] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:47.822093  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 131574us
I20260709 12:27:47.784123 12311 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:47.698129 12666 consensus_queue.cc:1059] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:27:47.709656 12708 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.825555 12708 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:47.825904 12535 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:47.710492 12555 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 12:27:47.728318 12153 raft_consensus.cc:3055] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:47.786669  6022 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:47.739957 12589 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.790820 12137 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:47.802011  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:47.832291  6023 raft_consensus.cc:2410] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 21 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:47.832386  5947 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 12:27:47.804116  5941 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:47.780555  5915 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:47.833212  5942 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:27:47.833597  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:47.833892  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 53458us
I20260709 12:27:47.815977  5944 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:27:47.834996  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:47.743799 12551 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.809430  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1755) took 1159 ms. Trace:
I20260709 12:27:47.806866 11835 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.811398  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:47.836321  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 112783us
I20260709 12:27:47.837219  6768 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 26: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:47.723604 12712 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:47.782819  6020 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:27:47.822506  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:47.822906  5990 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:47.842809  6021 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:47.843195  5990 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:47.843995 11835 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:47.844718  5990 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:47.825049 12666 consensus_queue.cc:1243] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 23.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:47.846026 12608 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:47.847347 12479 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:47.847539 12070 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:47.848943  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.839866  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:47.840194  6611 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:47.835925  6757 rpcz_store.cc:276] 0709 12:27:46.650462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.650641 (+   179us) service_pool.cc:224] Handling call
0709 12:27:47.809231 (+1158590us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b5400004e54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:47.809355 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1304106880}
I20260709 12:27:47.850948  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:47.843058 12551 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:47.855187  5990 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:47.847486 12503 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:47.855566  6611 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:47.829746 12153 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:47.831305 12516 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:47.846392 12576 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:47.836768 12589 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:47.859015  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:47.846380 12653 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:47.860601  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:47.851040 12712 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.862664  6611 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:47.873304 11835 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:47.861821 12070 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:47.856331  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:47.863687  5990 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:47.849464 12137 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.867389  6021 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:47.852497 12581 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:47.870810 12717 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:27:47.868968  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 231185us
I20260709 12:27:47.870549  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:47.859861 12655 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:27:47.856554 12315 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:47.874511  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:47.876514  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 236151us
I20260709 12:27:47.860033 12537 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:47.848865 12560 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:47.889616 12153 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:47.894608  5944 raft_consensus.cc:2376] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18. Re-sending same reply.
I20260709 12:27:47.871363  5941 raft_consensus.cc:2376] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 25. Re-sending same reply.
I20260709 12:27:47.905256  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:47.895533  6022 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:27:47.876152 12653 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.883116 12712 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:47.899935  5942 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 22: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:47.884351 12613 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:47.900820  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:47.890623 12581 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:47.939476 12581 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:47.940701 12581 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:47.931278  5838 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:47.933560  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:47.943011  5941 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:47.934770  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:47.943852 12092 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:47.944368 12092 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:27:47.944507  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 44259.2us
I20260709 12:27:47.944809 12092 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:47.935993  5868 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:47.898900 12315 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:47.900282 11835 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 19.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:27:47.928653  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:47.983410  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.901823 12560 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.902343 12717 raft_consensus.cc:697] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:47.890352 12137 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:47.909833 12647 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:27:48.021396  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 92967.1us
W20260709 12:27:48.019702  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.934499 12653 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:48.029757 12560 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:48.065855  5868 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:27:48.066500  5868 pending_rounds.cc:85] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 10
I20260709 12:27:48.066862  5868 pending_rounds.cc:107] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 21.11
I20260709 12:27:48.067174  5868 raft_consensus.cc:2889] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:47.911126 12570 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.121s
I20260709 12:27:47.923800 12686 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.113s
I20260709 12:27:48.051785  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:48.030714 12748 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:27:48.033496 12726 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:48.071388 12717 consensus_queue.cc:237] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:48.013536 12315 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:47.983477 12153 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:48.011765 12718 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.104136 12718 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.107316 12718 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:48.109935 12750 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:48.110972 12535 raft_consensus.cc:3055] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:48.111415 12535 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:48.112361 12535 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 22.10, Last appended by leader: 10, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:48.117887 12752 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:48.118464 12752 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:48.118848 12752 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:48.118830 12708 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:48.125715 12708 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:47.937342 12673 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.122s
I20260709 12:27:48.020841 12613 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 24 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:48.036087 12480 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:48.095971 12475 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:48.113380 12535 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:48.150101 12551 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:47.905190 12719 consensus_queue.cc:237] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.068272  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2132) took 1241 ms. Trace:
W20260709 12:27:48.089394 12455 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
I20260709 12:27:48.160265  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:48.167433  6020 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:48.161818 12515 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:48.161841 12137 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:48.091285 12555 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.071s
W20260709 12:27:48.159370  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 105242us
I20260709 12:27:48.087522 12647 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:48.165843  5868 rpcz_store.cc:276] 0709 12:27:46.826616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.391792 (+565176us) service_pool.cc:224] Handling call
0709 12:27:47.392091 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:47.653896 (+261805us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400555da0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bfadc0 00007f39d2c851c0 00007f39d2c86231 00007f39d2c82d8f 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c
0709 12:27:47.935386 (+281490us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b58004e68d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bdf0aa 00007f39d2c61447 00007f39d2c79fe7 00007f39d2c7b7de 00007f39d2c85b49 00007f39d2c86416 00007f39d2c82d8f 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4
0709 12:27:48.067849 (+132463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.068185 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766341888}
I20260709 12:27:48.186292  6611 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:48.185372  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:48.169869 12686 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:48.157178 12673 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:27:48.207480  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:48.185757 12137 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:48.213656  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 349971us
W20260709 12:27:48.218343  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 309535us
I20260709 12:27:48.219717  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:48.155000 12750 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:27:48.171227 12480 consensus_queue.cc:1059] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:27:48.167455 12572 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:48.221994  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:48.228379  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 283039us
I20260709 12:27:48.206565 12542 raft_consensus.cc:476] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Not starting pre-election -- already a leader
I20260709 12:27:48.231827  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:48.250123  6020 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 12:27:48.216554 12475 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:48.220366 12537 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:48.254962  5913 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:48.216929 12726 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:48.257668  6020 pending_rounds.cc:85] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 11
I20260709 12:27:48.260793  6020 pending_rounds.cc:107] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 12:27:48.261179  6020 raft_consensus.cc:2889] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:48.262985  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 402553us
I20260709 12:27:48.264632  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:48.262907  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:48.266657  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:48.269593  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 338383us
I20260709 12:27:48.220611 12503 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:48.271045  5838 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:48.271713  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:48.272233 12503 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:48.274250  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:48.275177  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:48.275491  6762 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:48.276088  6775 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:48.276861  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:48.278621  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:48.279477  6757 raft_consensus.cc:2376] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19. Re-sending same reply.
W20260709 12:27:48.283788  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.229043 12551 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:48.266746  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:48.227953 12516 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:48.268205 12475 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.286907  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:48.253947 12729 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:48.292409  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:48.292906  5942 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d001526e46b24f429f57c91e4a270036"
dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:48330
I20260709 12:27:48.293187  5943 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:48.259902 12655 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:48.288489  6756 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:48.291894  6768 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 19: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:48.253907 12686 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:48.293892  5942 raft_consensus.cc:606] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:48.259397 12515 raft_consensus.cc:3055] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:48.218750 12613 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.223519 12137 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:48.267594 12754 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:48.299350  6775 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 27.
I20260709 12:27:48.291719 12475 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:48.267294 12537 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.286185 12761 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:48.316651  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 53811.7us
W20260709 12:27:48.341827  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 110105us
I20260709 12:27:48.259305 12480 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:27:48.262041 12576 raft_consensus.cc:697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:48.298430 12729 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.369525 12729 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:48.364708 12768 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.370436 12768 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.373178 12768 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:48.373860 12542 consensus_queue.cc:1059] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:48.374774 12542 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:27:48.376971 12614 consensus_meta.cc:287] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.000s
I20260709 12:27:48.379820 12729 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:48.384135 12310 raft_consensus.cc:3055] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:48.384707 12310 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:48.385888 12760 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:48.386190 12774 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:48.386649 12729 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:48.386734 12760 raft_consensus.cc:697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:48.387094 12766 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:48.387787 12766 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:48.387974 12760 consensus_queue.cc:237] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:48.388867 12779 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:48.388763 12766 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:48.389556 12779 consensus_queue.cc:1243] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:27:48.343961 12713 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.004s
I20260709 12:27:48.389814 12761 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:48.390229 12713 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:48.391665 12726 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.392038 12555 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:48.392251 12726 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.393913 12578 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:48.394237  6785 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:27:48.394325 12578 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:48.395490 12578 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:48.397511 12765 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:48.379741 12315 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:48.399084  5947 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 15.
I20260709 12:27:48.401154 12315 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:48.403290 12070 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:48.404992  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:48.406503 12761 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:48.407487  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 136505us
I20260709 12:27:48.408530  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:48.360021  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 95467.6us
I20260709 12:27:48.354177 12754 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:48.414968  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:48.392606 12555 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:48.392658 12713 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:48.340889 12515 raft_consensus.cc:740] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:48.409598  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:48.410001  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:48.366436  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 71407.1us
I20260709 12:27:48.425019  5865 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:27:48.425221  5866 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:48.409672  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:48.413091  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:48.340745 12718 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:48.418175  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:48.432925  6022 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:48.431861  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:48.291612 12551 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.449587  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:48.449970  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:48.456398  5867 raft_consensus.cc:2376] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20. Re-sending same reply.
I20260709 12:27:48.365090 12537 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:48.509809  6022 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
I20260709 12:27:48.421444 12578 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:48.345785 12686 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.298393 12647 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.450645 12713 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.362169 12762 consensus_queue.cc:1059] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.163s
I20260709 12:27:48.511943 12608 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:48.571213  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 162730us
I20260709 12:27:48.513962 12070 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:48.403391 12092 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:48.419962 12589 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.511992 12750 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:27:48.408989 12576 consensus_queue.cc:237] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:48.565876 12742 log.cc:927] Time spent T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:27:48.419344 12310 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:48.595585  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 163816us
I20260709 12:27:48.598909 12480 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:48.604836 12780 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:48.570324 12568 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:48.628544  6757 raft_consensus.cc:1240] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:27:48.634058 12572 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:48.636277 12572 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:48.636765 12572 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:48.610193 12655 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 26 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:48.638854 12655 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:48.639488 12311 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:48.585741 12768 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 12:27:48.643430 12768 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:48.645920 12568 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:48.647220 12568 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:48.631876 12480 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:27:48.627440 12772 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:27:48.651997 12782 raft_consensus.cc:3055] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:48.652415 12782 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:48.557996 12776 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.100s
W20260709 12:27:48.609898 12153 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.002s
I20260709 12:27:48.653314 12782 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:48.667446  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.512351 12516 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.576682 12713 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.669061  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:48.679792  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.683347 12480 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.626097 12780 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.586725 12755 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.072s
I20260709 12:27:48.684791  5947 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:27:48.584146 12608 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:48.689544  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 280564us
W20260709 12:27:48.690200  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.660609 12786 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:48.678210 12653 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.600s	user 0.013s	sys 0.000s
I20260709 12:27:48.697499  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:48.575510 12515 consensus_queue.cc:260] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:48.702368  5838 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:48.707384  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:48.709161 12070 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:48.709275  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:48.731855  5948 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:48.709558 12775 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:27:48.731941  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.575784 12754 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:48.741245 12589 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:48.742023 12535 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.002s
I20260709 12:27:48.683980 12480 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.691625 12780 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:48.711099 12153 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:48.657701 12781 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.112s
I20260709 12:27:48.702404 12782 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:48.757448  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 160941us
W20260709 12:27:48.757840  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 60402.6us
I20260709 12:27:48.763021  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:48.713128 12784 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:48.764711  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 62415.2us
I20260709 12:27:48.767729  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:48.767879  5838 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:48.770115  5941 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:48.770682  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:48.773600  5838 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:48.771313 12754 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:48.771093 12785 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:48.806456 12785 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.809041 12785 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:48.675377 12572 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:48.811707 12572 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:48.755220 12770 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:27:48.814339 12788 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:48.815320 12788 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:48.815747 12788 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:48.816004 12480 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:48.816700 12788 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:48.820966  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 47441.7us
I20260709 12:27:48.821884  5838 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:48.823909  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 60975.3us
I20260709 12:27:48.827597  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:48.763836 12153 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:48.888121  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 117500us
I20260709 12:27:48.878472  5869 raft_consensus.cc:2410] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:27:48.878049  5948 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:48.892026  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:48.773172 12719 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:48.907521  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:48.890753  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:48.742805 12653 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.903658  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:48.875888 12565 consensus_meta.cc:287] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.052s	user 0.015s	sys 0.000s
W20260709 12:27:48.876133 12758 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.413s	user 0.001s	sys 0.000s
W20260709 12:27:48.904055  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 472736us
I20260709 12:27:48.922793  5915 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:48.907914  5990 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:48.820138 12768 consensus_queue.cc:1175] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6add331326ca47038308a447bf798fa7" responder_term: 18 status { last_received { term: 18 index: 9 } last_committed_idx: 7 last_received_current_leader { term: 18 index: 9 } }
I20260709 12:27:48.741356 12070 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:48.930053  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:48.730935 12786 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:48.887388 12770 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:48.944288  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:48.944566  6022 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:48.877110 12789 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:48.947975  5915 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:48.949688  6016 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:27:48.899015 12153 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:48.905836 12750 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:48.897380 12755 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:48.916548 12762 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:48.939807 12560 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.852s	user 0.014s	sys 0.000s
I20260709 12:27:48.913498 12796 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.972142  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:48.938958 12479 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:48.977288  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:49.037271  6015 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:48.978964  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:48.979040  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:48.985677  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 163838us
I20260709 12:27:48.947624 12766 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:48.951138 12795 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:48.955569 12791 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:48.956398 12785 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:49.000260  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:27:49.007938  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.038460  5948 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:27:49.039237  5838 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:48.937805 12092 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:48.963980 12770 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:48.931435 12565 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.794126 12784 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:48.986688 12776 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:48.988505 12762 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.055531  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:49.103585  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 48170us
I20260709 12:27:49.077315  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:49.077750  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 77634.6us
I20260709 12:27:48.995694 12516 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.036389 12560 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:49.047838 12479 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:49.119516  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1415) took 1312 ms. Trace:
I20260709 12:27:49.041446 12480 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.073s
W20260709 12:27:49.093264  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 54147.9us
I20260709 12:27:49.092613 12770 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:49.123759  5838 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:49.119969  5872 rpcz_store.cc:276] 0709 12:27:47.807481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.809712 (+  2231us) service_pool.cc:224] Handling call
0709 12:27:47.810129 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:47.814203 (+  4074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:47.814251 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:47.817426 (+  3175us) log.cc:844] Serialized 26 byte log entry
0709 12:27:48.218067 (+400641us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:48.218108 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.218174 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:49.119228 (+901054us) raft_consensus.cc:1685] finished
0709 12:27:49.119258 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:49.119436 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3568,"raft.queue_time_us":9712,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":9229,"thread_start_us":364532,"threads_started":1,"wal-append.queue_time_us":510705}
I20260709 12:27:49.102025 12535 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:49.113646 12516 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:49.102185 12755 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:48.981842 12789 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:49.042097 12796 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:49.112047 12762 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:49.114082 12613 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:49.111624 12713 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:49.122484 12773 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:27:49.131402  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:49.170215  5948 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:49.130455  5946 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:27:49.173831 12772 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:49.174431 12772 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:49.178467 12475 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.762s	user 0.012s	sys 0.000s
I20260709 12:27:49.179252 12475 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:49.181087  6766 raft_consensus.cc:2393] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:49.053005 12795 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:49.183557 12795 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:49.135115 12667 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:49.187579 12475 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:49.132412 12535 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:49.130396 12811 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:49.117664 12092 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:27:49.132135 12653 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:49.131028 12776 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:27:49.153893  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:49.038218 12760 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:27:49.158548 12786 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:49.159124 12789 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:49.159965 12808 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:49.160744  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.280328  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:49.283128  5874 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:27:49.287932  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:49.161589 12796 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:49.163000 12311 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.523s	user 0.015s	sys 0.000s
I20260709 12:27:49.163763 12153 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:49.165369 12479 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:49.166762 12565 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:49.157346 12784 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:49.169243 12560 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:49.169296 12613 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.308885 12613 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:49.312026 12535 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:49.317456 12153 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:49.146083 12804 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:49.124770 12608 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:49.120857  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:49.091648  5839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.018s	sys 0.044s
W20260709 12:27:49.324710  5839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.018s	sys 0.044s
W20260709 12:27:49.325228  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:49.325613  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:49.253681  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.272491  5948 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 27.
W20260709 12:27:49.199851 12727 consensus_meta.cc:287] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.408s	user 0.013s	sys 0.003s
W20260709 12:27:49.200731 12537 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.683s	user 0.014s	sys 0.000s
I20260709 12:27:49.176734 12772 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:49.261768 12581 consensus_meta.cc:287] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.260s	user 0.008s	sys 0.008s
W20260709 12:27:49.263182 12576 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.011s	sys 0.006s
W20260709 12:27:49.267707 12752 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.146s	user 0.014s	sys 0.000s
I20260709 12:27:49.300446  5949 raft_consensus.cc:2410] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
W20260709 12:27:49.299463  5838 leader_election.cc:336] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Timed out: RequestConsensusVote RPC to 127.2.4.195:43329 timed out after 1.689s (SENT)
I20260709 12:27:49.268079 12811 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:49.306640  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:49.330915  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:49.333689  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 212904us
I20260709 12:27:49.335691  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:49.337162  5871 pending_rounds.cc:85] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 9
W20260709 12:27:49.346517  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 47243.6us
I20260709 12:27:49.346868  5915 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:49.349416  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:49.350277  5948 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:49.347227  5871 pending_rounds.cc:107] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 12:27:49.353328  5871 raft_consensus.cc:2889] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:49.325263 12578 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.801s	user 0.009s	sys 0.005s
W20260709 12:27:49.346747  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.358753  5865 raft_consensus.cc:2368] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 16.
I20260709 12:27:49.360704  5913 leader_election.cc:400] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 16.
I20260709 12:27:49.361316  5913 leader_election.cc:403] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:27:49.362250  5870 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:27:49.296248 12808 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:49.376483  5948 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:49.305287 12475 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:27:49.337316 12311 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:49.410629  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 430757us
W20260709 12:27:49.415568  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 523510us
I20260709 12:27:49.343722 12809 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:49.420210  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:49.358397 12568 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.003s	sys 0.011s
I20260709 12:27:49.421137  6793 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:49.422284 12786 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:49.423324 12653 raft_consensus.cc:3055] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:49.423905 12653 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:27:49.344205 12576 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.011s	sys 0.007s
I20260709 12:27:49.425380  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:49.426939  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:49.429054  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:49.449426  6785 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:49.429618  6778 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:49.428653  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:49.451735  6757 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:49.338945 12727 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:49.378717 12560 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:49.357147 12811 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:49.361675 12760 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:49.456245  5944 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:49.343362 12537 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:49.457677  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 160547us
W20260709 12:27:49.426188 12754 consensus_meta.cc:287] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.627s	user 0.009s	sys 0.002s
I20260709 12:27:49.425081 12653 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:49.418044 12666 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 15. Current term is 16.
I20260709 12:27:49.419466 12311 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:49.454589 12794 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:49.466763 12801 log.cc:927] Time spent T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:27:49.468336 12808 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:49.470575 12727 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:49.426295 12780 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.662s	user 0.006s	sys 0.007s
W20260709 12:27:49.485355  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 125724us
I20260709 12:27:49.471661  5990 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:49.484131 12785 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:27:49.497128 12771 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.869s	user 0.009s	sys 0.004s
I20260709 12:27:49.359930 12752 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:49.536227 12791 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:49.537617 12311 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:49.538014 12752 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:49.538280 12311 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.539031  5942 raft_consensus.cc:2410] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:27:49.538808 12311 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:49.509249 12760 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:49.514873 12762 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:49.542095  5947 raft_consensus.cc:2410] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:27:49.542917  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:49.544009  5872 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
W20260709 12:27:49.544570  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 72997.9us
I20260709 12:27:49.498634  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:49.545480  5913 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:49.545944  5990 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:49.461869 12782 consensus_meta.cc:287] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.659s	user 0.008s	sys 0.004s
I20260709 12:27:49.510149  5948 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 19: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:49.427556 12719 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 27 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:49.525475  5915 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:49.489076 12576 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.534693 12766 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:49.523831 12717 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.535422 12516 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:49.534873 12785 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:49.510752 12825 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.038s
W20260709 12:27:49.430769 12712 consensus_meta.cc:287] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.445s	user 0.016s	sys 0.000s
I20260709 12:27:49.502519  6762 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
W20260709 12:27:49.547957  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.577865  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:49.578858  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:49.578986  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:49.554311  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:49.379678 12555 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:49.563503  6757 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:27:49.564234  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:49.541525 12760 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:49.569208  6778 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:49.576037  6793 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
I20260709 12:27:49.576632  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:49.584283  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:49.586934  6023 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 23. Current term is 24. Ops: [21.13-23.14]
I20260709 12:27:49.545332 12666 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:49.588040  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:49.588295  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:49.592638  6785 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 20: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:49.593940  6020 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:49.568073 12772 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:49.599252  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:49.595543  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:49.641670  6775 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:49.598313  6756 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 15.
I20260709 12:27:49.594388  6023 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:49.570786 12070 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:49.656601 12762 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:49.603039  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:49.600330  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:49.658281  6020 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:49.570837 12754 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.660652 12754 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:49.634239  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:49.543149 12780 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:49.567826 12752 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:49.643862  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:49.643600  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:49.647070 12311 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:49.647404 12537 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:49.668156 12754 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:49.670395 12515 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:49.633435 12727 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.674041  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:49.571123 12800 consensus_queue.cc:1059] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 12:27:49.674480 12727 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:49.649488  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 82011.9us
W20260709 12:27:49.650404  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 71637.8us
I20260709 12:27:49.642771 12479 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:49.600082 12666 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:49.532709 12794 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:49.651011 12613 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:49.651713 12822 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:49.570626 12576 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:49.588883 12712 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:49.577987 12790 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.570457 12795 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.571347 12817 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:49.677537  5990 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:49.656662 12070 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.681711  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1735) took 1396 ms. Trace:
I20260709 12:27:49.677035  5868 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:27:49.692667  5946 rpcz_store.cc:276] 0709 12:27:48.285299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.285445 (+   146us) service_pool.cc:224] Handling call
0709 12:27:48.291565 (+  6120us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:49.173894 (+882329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:49.681425 (+507531us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b5400602a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:49.681650 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1928514944}
I20260709 12:27:49.680939  5838 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:49.695531  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:49.573176 12717 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:49.702411  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:49.702816  6778 raft_consensus.cc:2393] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:49.676793 12727 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:49.705603 12572 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.893s	user 0.010s	sys 0.004s
I20260709 12:27:49.707480 12834 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.708151 12834 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:49.678434 12686 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:49.681802 12666 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:49.712805  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1776) took 1441 ms. Trace:
I20260709 12:27:49.713302  6766 rpcz_store.cc:276] 0709 12:27:48.271153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.271450 (+   297us) service_pool.cc:224] Handling call
0709 12:27:49.712577 (+1441127us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b54002a12d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:49.712734 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2088957312}
I20260709 12:27:49.715121  6773 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 12:27:49.703384  6768 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:49.721597  5990 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:49.676978 12516 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:49.687446 12817 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.724995  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:49.692438 12822 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:49.642179 12760 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:49.697161 12070 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:49.686249 12794 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:49.679243 12479 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:49.692945 12790 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:49.740790 12822 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:49.728260 12786 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:49.680881 12800 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:27:49.745054 12834 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:49.680538 12789 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:27:49.743321 12551 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:49.693148 12795 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:49.736241 12817 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:49.680047 12719 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:49.779066 12576 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:49.779906 12772 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:49.780435 12712 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:49.785961 12780 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:49.792742 12757 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 1.206s	user 0.000s	sys 0.007s
I20260709 12:27:49.792913 12811 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:49.795420 12653 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:49.802196 12763 raft_consensus.cc:3055] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:49.802717 12763 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:49.803514 12763 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:49.804658 12763 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:49.796859 12786 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:49.749181 12760 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:49.810490 12822 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:49.815073 12790 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:49.808866 12770 consensus_meta.cc:287] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.672s	user 0.014s	sys 0.000s
I20260709 12:27:49.819679 12795 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:49.819536 12770 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:49.819764 12811 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:49.798831 12772 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:49.822420 12770 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:49.859869  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 178670us
I20260709 12:27:49.983705 12613 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:49.862358  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.864063  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2063) took 2042 ms. Trace:
I20260709 12:27:49.990103  5945 rpcz_store.cc:276] 0709 12:27:47.821882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.828102 (+  6220us) service_pool.cc:224] Handling call
0709 12:27:47.828448 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:47.828835 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:47.828865 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:47.829573 (+   708us) log.cc:844] Serialized 52 byte log entry
0709 12:27:48.191514 (+361941us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:48.191590 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.191700 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:49.792039 (+1600339us) spinlock_profiling.cc:243] Waited 46.4 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:49.863124 (+ 71085us) raft_consensus.cc:1685] finished
0709 12:27:49.863186 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:49.863750 (+   564us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":332,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":106696448,"thread_start_us":331995,"threads_started":1,"wal-append.queue_time_us":614289}
I20260709 12:27:49.994858  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:49.908154  6768 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
I20260709 12:27:49.913388  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:49.912984  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:49.914407  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:49.922912 12153 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:49.925243 12310 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:49.926512 12503 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:49.799362 12647 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.191s	user 0.001s	sys 0.017s
I20260709 12:27:49.928841 12479 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:49.735970 12516 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:49.812202 12708 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:50.026538 12708 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:50.028489  5949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 2052) took 2377 ms (client timeout 1689 ms). Trace:
I20260709 12:27:49.812919 12717 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:49.815567 12726 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:49.968734 12760 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:49.933442 12542 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:50.016585  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:50.002935 12767 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.045081  6017 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:50.058127  5947 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:50.045900  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:50.058764 12647 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.450s	user 0.002s	sys 0.017s
I20260709 12:27:50.066421  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:50.059787 12767 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:50.043547 12811 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:50.053817 12153 raft_consensus.cc:697] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:50.101903  5945 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:27:50.101548 12811 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:50.102743 12153 consensus_queue.cc:237] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:50.103435  5941 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 17.
I20260709 12:27:50.076924  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1359) took 1806 ms. Trace:
I20260709 12:27:50.037981 12542 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:50.105115 12811 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:50.056958  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:50.066481 12712 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.079799  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:50.080327 12647 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.076377  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:50.081063 12503 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:50.081553  5990 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:50.081928  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1362) took 1795 ms. Trace:
W20260709 12:27:50.029134  5949 rpcz_store.cc:269] 0709 12:27:47.650802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.667268 (+ 16466us) service_pool.cc:224] Handling call
0709 12:27:50.027929 (+2360661us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54004fced4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:50.028233 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3709419136}
I20260709 12:27:50.088790  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.089403  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.050323  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:50.091403  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:50.092084  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:50.056205 12515 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.096421 12555 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:50.106837  5870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2148) took 1695 ms (client timeout 1859 ms). Trace:
I20260709 12:27:50.107055 12542 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:50.043807  5943 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:50.108825 12760 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:50.104748  6773 rpcz_store.cc:276] 0709 12:27:48.270694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.270842 (+   148us) service_pool.cc:224] Handling call
0709 12:27:48.271257 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:49.717004 (+1445747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.076028 (+359024us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54002c1ad4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:50.076565 (+   537us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3319624960}
I20260709 12:27:50.109930  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.112123 12824 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.047261  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:50.139734 12137 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:50.140789  6768 raft_consensus.cc:2393] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 19: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:50.078673  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:50.141528  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:50.141986  5946 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:50.131145  6762 rpcz_store.cc:276] 0709 12:27:48.285879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.286125 (+   246us) service_pool.cc:224] Handling call
0709 12:27:48.287328 (+  1203us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:49.577657 (+1290329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.081208 (+503551us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400004454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:50.081713 (+   505us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2793869056}
I20260709 12:27:50.142323  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:50.142987  6775 raft_consensus.cc:2410] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:27:50.061210  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1585) took 1109 ms. Trace:
I20260709 12:27:50.143736  6021 rpcz_store.cc:276] 0709 12:27:48.943735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.943915 (+   180us) service_pool.cc:224] Handling call
0709 12:27:50.053077 (+1109162us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b540065b3d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:50.053356 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2119942528}
I20260709 12:27:50.144205  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:50.144780  6762 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 23. Current term is 24. Ops: [21.13-23.14]
I20260709 12:27:50.145671  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:50.122970  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:50.144824  6775 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:27:50.116263  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:50.066448  6018 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:27:50.118427  5948 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 22. Current term is 23. Ops: []
W20260709 12:27:50.120667 12851 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.089s	user 0.002s	sys 0.000s
I20260709 12:27:50.078007  5874 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:50.125119  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:50.139370 12542 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:50.149463  6021 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 24: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:50.150264 12824 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:50.052521 12516 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:50.135346  5870 rpcz_store.cc:269] 0709 12:27:48.411636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.411831 (+   195us) service_pool.cc:224] Handling call
0709 12:27:50.106132 (+1694301us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b5400388954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:50.106447 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2082587520}
I20260709 12:27:50.158245 12760 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:50.160427  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.164746  6773 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:27:50.164891 12850 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.122s	user 0.004s	sys 0.000s
I20260709 12:27:50.166437  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:50.167407  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.145473 12712 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:50.170181  6020 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:27:50.196861  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:50.198779  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:50.200857  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.193401  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.202772  5871 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:27:50.204067  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.204447  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:50.204984  5871 raft_consensus.cc:2410] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:27:50.172410 12479 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:50.172883 12717 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.173940 12726 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.175477 12503 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:50.182971 12647 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:50.201267 12767 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:50.029187 12708 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.8, Last appended by leader: 8, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:50.200378  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:50.221696  5865 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:50.224256 12726 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:50.139803 12515 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:50.227180  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 146056us
I20260709 12:27:50.194697 12760 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:50.234138  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:50.222363  6017 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:50.232148 12503 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:50.266192  5869 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:50.273700  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.281313  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.291209  5872 raft_consensus.cc:2410] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:27:50.263693 12824 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:50.294028  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1416) took 1015 ms. Trace:
I20260709 12:27:50.294574  5873 rpcz_store.cc:276] 0709 12:27:49.278431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.278589 (+   158us) service_pool.cc:224] Handling call
0709 12:27:49.307393 (+ 28804us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:49.432479 (+125086us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:49.432545 (+    66us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:49.432611 (+    66us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:49.432661 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.277315 (+844654us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54006f2054. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:50.283683 (+  6368us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:50.293816 (+ 10133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":286946560}
I20260709 12:27:50.287740  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:50.302942  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:50.298444  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.293186 12479 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:50.312547  5871 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:50.276983 12767 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:50.316210  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.301352 12708 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:50.327111  5869 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:50.332742 12729 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.333937 12729 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:50.327857  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2070) took 2381 ms. Trace:
I20260709 12:27:50.328536 12565 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.313593 12712 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:50.338308 12565 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:50.341051 12565 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:50.344199  6611 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:50.345713 12537 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.346654 12712 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:50.346405 12537 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:50.347389 12712 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 24 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:50.348625 12712 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:50.337966  5944 rpcz_store.cc:276] 0709 12:27:47.892155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.908234 (+ 16079us) service_pool.cc:224] Handling call
0709 12:27:47.908692 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:49.488406 (+1579714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.220726 (+732320us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b540058fa54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:50.273467 (+ 52741us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3558253824}
I20260709 12:27:50.332264 12613 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:50.350687 12613 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:50.351444 12783 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:50.352876 12783 raft_consensus.cc:3055] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:50.353106 12613 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:50.353268 12783 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:50.330348 12846 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.364s	user 0.002s	sys 0.002s
I20260709 12:27:50.330766  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.280532 12717 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:50.353848  5949 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:50.354894 12537 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:50.358157 12760 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:50.359339 12752 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:50.359973 12479 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:50.360616  5941 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 14. (term mismatch)
I20260709 12:27:50.364809  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:50.332109 12853 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:27:50.361505  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:50.363651 12515 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:50.311126 12781 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.173s
I20260709 12:27:50.313923 12824 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.363760 12647 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:50.364095  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 56864.8us
I20260709 12:27:50.364931  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:50.370610  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:50.378804 12780 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:50.378556  5868 raft_consensus.cc:2410] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 23 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:27:50.382160  5941 pending_rounds.cc:85] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 13
I20260709 12:27:50.360826  5874 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:27:50.362195 12070 consensus_meta.cc:287] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.609s	user 0.013s	sys 0.014s
I20260709 12:27:50.380362  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:50.386004  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 66496.5us
I20260709 12:27:50.391028  5867 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:50.337116 12729 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:50.371584 12503 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:27:50.398983  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 55026.4us
I20260709 12:27:50.393366  5941 pending_rounds.cc:107] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 12:27:50.403921  5941 raft_consensus.cc:2889] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:50.401229  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:50.330426 12767 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:50.426254  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 44229.7us
I20260709 12:27:50.431262  6020 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:50.434144  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:50.437269  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.424584  5838 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:50.461205  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 61256.3us
W20260709 12:27:50.470415  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.477056  5870 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:50.481266  5943 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
W20260709 12:27:50.485056 12763 consensus_meta.cc:287] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.680s	user 0.017s	sys 0.008s
W20260709 12:27:50.486480  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 49944.5us
I20260709 12:27:50.484476 12719 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:27:50.489998 12719 consensus_queue.cc:1243] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 22.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:50.489743 12780 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:50.490535  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 83744.8us
I20260709 12:27:50.491979  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:50.492749 12780 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:50.493738 12752 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:50.497913 12763 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:50.499589 12783 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:50.501231 12772 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:50.502394  5940 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:50.509894  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 98118.5us
I20260709 12:27:50.521855  6024 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:27:50.403332 12070 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:50.436184 12824 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:50.528398  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:50.536458  6793 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:27:50.544883  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.440574 12647 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 12:27:50.548934  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:50.540102  5949 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:50.561533  6762 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:50.562583  5913 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:27:50.563576 12310 consensus_meta.cc:287] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.554s	user 0.007s	sys 0.010s
I20260709 12:27:50.578171  5941 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:50.582919 12775 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:50.584093 12775 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:50.578218 12770 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.587579 12770 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:50.590341 12783 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:50.591797  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:50.593272  6766 raft_consensus.cc:2410] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:27:50.551734 12719 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:27:50.595455  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 66675us
W20260709 12:27:50.597476  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 46677.6us
W20260709 12:27:50.608215  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 52116.9us
I20260709 12:27:50.599905 12719 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.640707  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:50.648232  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.662050  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.644925 12719 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:50.674211  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.680584 12796 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:27:50.643106 12763 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
W20260709 12:27:50.683171  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.682647  6611 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:50.685833  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.661981 12515 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:50.712955  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:50.688791 12858 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 12:27:50.621692 12503 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:50.681679 12647 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.723703  5874 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:50.732014  5941 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:50.740381  5943 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:50.767617  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 205275us
I20260709 12:27:50.718634 12515 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:50.730098 12717 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:50.774117  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.764191 12686 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:50.804978 12686 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:50.807369  5946 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:50.803236  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.775381  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:50.776393 12848 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:27:50.776978 12475 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:50.777489 12822 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:50.778144  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1598) took 1175 ms. Trace:
I20260709 12:27:50.779722 12820 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:50.780411 12727 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:50.782601  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.784819 12775 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:50.788460  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.796643 12767 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:50.798668  6756 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:27:50.802642 12515 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:50.830196  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:50.833405  5869 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 21: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:50.807473 12070 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:50.837584  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 169059us
I20260709 12:27:50.808400 12772 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:50.844013  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.829643  5941 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:50.836951  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.813067  6016 rpcz_store.cc:276] 0709 12:27:49.586414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.588918 (+  2504us) service_pool.cc:224] Handling call
0709 12:27:49.589279 (+   361us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:49.880175 (+290896us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:27:49.880985 (+   810us) log.cc:844] Serialized 14 byte log entry
0709 12:27:50.006689 (+125704us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:50.120994 (+114305us) log.cc:844] Serialized 52 byte log entry
0709 12:27:50.121256 (+   262us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:50.121613 (+   357us) log.cc:844] Serialized 14 byte log entry
0709 12:27:50.127385 (+  5772us) log.cc:844] Serialized 14 byte log entry
0709 12:27:50.127610 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.202335 (+ 74725us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b5400763754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:50.202518 (+   183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:50.753353 (+550835us) raft_consensus.cc:1685] finished
0709 12:27:50.753373 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:50.761938 (+  8565us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":107164,"prepare.run_cpu_time_us":155,"prepare.run_wall_time_us":160,"raft.queue_time_us":49045,"raft.run_cpu_time_us":1062,"raft.run_wall_time_us":44292,"spinlock_wait_cycles":665130752,"thread_start_us":123646,"threads_started":1,"wal-append.queue_time_us":302856}
I20260709 12:27:50.841205  5946 pending_rounds.cc:85] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 9
I20260709 12:27:50.845983  5946 pending_rounds.cc:107] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 22.10
I20260709 12:27:50.818150 12820 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:50.811513 12726 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:50.838611  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:50.823799 12555 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.846410  5946 raft_consensus.cc:2889] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:50.808696 12647 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:50.872560  5874 raft_consensus.cc:2410] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 12:27:50.884443  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:50.848152 12796 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.841144 12310 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:50.885289  5869 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:27:50.855168 12752 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:50.899623  5944 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:27:50.875276 12767 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:50.881770 12555 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:50.917858 12555 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:50.843137 12822 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:50.922964  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 84459.2us
I20260709 12:27:50.847609 12727 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:50.946858 12820 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:50.947815  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:50.951088  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 126886us
W20260709 12:27:50.961293 12857 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.309s	user 0.001s	sys 0.001s
I20260709 12:27:50.957366 12767 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:50.920941 12770 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:50.979465  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.975490  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:27:50.989224  5865 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.729s	user 0.010s	sys 0.004s
W20260709 12:27:50.989054  5867 consensus_meta.cc:287] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.578s	user 0.011s	sys 0.000s
I20260709 12:27:51.018486  5865 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
I20260709 12:27:51.018743  5867 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
I20260709 12:27:50.983770  5869 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:50.996493  5868 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:50.968439  6757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 1515) took 1389 ms. Trace:
I20260709 12:27:50.977463  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2157) took 1641 ms. Trace:
W20260709 12:27:50.993067  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.907210 12796 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:50.988461  5913 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:51.026482  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:50.945498 12310 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:51.045785  6611 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:51.046857 12613 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:51.047224  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 87152.9us
I20260709 12:27:51.047860 12310 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:51.048717 12763 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.049671  6756 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:51.049474 12763 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:51.051285  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:51.052027 12763 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:51.053028 12868 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.246s	user 0.004s	sys 0.000s
I20260709 12:27:50.969466 12070 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:50.975840 12767 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:51.061429 12770 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:27:51.063702  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.063647 12770 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:51.064347 12770 consensus_queue.cc:1243] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:51.065122  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1793) took 1384 ms. Trace:
I20260709 12:27:51.065586  6785 rpcz_store.cc:276] 0709 12:27:49.680262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.168020 (+487758us) service_pool.cc:224] Handling call
0709 12:27:50.168625 (+   605us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:50.169189 (+   564us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.169223 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:50.270648 (+101425us) log.cc:844] Serialized 26 byte log entry
0709 12:27:50.663944 (+393296us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:50.664001 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.664108 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:51.045342 (+381234us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:51.064770 (+ 19428us) raft_consensus.cc:1685] finished
0709 12:27:51.064795 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.065048 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83129216,"thread_start_us":341878,"threads_started":1,"wal-append.queue_time_us":513931}
W20260709 12:27:51.041267  5696 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50588 (request call id 59) took 20452 ms (client timeout 15000 ms). Trace:
W20260709 12:27:51.034116 12817 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.210s	user 0.007s	sys 0.019s
I20260709 12:27:50.979007 12475 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:51.069275 12817 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:51.070858 12860 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.276s	user 0.000s	sys 0.002s
W20260709 12:27:51.071894 12708 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.726s	user 0.011s	sys 0.002s
I20260709 12:27:51.072446  6015 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 12:27:51.029446  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:51.073215 12859 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.366s	user 0.001s	sys 0.001s
I20260709 12:27:51.075058  5870 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:51.076340  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1600) took 1489 ms. Trace:
I20260709 12:27:50.961310 12503 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:51.078085 12503 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:51.080603  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1586) took 2135 ms. Trace:
I20260709 12:27:51.053992 12070 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:51.082132  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:51.061694  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:51.052088  6766 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:51.025491  5874 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:51.076993  6015 rpcz_store.cc:276] 0709 12:27:49.587009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.587149 (+   140us) service_pool.cc:224] Handling call
0709 12:27:49.587464 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:51.073019 (+1485555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.075895 (+  2876us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400661cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:51.076260 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3413932928}
I20260709 12:27:51.022287  5866 rpcz_store.cc:276] 0709 12:27:49.335831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.336930 (+  1099us) service_pool.cc:224] Handling call
0709 12:27:49.337231 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:50.643174 (+1305943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.643242 (+    68us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:50.643312 (+    70us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:50.643368 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.976849 (+333481us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400026f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:50.976920 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:50.977209 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15386,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":25881,"spinlock_wait_cycles":2905215744}
I20260709 12:27:51.078464 12820 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:51.100663  6018 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.895s	user 0.017s	sys 0.002s
W20260709 12:27:51.068419 12542 consensus_meta.cc:287] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.877s	user 0.014s	sys 0.000s
I20260709 12:27:51.103506  5872 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 14. (term mismatch)
I20260709 12:27:51.103904  5872 pending_rounds.cc:85] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 13
I20260709 12:27:51.104146  5872 pending_rounds.cc:107] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 23.14
I20260709 12:27:51.104417  5872 raft_consensus.cc:2889] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:51.109884  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:51.110167  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:51.111089 12869 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 12:27:51.111114  5942 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:27:51.112176  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:51.112958  5941 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:51.120141 12824 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:51.121737  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1613) took 1034 ms. Trace:
I20260709 12:27:51.122490  5873 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:27:51.082887  6778 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:51.021607  6757 rpcz_store.cc:276] 0709 12:27:49.578888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.580975 (+  2087us) service_pool.cc:224] Handling call
0709 12:27:49.581355 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:49.732620 (+151265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:49.732652 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:49.740240 (+  7588us) log.cc:844] Serialized 26 byte log entry
0709 12:27:50.198119 (+457879us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b180012f1c0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:27:50.278100 (+ 79981us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:50.278172 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.966861 (+688689us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b540012a820. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bfadc0 00007f39d2c86a02 00007f39d2c83dde 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:27:50.967769 (+   908us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54002a21d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:50.967858 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:50.967949 (+    91us) raft_consensus.cc:1685] finished
0709 12:27:50.967971 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:50.968181 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14032,"raft.queue_time_us":129,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":6131,"spinlock_wait_cycles":715965952,"thread_start_us":268201,"threads_started":1,"wal-append.queue_time_us":271893}
I20260709 12:27:50.965448 12686 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:51.128201  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 170156us
I20260709 12:27:51.099700  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1797) took 1197 ms. Trace:
I20260709 12:27:50.978327 12752 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:51.065321 12866 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 12:27:51.101380  6018 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:51.122256  5865 rpcz_store.cc:276] 0709 12:27:50.087193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.091356 (+  4163us) service_pool.cc:224] Handling call
0709 12:27:51.121475 (+1030119us) spinlock_profiling.cc:243] Waited 85 ms on lock 0x7b5400193054. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:51.121660 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195584384}
W20260709 12:27:51.137446  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 76129.3us
W20260709 12:27:51.144974  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 44914.5us
I20260709 12:27:51.129794  6775 rpcz_store.cc:276] 0709 12:27:49.914882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.223187 (+308305us) service_pool.cc:224] Handling call
0709 12:27:50.252539 (+ 29352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:50.372996 (+120457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.373060 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:50.394547 (+ 21487us) log.cc:844] Serialized 26 byte log entry
0709 12:27:50.776847 (+382300us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:50.776921 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.777020 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:51.091795 (+314775us) raft_consensus.cc:1685] finished
0709 12:27:51.091820 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.099623 (+  7803us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54455,"raft.queue_time_us":54627,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":20373,"thread_start_us":296778,"threads_started":1,"wal-append.queue_time_us":538534}
W20260709 12:27:51.149180  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 173854us
I20260709 12:27:51.150086  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 1520) took 1565 ms. Trace:
I20260709 12:27:51.151639  5915 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:51.153138 12781 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:51.153769 12781 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:51.154235 12781 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:51.154291 12310 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:51.155071 12310 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.155460 12310 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:51.156364  6775 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e657475654794984a13c42a668f3158b"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:51.150645  6768 rpcz_store.cc:276] 0709 12:27:49.584420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.141692 (+557272us) service_pool.cc:224] Handling call
0709 12:27:50.142440 (+   748us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:50.212638 (+ 70198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.212692 (+    54us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:50.299887 (+ 87195us) log.cc:844] Serialized 78 byte log entry
0709 12:27:50.721143 (+421256us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:50.721750 (+   607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.722538 (+   788us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:51.064180 (+341642us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:51.149711 (+ 85531us) raft_consensus.cc:1685] finished
0709 12:27:51.149732 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.150000 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79613,"raft.queue_time_us":66088,"raft.run_cpu_time_us":616,"raft.run_wall_time_us":79208,"spinlock_wait_cycles":94748928,"thread_start_us":338372,"threads_started":1,"wal-append.queue_time_us":511425}
I20260709 12:27:51.157757 12542 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:51.157934  6775 raft_consensus.cc:606] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:51.161087  5838 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:51.159483 12825 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:51.091122 12726 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.163306 12726 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:51.165383 12755 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:51.165381 12825 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.171247 12825 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:51.166414 12555 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 12:27:51.174332  6766 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:51.165778 12354 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:51.175133 12354 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.092553  6019 rpcz_store.cc:276] 0709 12:27:48.945520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.945898 (+   378us) service_pool.cc:224] Handling call
0709 12:27:48.946389 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:48.946869 (+   480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:48.946898 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:50.042324 (+1095426us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b180043e460. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb642a6 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a
0709 12:27:50.092385 (+ 50061us) spinlock_profiling.cc:243] Waited 5.21 ms on lock 0x7b2c001ddd28. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:27:50.093526 (+  1141us) log.cc:844] Serialized 26 byte log entry
0709 12:27:50.240432 (+146906us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:50.259479 (+ 19047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.259654 (+   175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:51.072395 (+812741us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:51.080187 (+  7792us) raft_consensus.cc:1685] finished
0709 12:27:51.080210 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.080415 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3851,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":13929,"spinlock_wait_cycles":435432064,"thread_start_us":133742,"threads_started":1,"wal-append.queue_time_us":510050}
I20260709 12:27:51.175652 12354 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:51.158113 12776 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:51.176458 12776 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.176919 12776 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:51.176782  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:51.178632  5941 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
W20260709 12:27:51.079303  5696 rpcz_store.cc:269] 0709 12:27:30.649213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.706460 (+ 57247us) service_pool.cc:224] Handling call
0709 12:27:51.040949 (+20334489us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:41.471367 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:41.472150 (+   783us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:41.472211 (+    61us) write_op.cc:435] Acquired schema lock
0709 12:27:41.472229 (+    18us) tablet.cc:662] Decoding operations
0709 12:27:41.474005 (+  1776us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:41.474134 (+   129us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:41.474180 (+    46us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:27:41.476507 (+  2327us) tablet.cc:701] Row locks acquired
0709 12:27:41.476557 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:27:41.476723 (+   166us) write_op.cc:270] Start()
0709 12:27:41.476875 (+   152us) write_op.cc:276] Timestamp: P: 1783600061476660 usec, L: 0
0709 12:27:41.476910 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 12:27:41.482691 (+  5781us) log.cc:844] Serialized 7793 byte log entry
0709 12:27:43.330169 (+1847478us) op_driver.cc:464] REPLICATION: finished
0709 12:27:43.551990 (+221821us) write_op.cc:301] APPLY: starting
0709 12:27:43.604333 (+ 52343us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:44.041827 (+437494us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:44.069759 (+ 27932us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:45.370815 (+1301056us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:45.371370 (+   555us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 12:27:45.371451 (+    81us) write_op.cc:312] APPLY: finished
0709 12:27:45.372143 (+   692us) log.cc:844] Serialized 179 byte log entry
0709 12:27:45.537999 (+165856us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:45.562589 (+ 24590us) write_op.cc:454] Released schema lock
0709 12:27:45.722654 (+160065us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:45.767604 (+ 44950us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:45.767646 (+    42us) write_op.cc:454] Released schema lock
0709 12:27:46.568932 (+801286us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
Metrics: {"mutex_wait_us":5790860,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":178467,"apply.run_cpu_time_us":65804,"apply.run_wall_time_us":2224664,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87807,"num_ops":20,"prepare.queue_time_us":514984,"prepare.run_cpu_time_us":43683,"prepare.run_wall_time_us":857461,"raft.queue_time_us":234313,"raft.run_cpu_time_us":9084,"raft.run_wall_time_us":111232,"replication_time_us":1853060,"spinlock_wait_cycles":31557888,"thread_start_us":1523519,"threads_started":5,"wal-append.queue_time_us":1331956,"wal-append.run_cpu_time_us":25674,"wal-append.run_wall_time_us":3690867}]]}
I20260709 12:27:51.164089  5915 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:51.159662  6775 raft_consensus.cc:614] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:51.186573  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:51.189839  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:51.190588  5948 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:27:51.192649  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:51.186033  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:51.207264  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:51.212446  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:51.204295 12613 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:51.222798  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.222971  5940 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:51.221526  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:51.193841 12726 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:51.226603  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:51.219017 12686 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:51.227852  5943 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:27:51.225579  5838 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:51.219489 12783 consensus_meta.cc:287] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.628s	user 0.017s	sys 0.000s
I20260709 12:27:51.237031  5943 pending_rounds.cc:85] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 12
I20260709 12:27:51.248657  5943 pending_rounds.cc:107] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:27:51.249058  5943 raft_consensus.cc:2889] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:51.229120  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:51.238433  5838 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:51.239199  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:51.240581 12686 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:51.241308  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1799) took 1325 ms. Trace:
W20260709 12:27:51.250200  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 86592.1us
I20260709 12:27:51.251368  6766 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:51.243153 12796 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:51.244292 12874 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:51.245565 12769 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:51.256997  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:51.256107  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:51.264726  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:51.250661  6773 rpcz_store.cc:276] 0709 12:27:49.915824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.261779 (+345955us) service_pool.cc:224] Handling call
0709 12:27:50.302875 (+ 41096us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:50.437207 (+134332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.437508 (+   301us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:50.520187 (+ 82679us) log.cc:844] Serialized 26 byte log entry
0709 12:27:50.842183 (+321996us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:50.842234 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.842336 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:51.241013 (+398677us) raft_consensus.cc:1685] finished
0709 12:27:51.241040 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.241233 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21775,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":36653,"thread_start_us":282208,"threads_started":1,"wal-append.queue_time_us":603178}
W20260709 12:27:51.265888  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 79938.2us
I20260709 12:27:51.194041 12719 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:51.268813  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:51.269862 12503 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:51.270927 12863 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.271579 12837 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.275472 12686 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:51.276392  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:51.276782 12070 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:51.253638 12824 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:51.278617  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.280040  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:51.294974 12824 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:51.295704  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:51.296344  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:51.297502  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:51.286755  5869 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:51.238026 12613 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:51.258946 12783 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:51.286464 12863 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:51.292634 12837 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:51.301997  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:51.309660 12479 raft_consensus.cc:2764] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:27:51.309854 12354 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:51.312497 12354 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:51.314567 12354 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.315331 12354 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:51.322753 12479 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:51.297842 12769 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:51.303876 12686 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:51.330679  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 66113.7us
I20260709 12:27:51.304461 12874 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:27:51.339936 12871 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:27:51.302865 12783 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.307660 12717 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.343875 12686 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.349875 12783 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:51.386538 12686 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:51.389665 12354 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:51.388841 12712 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.016s	sys 0.018s
W20260709 12:27:51.429842  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 338636us
I20260709 12:27:51.370325 12717 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:51.426093 12712 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.016s	sys 0.019s
I20260709 12:27:51.439162  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1430) took 1319 ms. Trace:
W20260709 12:27:51.431533  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 266929us
I20260709 12:27:51.410019 12752 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:51.441210  5871 rpcz_store.cc:276] 0709 12:27:50.119457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.315659 (+196202us) service_pool.cc:224] Handling call
0709 12:27:50.365364 (+ 49705us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:50.410814 (+ 45450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.410881 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:50.517950 (+107069us) log.cc:844] Serialized 26 byte log entry
0709 12:27:50.875165 (+357215us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:50.875210 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.875288 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:51.438813 (+563525us) raft_consensus.cc:1685] finished
0709 12:27:51.438867 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.439078 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7157,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":17670,"thread_start_us":311237,"threads_started":1,"wal-append.queue_time_us":569141}
I20260709 12:27:51.446334  5915 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:51.450459  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:51.427012 12825 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:51.467072  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:51.469321  6766 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:51.456580 12874 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:51.481478  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 179560us
I20260709 12:27:51.484417  5915 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:51.486305  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:51.487790 12475 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:51.472643 12755 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:51.476704 12070 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:51.481923  5987 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.195:43329 timed out after 1.768s (SENT) [suppressed 10 similar messages]
I20260709 12:27:51.474905 12719 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:51.498633 12780 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:51.500334 12780 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:51.500605 12837 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:51.501571 12780 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:51.503293 12780 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:51.504534 12070 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:51.488263 12475 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.498498 12874 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:51.533118  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 48859us
I20260709 12:27:51.535821 12717 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:51.490283 12825 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
W20260709 12:27:51.537431 12867 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.733s	user 0.001s	sys 0.006s
I20260709 12:27:51.526124 12712 consensus_queue.cc:1059] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 12:27:51.549928  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:51.564121  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.564392 12712 raft_consensus.cc:3055] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:51.564800 12712 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:51.565169 12820 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.540292 12822 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:51.556269  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:51.554679  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:51.554872  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:51.555744  6773 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 24 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:27:51.541706 12863 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:51.556923  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:51.557875  6793 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:51.532370 12475 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:51.491808 12613 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:51.488456 12752 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 12:27:51.565582 12712 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:51.566013 12824 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:51.548288 12876 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.167s	user 0.003s	sys 0.000s
I20260709 12:27:51.606158  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:51.606910  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:51.577926 12576 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.009s	sys 0.031s
I20260709 12:27:51.607909  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:51.608222 12576 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.009s	sys 0.031s
I20260709 12:27:51.608641  6773 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:51.555069  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:51.579766  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:51.610702  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 55747.7us
W20260709 12:27:51.581151 12881 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:27:51.611945  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:51.612246  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:51.612820  6766 raft_consensus.cc:2393] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 24: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:51.613163 12796 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:51.613508  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:51.614908  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:51.552034 12070 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:51.586767 12770 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:51.547245 12781 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:51.589881 12503 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.616029 12770 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:51.616575 12503 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:51.618820 12770 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:51.567394  6785 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:51.567512  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:51.619087 12503 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:51.619952  5869 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:51.567128  5871 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:51.591848  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:51.621802 12781 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:51.596096  6019 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:51.597213  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:51.623528  6018 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:51.623832  6775 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:51.561432  6762 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:51.598338  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:51.625242  6015 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:51.560863 12817 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:51.600067  6016 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:51.567894  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:51.568326  6024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 1594) took 1986 ms (client timeout 1823 ms). Trace:
I20260709 12:27:51.594278 12310 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:51.569237  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:51.569542  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:51.571087  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:51.602975  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.610396 12822 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:51.582998 12727 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:51.618707  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:51.621908 12712 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:51.610179  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:51.590051  6611 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:51.641372  6778 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.004s
W20260709 12:27:51.654793  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 166906us
I20260709 12:27:51.708583  6785 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:51.626914  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:51.672124  6024 rpcz_store.cc:269] 0709 12:27:49.582156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.582313 (+   157us) service_pool.cc:224] Handling call
0709 12:27:51.568034 (+1985721us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b54003642d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:51.568225 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":960976768}
I20260709 12:27:51.738343  6778 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:51.641016  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:51.644950  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:51.580799 12515 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.724s	user 0.009s	sys 0.004s
I20260709 12:27:51.620913  5945 raft_consensus.cc:2410] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
W20260709 12:27:51.668506  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.642283  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:51.625506  6775 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 12:27:51.715041  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:51.739717  6766 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
W20260709 12:27:51.693727  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.747491  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:51.737226  6793 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:51.737010  5948 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:51.714953  6756 raft_consensus.cc:2368] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24.
I20260709 12:27:51.747656  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:51.749954  6785 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 16.
I20260709 12:27:51.749130  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:51.750140  6778 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:51.750902  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:51.620836 12555 raft_consensus.cc:3055] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:51.745651  6775 pending_rounds.cc:85] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 12
I20260709 12:27:51.778321  6775 pending_rounds.cc:107] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:27:51.778748  6775 raft_consensus.cc:2889] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:51.759096  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:51.759663  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:51.759991  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 145145us
I20260709 12:27:51.760685  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:51.766098  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:51.781342  6018 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 21: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:51.766922  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:51.767575 12863 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.767367 12887 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:51.787901  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:51.788265  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:51.788689  6016 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:51.789664  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:51.766801  6768 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 27.
I20260709 12:27:51.768635 12888 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.774636  5869 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:51.774412  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:51.798487  6024 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:51.777179 12781 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.567765 12760 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:51.571152 12647 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:51.779206 12515 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:51.706193 12767 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:51.821843  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:51.822687  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:51.793176  6023 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:51.795324  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:51.795512 12717 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:51.796023 12825 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:51.800011  6019 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:27:51.791531  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:51.742020 12796 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:27:51.774299  5840 connection.cc:256] RPC call timeout handler was delayed by -1.3068487959994854s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
I20260709 12:27:51.796248  6609 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:51.860064  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 63895us
I20260709 12:27:51.642829 12686 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:51.708319 12783 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:51.811578  6766 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21. Ops: []
W20260709 12:27:51.813140  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 223169us
I20260709 12:27:51.600104 12776 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:51.566067 12820 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:51.823803  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 64845.4us
I20260709 12:27:51.920042  5915 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:51.778419 12727 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:51.779423 12555 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:51.830518  6016 raft_consensus.cc:2393] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:51.830961 12780 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:51.832324  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:51.934813  5945 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:27:51.935276  5869 raft_consensus.cc:2410] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:27:51.833734  6021 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:27:51.833968  5690 catalog_manager.cc:5697] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:51.839581 12475 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:51.949932  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:51.842691 12515 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:51.843535  5943 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:27:51.804843 12781 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:51.859064  5840 connection.cc:256] RPC call timeout handler was delayed by -1.0602487959995415s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19890000000000002s for a response
I20260709 12:27:51.903312  6773 raft_consensus.cc:2410] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:27:51.903584  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:51.905200  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.910180  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:51.913414  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:51.963142  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 49831.4us
I20260709 12:27:51.945540 12315 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.925959  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:51.929970  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:51.970783  6019 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:51.857896  5948 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:51.939090  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:51.971468  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 51554.4us
I20260709 12:27:51.943054  6756 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:51.948880  6020 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:51.912289 12892 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:51.811401 12863 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:51.964771  5944 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 19: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:51.964088  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:51.926252  6766 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:51.967149  6778 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:51.967566  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:51.971736  6023 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21. Ops: []
W20260709 12:27:51.861275 12708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.008s	sys 0.033s
I20260709 12:27:51.929232 12717 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:51.934487 12775 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:52.003784  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:51.977730  6773 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 21: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:52.008152  6021 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:27:52.008818  6018 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:51.960528 12837 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:52.014644  6016 raft_consensus.cc:2410] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:27:51.902624 12767 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:52.015787  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:52.016124  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 106056us
W20260709 12:27:52.001724 12708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.008s	sys 0.034s
I20260709 12:27:52.026189  5869 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:51.964857 12647 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:51.977689 12613 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:52.025206  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:52.034157  6756 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:27:52.001724 12503 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:51.979730 12892 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.951752 12727 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:52.023432  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:52.038062  5913 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:52.038868 12772 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:52.099632  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:52.111505  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 73519.1us
I20260709 12:27:52.021389 12837 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:52.040259  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:51.958956 12354 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:27:52.088994  6766 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 24: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:27:52.094806  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 52706.7us
I20260709 12:27:52.095552 12719 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:52.097121 12708 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:52.135463  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:52.034219 12613 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.136919 12613 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:52.083763 12892 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:52.122447  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:52.020754 12775 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:52.098881 12772 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:52.142679  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:52.013450 12315 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:52.144759  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:52.149457  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:52.119629 12837 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:52.117594 12310 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:52.135041 12729 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.033s	sys 0.054s
W20260709 12:27:52.182338 12729 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.033s	sys 0.054s
I20260709 12:27:52.122928 12515 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:52.098471 12647 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:52.084679 12796 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.057s
W20260709 12:27:52.137476  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 97278.8us
I20260709 12:27:52.150084  6023 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:52.183594  6016 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:27:52.185060  6778 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:52.041270 12887 consensus_queue.cc:1059] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
W20260709 12:27:52.189464  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.149170 12772 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:52.206140 12887 raft_consensus.cc:3055] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:27:52.211288  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 50460.4us
I20260709 12:27:52.197619  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:52.208364 12895 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:27:52.185818 12515 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.225855  6024 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:27:52.202494 12780 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.138391 12896 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.103s
W20260709 12:27:52.248427  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 284409us
I20260709 12:27:52.249163  6016 pending_rounds.cc:85] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 8
I20260709 12:27:52.255782  6016 pending_rounds.cc:107] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:27:51.814841 12888 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:52.123209 12874 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.103s
I20260709 12:27:52.213613 12887 raft_consensus.cc:740] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:52.238603 12515 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:52.260869  6016 raft_consensus.cc:2889] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:52.267745  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46174us
I20260709 12:27:52.244307 12783 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:52.247187 12796 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:52.246416 12686 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:52.253139 12503 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:52.319363  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:52.322692  6023 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:52.259846 12820 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:52.331584  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 209253us
I20260709 12:27:52.266158 12475 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:52.269240 12776 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:52.338768  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:52.310631 12755 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:52.209851 12717 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:52.315050 12775 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:52.324594 12781 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:52.252979 12780 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:52.364866  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:52.365109  5872 consensus_meta.cc:287] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.617s	user 0.010s	sys 0.001s
I20260709 12:27:52.372779  5872 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:52.335150 12863 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:52.340243 12896 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:52.214329  5690 catalog_manager.cc:5697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 6 to 14. New cstate: current_term: 14 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:52.412643  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:52.360844 12894 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.342s	user 0.002s	sys 0.000s
I20260709 12:27:52.413869  6023 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
W20260709 12:27:52.490375  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 372641us
I20260709 12:27:52.342541 12755 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:52.366044 12475 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:52.491258  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:52.492236  5944 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:52.359221 12727 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:52.458491  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:52.503046  6019 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:52.462483  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:52.475481 12503 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:27:52.480662  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.482879 12909 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:52.487720 12315 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:52.492002  5913 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:52.506129  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:52.536455  5949 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:52.504029  6017 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
W20260709 12:27:52.512293 12906 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:52.385959 12767 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:52.528152 12907 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:27:52.359891 12887 consensus_queue.cc:260] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:52.529088 12916 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:52.549878 12916 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:52.550457 12916 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:52.552887 12647 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:52.505328  5873 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:27:52.381127 12775 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.108s
I20260709 12:27:52.339241 12555 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:52.565249  5944 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
W20260709 12:27:52.542164  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 50231.4us
I20260709 12:27:52.416021 12896 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:52.498471 12475 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.518705 12863 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:27:52.547232  5690 catalog_manager.cc:5697] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 8 to 16. New cstate: current_term: 16 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:27:52.570518  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:52.633524 12475 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:52.636314  5948 raft_consensus.cc:2410] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 23 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
W20260709 12:27:52.640057  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.578534  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:52.584961  5949 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
I20260709 12:27:52.585853 12888 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:52.587932 12613 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:52.594177 12755 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:52.596722 12515 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:52.597997 12555 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:52.598866  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 560397us
W20260709 12:27:52.602219 12712 consensus_meta.cc:287] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.858s	user 0.012s	sys 0.003s
I20260709 12:27:52.557686 12315 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:52.585180 12837 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:52.612450 12772 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:52.619117  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 254320us
W20260709 12:27:52.620069  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:52.685056  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.620675 12914 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:52.623123 12898 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.354s	user 0.000s	sys 0.003s
I20260709 12:27:52.689858  5913 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:52.627352 12780 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:52.628778  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.570235  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:52.697348  6756 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 24: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:52.697999  5944 raft_consensus.cc:2368] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18.
W20260709 12:27:52.557307 12894 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:27:52.649537  6019 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:52.641551  5940 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
W20260709 12:27:52.654424  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 340033us
I20260709 12:27:52.657101  5949 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23. Ops: [20.12-22.13]
I20260709 12:27:52.644037 12503 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:52.675200  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:52.689733  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 433825us
I20260709 12:27:52.699705  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:52.717415  5873 raft_consensus.cc:2410] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:27:52.698335  5838 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:52.717990  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:52.718606  5874 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:52.719020  5838 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:52.720655  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:52.723300  5943 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:52.720436  5838 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:52.683363 12727 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.689195 12874 raft_consensus.cc:3055] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:52.634815  5690 catalog_manager.cc:5697] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 10 to 13, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 13 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:52.716801 12576 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:52.746237 12686 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:52.727862 12542 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:52.750952  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:52.754271  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:52.756088  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2099) took 1531 ms. Trace:
I20260709 12:27:52.791293  5941 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
W20260709 12:27:52.765952  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.768307 12310 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:52.740072  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:52.779562  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 59205.9us
I20260709 12:27:52.723449 12315 raft_consensus.cc:697] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:52.748363  5840 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:52.780500  5871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.005s	sys 0.034s
W20260709 12:27:52.803826  5871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.006s	sys 0.034s
W20260709 12:27:52.797267  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 463829us
W20260709 12:27:52.802681  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 58670.8us
I20260709 12:27:52.737022 12755 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:52.788311 12898 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:27:52.813792  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:52.814133  5944 raft_consensus.cc:2393] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 21: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:52.726130 12888 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 12:27:52.743284 12874 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:52.797026  5838 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:52.822429  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 363096us
I20260709 12:27:52.837878  5990 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:52.789358 12686 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.791594  5947 rpcz_store.cc:276] 0709 12:27:51.225081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.280087 (+ 55006us) service_pool.cc:224] Handling call
0709 12:27:51.291081 (+ 10994us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:51.349184 (+ 58103us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:27:51.404576 (+ 55392us) log.cc:844] Serialized 14 byte log entry
0709 12:27:51.798427 (+393851us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:51.862349 (+ 63922us) log.cc:844] Serialized 26 byte log entry
0709 12:27:51.862774 (+   425us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:51.862852 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.008398 (+145546us) spinlock_profiling.cc:243] Waited 36.4 ms on lock 0x7b540043f8d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:52.008479 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:52.755610 (+747131us) raft_consensus.cc:1685] finished
0709 12:27:52.755765 (+   155us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.755990 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8571,"prepare.queue_time_us":58385,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":3959,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":7614,"spinlock_wait_cycles":85154944,"thread_start_us":364827,"threads_started":1,"wal-append.queue_time_us":508900}
I20260709 12:27:52.788555 12914 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:52.725353 12503 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:52.789762 12780 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.828330  5874 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
W20260709 12:27:52.809937 12719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.005s	sys 0.028s
W20260709 12:27:52.848045  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 51089.2us
I20260709 12:27:52.867286  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:52.867532  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:52.811152 12576 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:52.869290  6611 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:52.782335 12727 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:52.848759 12686 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:52.739163 12896 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:52.886456  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:52.851487 12783 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:52.815968 12863 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:27:52.889096  5869 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:52.849998 12752 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:27:52.871860 12719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.005s	sys 0.028s
W20260709 12:27:52.890640  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 77008us
W20260709 12:27:52.894289  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 146067us
I20260709 12:27:52.904903  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:52.877066 12914 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:52.884615 12915 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:52.914451 12837 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:52.916920 12888 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:52.918179  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 88397.4us
I20260709 12:27:52.914451  5840 leader_election.cc:400] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24.
I20260709 12:27:52.918951  5840 leader_election.cc:403] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
W20260709 12:27:52.914695 12921 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 12:27:52.875638 12780 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:52.924511  6757 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16.
I20260709 12:27:52.926148  5866 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 12:27:52.887316 12927 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:52.937052  5866 pending_rounds.cc:85] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 10
I20260709 12:27:52.937775  5866 pending_rounds.cc:107] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 21.11
I20260709 12:27:52.938138  5866 raft_consensus.cc:2889] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:52.946435  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:52.878592 12929 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 12:27:52.950713  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:52.897238 12515 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:52.953166  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:52.905500 12752 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:52.972173  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:52.972936  5873 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:52.920979 12783 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:52.976361  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:52.925806 12893 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:52.911329 12934 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:27:52.990424  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.986358  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:52.942799 12775 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:52.942713 12927 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.993029  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:52.993145  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:52.995373  5867 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:52.993269  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1690) took 1085 ms. Trace:
I20260709 12:27:52.950119 12613 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:52.949993 12915 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 24 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:52.999262  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:53.005084  5873 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 17: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:53.002228 12927 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:52.999604  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:53.010123  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:53.006829  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2173) took 1235 ms. Trace:
I20260709 12:27:52.917575 12888 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:53.013401  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:53.022702  5867 raft_consensus.cc:2376] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24. Re-sending same reply.
I20260709 12:27:52.936125 12837 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
W20260709 12:27:53.014477  6609 connection.cc:256] RPC call timeout handler was delayed by -1.3593440219997319s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
I20260709 12:27:53.018330  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:53.014647  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 124868us
I20260709 12:27:53.005800  5913 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:53.019315  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 140683us
I20260709 12:27:53.026024  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:53.026840  5873 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
I20260709 12:27:53.016382  5871 rpcz_store.cc:276] 0709 12:27:51.771552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.775122 (+  3570us) service_pool.cc:224] Handling call
0709 12:27:51.775443 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:51.775763 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:52.017197 (+241434us) log.cc:844] Serialized 14 byte log entry
0709 12:27:52.812710 (+795513us) log.cc:844] Serialized 14 byte log entry
0709 12:27:53.006129 (+193419us) log.cc:844] Serialized 14 byte log entry
0709 12:27:53.006320 (+   191us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:53.006388 (+    68us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:53.006467 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.006571 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:53.006763 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32064,"prepare.queue_time_us":433276,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":43872,"raft.run_cpu_time_us":468,"raft.run_wall_time_us":47123,"spinlock_wait_cycles":25050240,"thread_start_us":1169285,"threads_started":3,"wal-append.queue_time_us":896969}
I20260709 12:27:52.994043 12775 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.989173 12893 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:53.030882  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:53.024772  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:53.000614 12770 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24.
I20260709 12:27:52.938730 12315 consensus_queue.cc:237] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:53.037657  5866 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:53.001965 12772 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:52.997838  6022 rpcz_store.cc:276] 0709 12:27:51.731577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.731764 (+   187us) service_pool.cc:224] Handling call
0709 12:27:51.758267 (+ 26503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:51.758611 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:52.029574 (+270963us) log.cc:844] Serialized 14 byte log entry
0709 12:27:52.815842 (+786268us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:52.815940 (+    98us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:52.816019 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.816121 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.816324 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":419775,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":3818,"raft.queue_time_us":59251,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":8392,"thread_start_us":968439,"threads_started":3,"wal-append.queue_time_us":416310}
I20260709 12:27:52.909992 12874 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:53.047327  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:53.057899  5870 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 20: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:53.061481  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2179) took 1131 ms. Trace:
W20260709 12:27:53.062678  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 149963us
I20260709 12:27:53.015868 12933 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.038166 12775 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:53.065810  5868 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 23.
I20260709 12:27:53.066697  5865 rpcz_store.cc:276] 0709 12:27:51.930519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.930815 (+   296us) service_pool.cc:224] Handling call
0709 12:27:51.953495 (+ 22680us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:51.955031 (+  1536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:51.955074 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:51.971312 (+ 16238us) log.cc:844] Serialized 26 byte log entry
0709 12:27:52.417024 (+445712us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:52.417067 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.417149 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:53.061163 (+644014us) raft_consensus.cc:1685] finished
0709 12:27:53.061189 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:53.061409 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":386563,"threads_started":1,"wal-append.queue_time_us":707217}
I20260709 12:27:53.073186  5915 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:53.044826 12893 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:53.043776 12863 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:27:53.092659  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 67995.2us
W20260709 12:27:53.064265 12760 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.252s	user 0.014s	sys 0.002s
I20260709 12:27:53.096935  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:53.031353 12916 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:52.998342 12576 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:53.114748  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:53.115531  5942 raft_consensus.cc:2410] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 25 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:27:53.115872  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:53.107674  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:53.118362  6019 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:27:53.123088  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:53.128291  6022 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:53.128974  6020 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 22: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:53.147356  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:53.148388  6017 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:53.149250  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:53.071487 12822 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
W20260709 12:27:53.161228  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 134908us
W20260709 12:27:53.165282  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 212219us
I20260709 12:27:53.167303  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:53.113458 12874 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:53.168653  6018 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:27:53.166177  6611 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:53.170480  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:53.091673 12772 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:27:53.122762 12932 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:53.122416 12825 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.188889  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:53.160508 12503 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:53.169422 12717 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:53.190853  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:53.080945 12933 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:53.091531 12752 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:53.207141 12937 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:27:53.204356  6018 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:53.197031  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:53.218132  6609 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:53.179138 12934 raft_consensus.cc:697] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:53.103981 12783 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:53.283934  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:53.194090 12925 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.263s	user 0.000s	sys 0.002s
I20260709 12:27:53.196187 12825 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:53.075420 12914 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:53.242242  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.298470  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:53.208599 12930 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:27:53.385218 12475 consensus_meta.cc:287] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.751s	user 0.010s	sys 0.010s
W20260709 12:27:53.421029  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 224080us
W20260709 12:27:53.297538 12936 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:27:53.428438  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:53.385210  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:53.430155  6019 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:53.289330 12939 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 12:27:53.189518 12760 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:53.427002 12930 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.436143 12930 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:53.381680 12542 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.000s	sys 0.010s
I20260709 12:27:53.436369 12475 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:53.190253 12863 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:53.437407 12542 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:53.196323 12945 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:53.438650 12863 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:53.437945 12767 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:53.446679  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 414270us
I20260709 12:27:53.448441  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:53.438269  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 249450us
I20260709 12:27:53.209808 12780 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:53.447803  5990 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:53.211725 12772 consensus_queue.cc:1243] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:53.449338  6018 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:53.466627  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:53.468652  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 249851us
I20260709 12:27:53.469194 12717 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:53.294512 12940 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
I20260709 12:27:53.471593  5990 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:53.395982 12892 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.208s	user 0.009s	sys 0.010s
I20260709 12:27:53.165848 12915 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:53.475670  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 47400.1us
I20260709 12:27:53.481262  6019 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:27:53.209054 12929 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:53.484141  5990 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:53.209610 12727 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:53.488833  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:53.447990 12767 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:53.472579 12503 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:53.494590 12947 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:53.495694 12970 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:53.498307 12948 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.504642 12515 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:53.516055 12948 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:53.516616 12822 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:53.192045  5690 catalog_manager.cc:5697] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:53.498024 12958 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:53.498234 12953 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:53.518051 12717 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:53.499068 12944 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 12:27:53.498744 12952 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.113s
W20260709 12:27:53.498924 12935 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.448s	user 0.001s	sys 0.001s
I20260709 12:27:53.497042  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1605) took 1266 ms. Trace:
W20260709 12:27:53.497699 12931 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.448s	user 0.002s	sys 0.000s
I20260709 12:27:53.526019 12950 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:53.506958 12934 consensus_queue.cc:237] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:53.526789 12950 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:53.472925 12888 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:53.529769  6016 rpcz_store.cc:276] 0709 12:27:52.230591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.242748 (+ 12157us) service_pool.cc:224] Handling call
0709 12:27:52.243248 (+   500us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:52.366329 (+123081us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:52.366389 (+    60us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:53.064447 (+698058us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b2c001ddd28. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:27:53.065190 (+   743us) log.cc:844] Serialized 78 byte log entry
0709 12:27:53.284768 (+219578us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:53.285670 (+   902us) log.cc:844] Serialized 14 byte log entry
0709 12:27:53.394046 (+108376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.394162 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:53.496683 (+102521us) raft_consensus.cc:1685] finished
0709 12:27:53.496794 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:53.496974 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":107887,"prepare.queue_time_us":332,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":175022,"raft.run_cpu_time_us":404,"raft.run_wall_time_us":70718,"spinlock_wait_cycles":66966784,"thread_start_us":92847,"threads_started":1,"wal-append.queue_time_us":398705}
I20260709 12:27:53.511346  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:53.512436  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:53.507078 12947 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:53.516023 12970 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:53.207448 12820 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.283s
I20260709 12:27:53.494756 12475 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:53.493537 12767 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:53.296231 12933 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:53.499940 12955 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 12:27:53.549209  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 60371.2us
I20260709 12:27:53.567741  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:53.438642 12542 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:53.538319 12727 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:53.531260 12772 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:53.540042 12822 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.546798 12947 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:53.552460 12515 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:53.518899 12953 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:53.553875 12555 consensus_meta.cc:287] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.886s	user 0.013s	sys 0.003s
I20260709 12:27:53.529505 12950 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:53.531761 12948 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:53.522085 12952 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:53.548765 12929 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:53.552922  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 399320us
I20260709 12:27:53.588354  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:53.582018 12727 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:53.590982 12916 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:53.612504  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:27:53.614419 12904 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 1.272s	user 0.003s	sys 0.002s
W20260709 12:27:53.614506 12938 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.448s	user 0.001s	sys 0.001s
I20260709 12:27:53.599802 12953 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:53.610769 12929 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:53.608475 12952 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:53.644240 12952 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:53.617000 12957 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:53.620451 12727 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:53.621819 12775 consensus_meta.cc:287] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.002s
W20260709 12:27:53.622905 12941 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:27:53.622673  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:53.591524 12515 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:53.625648 12963 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:27:53.677448  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:53.677932 12963 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:53.678927  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:53.679289 12963 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:53.610468 12966 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:53.608294 12948 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:27:53.592201 12820 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:27:53.629345 12950 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:53.599524 12780 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:27:53.591830 12822 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:53.646795 12310 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:53.599464 12958 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:27:53.682958  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:53.648303 12887 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:53.672245 12775 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:53.487017 12892 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:53.692607  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1613) took 1934 ms. Trace:
I20260709 12:27:53.680377 12515 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:53.518971  5690 catalog_manager.cc:5697] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 8 to 12, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 12 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:53.679015 12923 raft_consensus.cc:3055] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:53.705915  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:53.709796  6775 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 24: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:53.746076  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:53.755836  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:53.757400  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:53.752660  6021 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:27:53.745836  6015 rpcz_store.cc:276] 0709 12:27:51.757873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.811476 (+ 53603us) service_pool.cc:224] Handling call
0709 12:27:51.828186 (+ 16710us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:51.833202 (+  5016us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:51.833242 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:51.833987 (+   745us) log.cc:844] Serialized 26 byte log entry
0709 12:27:52.111551 (+277564us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:52.111614 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.111712 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:53.572337 (+1460625us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:53.692284 (+119947us) raft_consensus.cc:1685] finished
0709 12:27:53.692306 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:53.692534 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4415,"raft.queue_time_us":6081,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":108,"spinlock_wait_cycles":24484224,"thread_start_us":277053,"threads_started":1,"wal-append.queue_time_us":417681}
W20260709 12:27:53.766335  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 211261us
I20260709 12:27:53.770251  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:53.692824 12310 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:53.794001  6756 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:53.797124  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:53.797832  6768 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 20: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:27:53.763317 12874 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.576s	user 0.004s	sys 0.008s
I20260709 12:27:53.797917  6015 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:53.690176 12950 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:53.753991 12923 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:53.831419  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:53.828827  6775 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 12:27:53.821609  6017 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.646s	user 0.007s	sys 0.006s
I20260709 12:27:53.832947  6017 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
W20260709 12:27:53.833482  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.832213  6015 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:53.845345  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:53.831151 12938 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:27:53.688088 12966 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:53.850854  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:53.841455 12965 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:53.842929 12969 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:53.848052 12820 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:53.890110  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:53.855121 12964 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:53.852648 12874 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:53.857188 12970 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:53.855631 12956 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:53.896404 12975 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
W20260709 12:27:53.917023  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 73162.4us
I20260709 12:27:53.901305 12820 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:53.902397 12954 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:27:53.935796  5838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.005s	sys 0.008s
W20260709 12:27:53.937048  5838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.005s	sys 0.008s
I20260709 12:27:53.907042 12892 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:53.908098 12874 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.942500  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2110) took 1452 ms. Trace:
I20260709 12:27:53.903465 12972 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:53.937422 12755 consensus_meta.cc:287] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.120s	user 0.011s	sys 0.004s
I20260709 12:27:53.893244 12969 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:53.971863  5943 rpcz_store.cc:276] 0709 12:27:52.489956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.490855 (+   899us) service_pool.cc:224] Handling call
0709 12:27:52.528597 (+ 37742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:52.729237 (+200640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.942176 (+1212939us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54006761d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:53.942419 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":447084416}
I20260709 12:27:53.927866 12820 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:53.987354  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:53.984760  5840 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:53.996352  5874 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
W20260709 12:27:54.000706  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 388267us
W20260709 12:27:53.820650 12893 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.714s	user 0.008s	sys 0.006s
I20260709 12:27:54.003859  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:54.059572  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:54.005334  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:54.005806  5840 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:54.011250  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:54.012178  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:53.899456 12310 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:54.011957 12914 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:53.897732 12965 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:53.947258 12934 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:53.981695 12874 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:53.982514 12971 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.058806  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:54.000100 12555 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.062182  5872 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:54.066115  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 60475.3us
I20260709 12:27:54.068441  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:53.945824 12719 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.108s
I20260709 12:27:54.005596 12929 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:53.926954 12956 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:53.911998 12923 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:54.098090  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:54.065904 12893 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:53.984963 12783 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.078960  6778 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:54.079157  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:54.018947 12775 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:54.105072  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 101299us
I20260709 12:27:54.107163  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:54.079368 12954 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:54.074060 12914 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:54.119625  5990 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:54.123878  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:54.125211  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:54.071261 12945 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.004s	sys 0.026s
I20260709 12:27:53.998622 12972 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:54.078784 12915 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:27:54.135044  5940 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:27:54.137512  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:54.140115  5949 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:54.103125 12822 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:54.126564 12686 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.230s	user 0.013s	sys 0.003s
I20260709 12:27:53.923731  5690 catalog_manager.cc:5697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 10 to 11, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 11 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:54.138159  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:54.059583 12932 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 12:27:54.163357  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:54.163905  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.117790 12542 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:27:54.135308 12945 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.004s	sys 0.027s
I20260709 12:27:54.114565 12954 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:54.167827  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:54.099108 12555 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:54.093416 12934 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:54.172241  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 58469.8us
I20260709 12:27:54.092516 12971 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:54.199092 12971 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:54.199281  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1448) took 1218 ms. Trace:
I20260709 12:27:54.130795 12914 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:54.171406  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:54.175232  6778 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:54.178014  5872 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:54.174218 12686 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:54.226462  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:54.227062 12934 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:54.227281  6023 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:54.228446 12686 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:54.229373  5869 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 23: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:54.233033  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:54.234097  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:54.161214 12950 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:54.235085  5871 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:54.226565 12503 raft_consensus.cc:3055] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:54.236436 12503 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:54.237376 12503 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:54.238658 12503 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:54.238890  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:54.233747  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 1540) took 2288 ms. Trace:
I20260709 12:27:54.242642  6024 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:54.243379  6022 raft_consensus.cc:1240] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21. Ops: [20.10-20.10]
W20260709 12:27:54.246683 12985 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 12:27:54.246809  5869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1640) took 1295 ms (client timeout 1592 ms). Trace:
W20260709 12:27:54.247478  5869 rpcz_store.cc:269] 0709 12:27:52.951065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.970544 (+ 19479us) service_pool.cc:224] Handling call
0709 12:27:54.246559 (+1276015us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400388954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:54.246739 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2837714816}
I20260709 12:27:54.249169 12310 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:54.170861 12943 consensus_queue.cc:1059] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.316s
I20260709 12:27:54.191416 12954 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:54.191401  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:54.237244 12896 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:54.114657 12783 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:54.111892 12993 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:54.194659 12965 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:54.196894 12893 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:54.185360 12727 consensus_meta.cc:287] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.510s	user 0.008s	sys 0.001s
I20260709 12:27:54.199879  5867 rpcz_store.cc:276] 0709 12:27:52.980580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.023626 (+ 43046us) service_pool.cc:224] Handling call
0709 12:27:54.199001 (+1175375us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540083f254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:54.199198 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2321084800}
I20260709 12:27:54.201913 12542 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:54.202586  6757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1375) took 2639 ms (client timeout 1609 ms). Trace:
I20260709 12:27:54.202746 12969 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:54.204754 12767 consensus_meta.cc:287] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.619s	user 0.013s	sys 0.001s
I20260709 12:27:54.176602 12989 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:54.178750 12983 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:54.186187 12960 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 12:27:54.206108 12966 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:54.206388  6015 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 12:27:54.206856  5874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1630) took 1262 ms (client timeout 1599 ms). Trace:
I20260709 12:27:54.209898 12770 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:54.187870 12945 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:54.181216 12979 log.cc:927] Time spent T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:27:54.216188  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.216656  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:54.186928 12978 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 12:27:54.180403 12987 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:27:54.140950 12923 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:54.254298  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 63013.6us
I20260709 12:27:54.343169  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:54.256928 12993 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:27:54.272167  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 50170.1us
I20260709 12:27:54.286944  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:54.384684  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:54.271273  6611 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:54.288180  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:54.385778  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 114597us
I20260709 12:27:54.386440  5940 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 21: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:54.265606  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:54.233846  5865 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:54.278265  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:54.388690  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 110535us
I20260709 12:27:54.389017  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:54.272547  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:54.291900  6757 rpcz_store.cc:269] 0709 12:27:51.562734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.568977 (+  6243us) service_pool.cc:224] Handling call
0709 12:27:54.202333 (+2633356us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b5400007b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:54.202481 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2216279424}
W20260709 12:27:54.264846 12927 consensus_meta.cc:287] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.228s	user 0.012s	sys 0.002s
I20260709 12:27:54.308984  5990 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:54.393435  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:54.301162  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:54.317867  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:54.322505 12973 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.322878 12542 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:54.324033  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:54.324153 12770 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.324067  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:54.398021  5840 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:27:54.328001 12991 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:27:54.328946  5866 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:27:54.328723 12820 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:54.306740  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:54.409379  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 102714us
I20260709 12:27:54.410972  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:54.413560 12986 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.408524 12973 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:54.414194 12986 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:54.416108 12930 consensus_meta.cc:287] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.979s	user 0.002s	sys 0.016s
I20260709 12:27:54.416312 12986 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:54.404870 12927 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:54.417938  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.400991 12770 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:54.419348 12927 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:54.420184 12920 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:54.242656  6762 rpcz_store.cc:276] 0709 12:27:51.946164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.005564 (+ 59400us) service_pool.cc:224] Handling call
0709 12:27:52.030313 (+ 24749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:52.138155 (+107842us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:52.577842 (+439687us) log.cc:844] Serialized 14 byte log entry
0709 12:27:52.927403 (+349561us) log.cc:844] Serialized 14 byte log entry
0709 12:27:53.127088 (+199685us) log.cc:844] Serialized 14 byte log entry
0709 12:27:53.171186 (+ 44098us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:53.171251 (+    65us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:53.171297 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.233299 (+1062002us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400574ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:54.233360 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:54.233615 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32308,"prepare.queue_time_us":228492,"prepare.run_cpu_time_us":4964,"prepare.run_wall_time_us":43303,"raft.queue_time_us":325541,"raft.run_cpu_time_us":436,"raft.run_wall_time_us":19018,"spinlock_wait_cycles":253349504,"thread_start_us":953805,"threads_started":3,"wal-append.queue_time_us":486472}
I20260709 12:27:54.301265  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:54.257258  5942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1758) took 3075 ms (client timeout 3269 ms). Trace:
I20260709 12:27:54.365654  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:54.366638  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:54.427448  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:54.427721  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:54.371619  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:54.430738  5867 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:54.371933  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:54.432300  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:54.433058  5944 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:54.396242 12969 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:54.293510 12989 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.301185 12983 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:54.377815  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:54.249541 12874 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:54.428956  5942 rpcz_store.cc:269] 0709 12:27:51.181856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.238799 (+ 56943us) service_pool.cc:224] Handling call
0709 12:27:54.256992 (+3018193us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b54006a3bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:54.257186 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4288370048}
I20260709 12:27:54.451218  6020 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
W20260709 12:27:54.368280 12957 consensus_meta.cc:287] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.723s	user 0.007s	sys 0.006s
I20260709 12:27:54.300995 12613 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:54.256295 12783 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:54.447870 12920 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:54.383174  5987 connection.cc:256] RPC call timeout handler was delayed by -1.4937586299997747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
W20260709 12:27:54.393611  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 84735.1us
I20260709 12:27:54.317503  6023 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:27:54.362658 12943 consensus_queue.cc:1243] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:54.506767 12973 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:54.395720  5915 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:54.397662  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:54.398558  5871 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
W20260709 12:27:54.319052  5874 rpcz_store.cc:269] 0709 12:27:52.944214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.965877 (+ 21663us) service_pool.cc:224] Handling call
0709 12:27:54.206668 (+1240791us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400027454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:54.206790 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2309000576}
I20260709 12:27:54.398708  6793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:54.293354 12727 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:54.401175  5943 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 24: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:54.403779  5866 pending_rounds.cc:85] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 11
I20260709 12:27:54.396909  6611 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:54.566201  6793 raft_consensus.cc:606] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:27:54.566496  5866 pending_rounds.cc:107] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:27:54.557451 12934 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:54.566788  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 169933us
I20260709 12:27:54.567750 12934 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:54.568097  6611 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:54.407495  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:54.386736 12993 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:54.572589 12993 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:54.572921  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 165491us
I20260709 12:27:54.573035 12934 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:54.414565 12772 consensus_meta.cc:287] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.827s	user 0.014s	sys 0.000s
I20260709 12:27:54.260665 12918 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.411742  5940 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:54.420364  6785 raft_consensus.cc:2368] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21.
I20260709 12:27:54.419432  6017 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:27:54.404675 12780 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:54.256786 12708 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:54.429924  5865 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:54.430188  6022 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:27:54.379878  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:54.368793  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:54.442906  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:54.301332 12767 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:54.422432 12310 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:54.422607 12966 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:54.409461 12933 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:54.440300 12515 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.673s	user 0.013s	sys 0.000s
I20260709 12:27:54.319327 12909 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.437436 12760 raft_consensus.cc:3055] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:54.396999 12755 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:27:54.521736  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:54.580303 12980 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.441557 12892 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:54.450649 12983 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:54.466595 12947 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.866s	user 0.002s	sys 0.012s
I20260709 12:27:54.502424  5941 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 23: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:54.416636 12930 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:54.462832 12945 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:54.470137 12953 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.819s	user 0.013s	sys 0.001s
I20260709 12:27:54.464882 12820 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:54.422312 12824 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:27:54.454774 12989 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:54.548902  5867 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:27:54.553900  5987 leader_election.cc:400] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18.
I20260709 12:27:54.504136 12957 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:54.492851 12613 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:54.310511 12315 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:54.568141  5949 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: []
W20260709 12:27:54.565488  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 169825us
I20260709 12:27:54.571614 12727 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:54.566828  5866 raft_consensus.cc:2889] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:54.580258 12967 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:54.580711 12772 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:54.578398 12943 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:54.595667  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:54.595646  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:54.607352  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:54.609941  6023 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:27:54.551795 12956 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:54.619333  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:54.579054 12893 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:54.581549 12981 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:27:54.633885  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.637282  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:54.467474  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:54.642191  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:54.594959 12780 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.643452  6778 raft_consensus.cc:1240] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:27:54.595013 12708 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:54.644090  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:54.777371  6757 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:54.648172  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:54.578048 12686 consensus_queue.cc:1059] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:27:54.650955  5942 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 24: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:27:54.650480  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 229121us
I20260709 12:27:54.600558 12966 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.633204 12760 raft_consensus.cc:740] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:54.698424  5987 leader_election.cc:403] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:54.695657  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:54.715852  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:54.716076  6775 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 20: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:54.598156 12918 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:54.752214  6766 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:54.635046 12515 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:54.632376 12783 consensus_queue.cc:1059] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:27:54.651329 12892 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:54.651536 12947 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:54.775460  5870 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:54.777904  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:54.778366  5940 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
W20260709 12:27:54.784564  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.649008 12909 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:54.769768 12503 consensus_meta.cc:287] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.531s	user 0.013s	sys 0.001s
I20260709 12:27:54.686093 12953 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:54.820777  6756 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 21: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:54.720976 12315 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:54.870697  5945 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:54.820669 12966 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:54.824272 12542 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.652263 12980 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:54.849977  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 134220us
I20260709 12:27:54.780259 12780 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:54.856520  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 302783us
I20260709 12:27:54.883353  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:54.772629 12981 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:54.808377 12874 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:54.856968 12820 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18.
I20260709 12:27:54.870944  6024 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:54.849021  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:54.894980  6023 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:54.845085 12893 raft_consensus.cc:697] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:54.848069  6757 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:54.900125  6757 raft_consensus.cc:606] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:27:54.891932  6021 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:54.715085 12920 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.8, Last appended by leader: 8, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:54.897050  6016 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:54.916234  5945 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:54.922904  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:54.895783  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:54.906586 12647 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:54.906956  6756 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:54.896895 12837 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 12:27:54.943866 12647 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.944330 12647 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:54.910315 12934 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:54.945996 12934 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:54.946200  6756 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 12:27:54.946554 12934 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:54.947901  6756 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:54.947667 12934 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:54.949241 12934 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:54.942441  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:54.951611  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:54.952643 12951 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:27:54.953783  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 58099.2us
I20260709 12:27:54.873248 13018 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:54.892778 12981 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:54.963269  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.973798  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:54.974968  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:54.975744  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:54.978297  6024 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:54.958215 12887 consensus_meta.cc:287] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.194s	user 0.012s	sys 0.003s
W20260709 12:27:54.987589  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 88790.8us
I20260709 12:27:54.988960  5869 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:54.896889 12310 consensus_queue.cc:1059] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 12:27:55.001606  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:55.005345  5867 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:55.004226  6023 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:27:54.977022 12951 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:55.010720  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:55.014634 12515 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:54.893447 12760 consensus_queue.cc:260] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:54.974020 12767 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:55.015372 12953 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:55.016896 12770 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.598s	user 0.009s	sys 0.004s
I20260709 12:27:54.890563 12542 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:54.996362 13004 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.347s	user 0.002s	sys 0.000s
I20260709 12:27:54.990396 12354 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.990795 13018 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:54.899377 12708 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:55.050854  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:55.022715 12951 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:55.042264 12874 consensus_queue.cc:1059] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.146s
W20260709 12:27:55.184327  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 213989us
I20260709 12:27:55.152170  5867 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:55.106348  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1821) took 1098 ms. Trace:
I20260709 12:27:55.152397  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:55.149206  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1819) took 1142 ms. Trace:
I20260709 12:27:55.200590  5913 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:55.154263  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:55.128463  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 77710.9us
I20260709 12:27:55.199682  6773 rpcz_store.cc:276] 0709 12:27:54.007491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.010018 (+  2527us) service_pool.cc:224] Handling call
0709 12:27:54.010532 (+   514us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:54.052690 (+ 42158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:54.052738 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:54.251840 (+199102us) spinlock_profiling.cc:243] Waited 4.77 ms on lock 0x7b2c0001ffa8. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:27:54.252489 (+   649us) log.cc:844] Serialized 52 byte log entry
0709 12:27:54.441178 (+188689us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:54.441225 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.441297 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:55.105861 (+664564us) raft_consensus.cc:1685] finished
0709 12:27:55.105897 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:55.106254 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41356,"raft.queue_time_us":56383,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":20704,"spinlock_wait_cycles":10978560,"thread_start_us":187418,"threads_started":1,"wal-append.queue_time_us":322998}
I20260709 12:27:55.202003  6775 rpcz_store.cc:276] 0709 12:27:54.006750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.017831 (+ 11081us) service_pool.cc:224] Handling call
0709 12:27:55.148866 (+1131035us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b540052a154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:55.149076 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1466614528}
W20260709 12:27:55.142376 13016 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
W20260709 12:27:55.143644 12993 consensus_meta.cc:287] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.570s	user 0.012s	sys 0.002s
I20260709 12:27:55.056416 12837 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:55.047384 12770 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:55.128043 13009 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.359s	user 0.001s	sys 0.001s
W20260709 12:27:55.092886 13010 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 12:27:55.090802 12613 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:55.135000 12315 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:55.102736 12909 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:55.122364 13012 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
W20260709 12:27:55.195873 12988 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:27:55.261202  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1344) took 2511 ms. Trace:
I20260709 12:27:55.257561  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:55.258808 12874 consensus_queue.cc:1243] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:55.081529 12515 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:55.206033 12962 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.000s	sys 0.037s
I20260709 12:27:55.244701 12310 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.092070 12825 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:55.236135  6024 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:55.097348 12647 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:55.127389 12918 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:55.216688 12708 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:55.108384 13027 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:55.249711 12767 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:55.305275 12767 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:55.307348 12896 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.053s	user 0.008s	sys 0.004s
I20260709 12:27:55.308137 12767 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:55.260058 12978 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.876s	user 0.003s	sys 0.006s
I20260709 12:27:55.301326 12310 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:55.311407 12310 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:55.285254  5870 raft_consensus.cc:2368] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20.
I20260709 12:27:55.057571 12354 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:55.265203  5872 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:27:55.286609  5948 rpcz_store.cc:276] 0709 12:27:52.750129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.755064 (+  4935us) service_pool.cc:224] Handling call
0709 12:27:54.257349 (+1502285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.257838 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:54.257867 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:54.257928 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:54.257962 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.258038 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:55.260989 (+1002951us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54006a3bd0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:55.261132 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":180,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"spinlock_wait_cycles":3453867904}
I20260709 12:27:55.199059 12772 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.126061 12930 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:55.299798 12962 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.019s	sys 0.066s
I20260709 12:27:55.301695 12515 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:55.319494 13028 log.cc:927] Time spent T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:27:55.321775  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:55.336313 13020 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 12:27:55.284981 12315 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:27:55.339341  6793 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.021s	sys 0.043s
W20260709 12:27:55.368362  6793 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.021s	sys 0.043s
W20260709 12:27:55.368499  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 167995us
I20260709 12:27:55.205021 12893 consensus_queue.cc:237] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:55.257536 12993 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:55.381250  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 177444us
I20260709 12:27:55.385149 12315 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.386205 12315 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:55.369554 12947 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:55.353551 12772 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:55.375993 12980 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:55.402894  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.358331 12966 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:55.375474 12949 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.299978 13018 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:55.412161  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1464) took 1068 ms. Trace:
I20260709 12:27:55.415114  5868 rpcz_store.cc:276] 0709 12:27:54.343533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.598038 (+254505us) service_pool.cc:224] Handling call
0709 12:27:54.598554 (+   516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:54.599656 (+  1102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:54.599688 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:54.618586 (+ 18898us) log.cc:844] Serialized 26 byte log entry
0709 12:27:54.852994 (+234408us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:54.853049 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.853132 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:55.411842 (+558710us) raft_consensus.cc:1685] finished
0709 12:27:55.411872 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:55.412070 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":840,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":16068,"spinlock_wait_cycles":28032,"thread_start_us":173006,"threads_started":1,"wal-append.queue_time_us":327835}
I20260709 12:27:55.389432 12315 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:55.428715  5868 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:27:55.432204  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1828) took 1419 ms. Trace:
W20260709 12:27:55.421674 12914 consensus_meta.cc:287] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.214s	user 0.014s	sys 0.001s
W20260709 12:27:55.427045 12780 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.007s
I20260709 12:27:55.445916 12780 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:55.448434 12780 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:55.449890 12966 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:55.450937 12354 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:55.452682 12888 raft_consensus.cc:3055] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:55.453231 12888 raft_consensus.cc:740] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:55.454742 12647 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:55.433318  6793 rpcz_store.cc:276] 0709 12:27:54.012978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.569088 (+556110us) service_pool.cc:224] Handling call
0709 12:27:54.569425 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:54.583605 (+ 14180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:54.583655 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:54.584282 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 12:27:55.368920 (+784638us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:55.368971 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.369070 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:55.431883 (+ 62813us) raft_consensus.cc:1685] finished
0709 12:27:55.431907 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:55.432123 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13615,"raft.queue_time_us":14144,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"thread_start_us":784193,"threads_started":1,"wal-append.queue_time_us":842236}
I20260709 12:27:55.454777 12993 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:55.460309  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1823) took 1452 ms. Trace:
I20260709 12:27:55.462375  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 1356) took 1120 ms. Trace:
I20260709 12:27:55.462867  5945 rpcz_store.cc:276] 0709 12:27:54.342094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.365252 (+ 23158us) service_pool.cc:224] Handling call
0709 12:27:55.462142 (+1096890us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b54003093d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:55.462268 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1015985920}
I20260709 12:27:55.442119 12770 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:55.444998 12954 consensus_meta.cc:287] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.193s	user 0.000s	sys 0.013s
I20260709 12:27:55.464246 12954 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:55.466032 12892 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:55.467713 12972 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:55.441164 12772 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:55.445220 13026 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260709 12:27:55.470330 12983 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.790s	user 0.012s	sys 0.002s
I20260709 12:27:55.462214  5840 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:55.460834  6766 rpcz_store.cc:276] 0709 12:27:54.008155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.011857 (+  3702us) service_pool.cc:224] Handling call
0709 12:27:54.021181 (+  9324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.825969 (+804788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.460015 (+634046us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b5400007154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:55.460237 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1680550784}
W20260709 12:27:55.444474 13024 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 12:27:55.454355 12888 consensus_queue.cc:260] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:55.471668 12915 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:55.473562 12915 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:55.474035 12915 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:55.429700 12515 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:55.479112 13030 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:27:55.472883 12888 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:55.482668 12923 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.112s	user 0.015s	sys 0.000s
I20260709 12:27:55.483397  5946 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:27:55.470916 12983 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:55.485901 13035 log.cc:927] Time spent T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: Append to log took a long time: real 0.054s	user 0.002s	sys 0.000s
I20260709 12:27:55.534314 12954 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:55.534580 12983 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:55.535578 12892 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.488096 12914 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:55.536257 12892 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:55.536765  5873 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:55.537855 12989 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.819s	user 0.011s	sys 0.003s
I20260709 12:27:55.538329 12989 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:55.538864 12892 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:55.539063 12914 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:55.411459 12949 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:55.540756 12989 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:55.542114 12708 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:55.424443 12837 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:55.544361 12837 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:55.550069  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:55.550961  6766 raft_consensus.cc:2376] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 16. Re-sending same reply.
I20260709 12:27:55.552084  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:55.552899  6773 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 24: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:55.553489  6793 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:27:55.555732  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:55.552546  6611 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:55.569357  6766 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:55.573873  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:55.566743  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:55.580171  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.582628  6757 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 21: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:55.568173 12950 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:55.586603 12712 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:55.606160  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:55.613343  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1632) took 1681 ms. Trace:
I20260709 12:27:55.614538  6019 rpcz_store.cc:276] 0709 12:27:53.931717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.184629 (+252912us) service_pool.cc:224] Handling call
0709 12:27:54.184920 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:54.189605 (+  4685us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:54.189662 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:55.356642 (+1166980us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b2c001ddd28. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:27:55.357340 (+   698us) log.cc:844] Serialized 26 byte log entry
0709 12:27:55.558954 (+201614us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:55.559007 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.559103 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:55.609065 (+ 49962us) raft_consensus.cc:1685] finished
0709 12:27:55.609090 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:55.613225 (+  4135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4252,"raft.queue_time_us":7078,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":6666,"spinlock_wait_cycles":3876096,"thread_start_us":179089,"threads_started":1,"wal-append.queue_time_us":184868}
I20260709 12:27:55.617754 12966 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:55.618613 12966 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:55.619678 12966 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:55.621803  5840 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:55.603471 12712 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:55.622663 12712 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:55.623342 12915 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:55.624094  5840 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:55.625200 12983 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:55.625784 12983 raft_consensus.cc:697] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:55.626067 12708 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:55.625555 12915 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:55.626731 12983 consensus_queue.cc:237] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:55.630683 13037 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.631398 13037 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:55.633966 13037 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:55.639253 12888 consensus_queue.cc:1059] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:55.643329 12951 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:55.644089  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2187) took 1312 ms. Trace:
W20260709 12:27:55.645529  6021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1705) took 1793 ms (client timeout 1619 ms). Trace:
I20260709 12:27:55.644976  5873 rpcz_store.cc:276] 0709 12:27:54.331361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.331526 (+   165us) service_pool.cc:224] Handling call
0709 12:27:54.331964 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:55.537451 (+1205487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.643685 (+106234us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400210a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:55.644009 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2770150784}
W20260709 12:27:55.646277  6021 rpcz_store.cc:269] 0709 12:27:53.852396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.852974 (+   578us) service_pool.cc:224] Handling call
0709 12:27:55.645140 (+1792166us) spinlock_profiling.cc:243] Waited 984 ms on lock 0x7b54006b8fd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:55.645402 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2262763264}
I20260709 12:27:55.645769  6018 raft_consensus.cc:1240] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:55.650908 12951 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:55.599632 12930 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:27:55.657769  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 1355) took 1363 ms. Trace:
I20260709 12:27:55.663168 12915 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:55.667500 13007 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.904s	user 0.000s	sys 0.005s
I20260709 12:27:55.668622 12712 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:55.673607 12555 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:55.678619 12824 raft_consensus.cc:3055] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:55.679180 12824 raft_consensus.cc:740] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:55.680326 12824 consensus_queue.cc:260] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:55.658437  5946 rpcz_store.cc:276] 0709 12:27:54.294469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.294609 (+   140us) service_pool.cc:224] Handling call
0709 12:27:55.657461 (+1362852us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54008dfed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:55.657668 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2495562496}
I20260709 12:27:55.599964 12909 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 12:27:55.685971  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1617) took 1083 ms. Trace:
I20260709 12:27:55.686896 12909 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:55.687335 12909 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:55.688108  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1456) took 1457 ms. Trace:
I20260709 12:27:55.688431 12909 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:55.691004 13037 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:27:55.691682 12981 consensus_meta.cc:287] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.703s	user 0.011s	sys 0.001s
I20260709 12:27:55.691910 13037 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:55.692219 12981 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:55.694084  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2192) took 1361 ms. Trace:
I20260709 12:27:55.688539  5866 rpcz_store.cc:276] 0709 12:27:54.230528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.241793 (+ 11265us) service_pool.cc:224] Handling call
0709 12:27:54.242200 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.760161 (+517961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.687796 (+927635us) spinlock_profiling.cc:243] Waited 86.3 ms on lock 0x7b5400118ad4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:55.688046 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198464384}
I20260709 12:27:55.697916  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:55.698282  5945 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:55.698693  5948 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:55.699079  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:55.699906  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:55.700174  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:55.694500  5865 rpcz_store.cc:276] 0709 12:27:54.332698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.598153 (+265455us) service_pool.cc:224] Handling call
0709 12:27:54.598621 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.598920 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:54.648260 (+ 49340us) log.cc:844] Serialized 14 byte log entry
0709 12:27:55.660977 (+1012717us) spinlock_profiling.cc:243] Waited 8.07 ms on lock 0x7b1800297460. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39d282b4bc 00007f39cfb7948b 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287
0709 12:27:55.661147 (+   170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:55.661227 (+    80us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:55.661299 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.661426 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:55.661619 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19955,"prepare.queue_time_us":34671,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":3025,"raft.queue_time_us":88147,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":6999,"spinlock_wait_cycles":18576384,"thread_start_us":212669,"threads_started":2,"wal-append.queue_time_us":204042}
I20260709 12:27:55.706277  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:55.707276  6024 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:55.686446  6017 rpcz_store.cc:276] 0709 12:27:54.602672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.603089 (+   417us) service_pool.cc:224] Handling call
0709 12:27:54.638938 (+ 35849us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.639200 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:54.833495 (+194295us) log.cc:844] Serialized 14 byte log entry
0709 12:27:55.443777 (+610282us) spinlock_profiling.cc:243] Waited 4.36 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:55.685174 (+241397us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b18000eeea0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb10c0a 00007f39d9250219 00007f39cfb98a40 00007f39cfb934e4 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287 00007f39d2c9e208
0709 12:27:55.685507 (+   333us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:55.685577 (+    70us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:55.685659 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.685740 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:55.685895 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1967,"prepare.queue_time_us":252461,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":14204,"spinlock_wait_cycles":47766272,"thread_start_us":299117,"threads_started":2,"wal-append.queue_time_us":189382}
I20260709 12:27:55.713548  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:55.714896  5946 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:27:55.716454  5948 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:27:55.677258 12542 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:55.716569  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:55.717350  6773 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:27:55.716744  6757 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:55.720490  6757 raft_consensus.cc:606] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:55.720885  6757 raft_consensus.cc:614] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:55.721839  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:55.721910  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:55.724030  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:55.725234  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:55.728443  6021 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:55.729072  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:55.730118  6017 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:55.720933  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:55.716866 12909 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:55.725981  6778 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:55.755676  5946 raft_consensus.cc:1240] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:55.731710  5948 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:27:55.708246 12874 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.705518 12712 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:55.764876  6773 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
I20260709 12:27:55.734186  6022 raft_consensus.cc:2393] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 21: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:55.761499  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:55.770272  5943 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:55.770417  6756 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:55.772490  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:55.726840 12959 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:55.731539 12820 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.731770 12824 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:55.823882  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:27:55.831391  6785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43760 (request call id 1541) took 1818 ms (client timeout 1558 ms). Trace:
W20260709 12:27:55.831919  6785 rpcz_store.cc:269] 0709 12:27:54.013358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.073140 (+ 59782us) service_pool.cc:224] Handling call
0709 12:27:55.831127 (+1757987us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b54002a17d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:55.831308 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":738624896}
I20260709 12:27:55.833823 12943 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:55.731510 12914 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.087s
W20260709 12:27:55.816815 13032 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:27:55.816226 12542 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:55.836592  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:55.883234  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:55.903225  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 1.07817e+06us
I20260709 12:27:55.904939  5940 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:55.874095  6017 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
W20260709 12:27:55.908054  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:55.908762  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:55.911240  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 74720.4us
I20260709 12:27:55.794775 12874 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:55.913154  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:55.859122 12943 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:55.945125 12943 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:55.946233 12943 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:55.947561 12943 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:55.827278 12950 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.104s
I20260709 12:27:55.950039 12892 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:55.950651 12892 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:55.950820 12760 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:55.951980 12892 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:55.955156  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1611) took 2417 ms. Trace:
I20260709 12:27:55.955730  6018 rpcz_store.cc:276] 0709 12:27:53.538410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.109185 (+570775us) service_pool.cc:224] Handling call
0709 12:27:55.645380 (+1536195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:55.648265 (+  2885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.954860 (+306595us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54006b8fd0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:55.955075 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3473949952}
I20260709 12:27:55.921773  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:55.845510 12647 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:55.958715  6022 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:27:55.958863 12647 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:55.903795 12542 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:55.960552 12542 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:55.961239 12613 raft_consensus.cc:993] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: : Instructing follower 6add331326ca47038308a447bf798fa7 to start an election
I20260709 12:27:55.856233 12934 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:55.961830 12613 raft_consensus.cc:1081] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Signalling peer 6add331326ca47038308a447bf798fa7 to start an election
I20260709 12:27:55.962231 12934 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:55.918257  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:55.820387 12959 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:55.964812  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 46644.6us
I20260709 12:27:55.902513 12953 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.965504  6609 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:55.966075 12953 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:55.967849 12772 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:55.968556 12772 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:55.936614  5945 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:27:55.971294  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:55.973008  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:55.976832  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1642) took 1003 ms. Trace:
I20260709 12:27:55.981215  6609 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:55.982738  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1714) took 1707 ms. Trace:
W20260709 12:27:55.982877  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 69823.6us
I20260709 12:27:55.836055 12820 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:55.918560 12555 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:55.914829 12993 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:55.943835 13022 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:55.936508  6785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:55.983556  6020 rpcz_store.cc:276] 0709 12:27:54.275014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.424017 (+149003us) service_pool.cc:224] Handling call
0709 12:27:54.424446 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.451759 (+ 27313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.982422 (+1530663us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b5400641754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:55.982645 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48303488}
I20260709 12:27:55.918887 12951 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:55.993853  6024 raft_consensus.cc:2376] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23. Re-sending same reply.
I20260709 12:27:55.989157  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:55.892935 12958 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.002637  6785 raft_consensus.cc:606] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Received request to transfer leadership to TS 6add331326ca47038308a447bf798fa7
I20260709 12:27:55.996413 12892 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:55.977469  5869 rpcz_store.cc:276] 0709 12:27:54.973731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.975504 (+  1773us) service_pool.cc:224] Handling call
0709 12:27:55.976554 (+1001050us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b54003890d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:55.976745 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25119104}
I20260709 12:27:56.003304  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1623) took 1387 ms. Trace:
I20260709 12:27:56.000064 12613 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:56.005125 12930 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:56.004731  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1720) took 1082 ms. Trace:
I20260709 12:27:56.009665  6018 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:56.008124  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:56.031594  6024 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 19: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:56.009518  6021 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:56.001382  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:55.982515 12981 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:56.009579 13037 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:56.018286  6015 rpcz_store.cc:276] 0709 12:27:54.615603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.810548 (+194945us) service_pool.cc:224] Handling call
0709 12:27:54.822300 (+ 11752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.822578 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:55.043300 (+220722us) log.cc:844] Serialized 14 byte log entry
0709 12:27:55.702283 (+658983us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.002929 (+300646us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:56.002987 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:56.003036 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.003106 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.003242 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":501947,"prepare.queue_time_us":160288,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":125,"raft.queue_time_us":534166,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":26849,"spinlock_wait_cycles":49778688,"thread_start_us":677013,"threads_started":3,"wal-append.queue_time_us":336449}
I20260709 12:27:56.011615  6785 raft_consensus.cc:614] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:55.997900 13042 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:27:56.009209 13022 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:56.013291  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:56.005811 12934 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:56.003201 12719 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:56.001395  6756 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:56.028651  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:56.013459 12951 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:56.029865 12772 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:56.004521 12993 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:56.035210  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:56.047866  6024 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:56.033023  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:56.022297 12613 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:56.015681 12989 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.036743  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:56.068275  6015 raft_consensus.cc:2393] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 21: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:56.070250  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:56.072278  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:56.062842  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:56.078197  6020 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:27:56.079787  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:56.080554  6020 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:56.019027 12958 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:56.063783 12934 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:56.063920  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 50770.1us
I20260709 12:27:56.083518  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:56.083853 12958 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:56.060824 12613 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:56.086553 13052 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:56.087308 13052 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:27:56.088946  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.090483  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1766) took 1726 ms. Trace:
I20260709 12:27:56.090986  5941 rpcz_store.cc:276] 0709 12:27:54.364592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.364778 (+   186us) service_pool.cc:224] Handling call
0709 12:27:56.090255 (+1725477us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b5400530f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:56.090405 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257995008}
I20260709 12:27:56.091284  5987 leader_election.cc:400] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20.
I20260709 12:27:56.091851  5987 leader_election.cc:403] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:27:56.092316  6016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1612) took 2553 ms (client timeout 2596 ms). Trace:
W20260709 12:27:56.092767  6016 rpcz_store.cc:269] 0709 12:27:53.538793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.548202 (+  9409us) service_pool.cc:224] Handling call
0709 12:27:56.092100 (+2543898us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400160654. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:56.092231 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3043641600}
I20260709 12:27:56.093492  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:56.030637 12930 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:56.053936 12719 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:56.048693 12874 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:56.065608 12951 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:56.073520 12916 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:56.074383 12712 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:56.047623 12950 raft_consensus.cc:2764] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:27:56.065616 12989 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:56.088660 13052 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:56.090370  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:56.093559  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:56.050107  6023 rpcz_store.cc:276] 0709 12:27:54.921934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.008104 (+ 86170us) service_pool.cc:224] Handling call
0709 12:27:55.008599 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:55.008909 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:55.009233 (+   324us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.004365 (+995132us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:56.004418 (+    53us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:56.004463 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.004533 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.004664 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":653315,"prepare.queue_time_us":12186,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":673034,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":7068,"thread_start_us":310691,"threads_started":1}
I20260709 12:27:56.094201  5947 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:27:56.094218  5940 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:27:56.094524  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:56.094798  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:56.095077  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:56.095227  5948 raft_consensus.cc:1240] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:56.117878  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:56.163982  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:56.142184  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:56.108409  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:56.081594  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:56.093866  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:56.154755  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.124610  6022 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:56.119838  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:56.098955  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.145242  6024 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
W20260709 12:27:56.096223 13044 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:27:56.168946  6020 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:56.166368  5944 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:56.168097  6017 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:56.165901  6021 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:27:56.184445  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:56.095611 12949 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:56.202370 12951 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:56.178156  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:56.202963 12951 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:56.203452 12951 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:56.204278  5913 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:56.099053 12930 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:56.098726 13043 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
W20260709 12:27:56.098222 13045 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:27:56.170614 13055 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20.
W20260709 12:27:56.175108  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 81639.1us
I20260709 12:27:56.183099 13018 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:56.220719  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:56.220922  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:56.222110  5945 raft_consensus.cc:2410] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 25 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:27:56.222486  5949 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:56.170982  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 80716.5us
I20260709 12:27:56.190529  6023 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:56.165773  6015 raft_consensus.cc:2368] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21.
I20260709 12:27:56.167021  6018 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:56.171222  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:56.100391 12887 raft_consensus.cc:3055] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:56.182435  6016 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:27:56.124985 12916 raft_consensus.cc:697] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:56.096145 12953 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:56.115020 12959 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:56.109604 12647 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:56.210309  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:56.100555 12993 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:56.211440  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:56.212297  6773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570"
dest_uuid: "1fc57407d56241b38a94d543e073f8af"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:55542
I20260709 12:27:56.226845  6022 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:27:56.243000  6022 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:56.213312 12874 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:56.233718  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:56.109033 13022 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:56.240557  6020 raft_consensus.cc:2393] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 21: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:56.209584 12949 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:56.252031  6023 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 12:27:56.255831  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:56.224483 13018 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:56.228200 12887 raft_consensus.cc:740] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:56.265962  5945 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:56.269850  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:56.272893  6023 pending_rounds.cc:85] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 12
I20260709 12:27:56.275787  6023 pending_rounds.cc:107] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:27:56.276176  6023 raft_consensus.cc:2889] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:56.273378  6022 raft_consensus.cc:1240] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:27:56.299916  6022 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:56.316730  5949 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:56.337306  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:56.262599 12874 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:56.338950  5942 raft_consensus.cc:2410] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:27:56.346156  6023 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:56.336524 12916 consensus_queue.cc:237] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:56.341457 12959 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:56.364176 12959 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:56.366974 12959 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:56.369274  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.274835 13018 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:56.379209  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 388528us
I20260709 12:27:56.383176  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:56.385354  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:56.368242 12953 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.371228 12874 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:56.342664 12647 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:56.376410 12989 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:56.394397  5945 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:27:56.377007 12820 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:56.392795  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:56.392201 12959 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:56.405175  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 1550) took 2286 ms. Trace:
W20260709 12:27:56.407524  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 426380us
I20260709 12:27:56.409617  6762 rpcz_store.cc:276] 0709 12:27:54.118388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.873840 (+755452us) service_pool.cc:224] Handling call
0709 12:27:54.897899 (+ 24059us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.898112 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:54.962579 (+ 64467us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.197005 (+1234426us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b18000750a0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:27:56.404450 (+207445us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:56.404531 (+    81us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:56.404613 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.404904 (+   291us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.405094 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":62264,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":549,"spinlock_wait_cycles":4391168,"thread_start_us":207097,"threads_started":1}
I20260709 12:27:56.412431  5949 raft_consensus.cc:2410] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:27:56.403514  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:56.400465 12824 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.549s	user 0.014s	sys 0.000s
I20260709 12:27:56.418396  6773 raft_consensus.cc:606] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:56.419041  6773 raft_consensus.cc:614] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:56.421072  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:56.409250 12503 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.042s	sys 0.047s
I20260709 12:27:56.394832 12953 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:56.432616 12953 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:56.438850  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:56.440605  5949 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 17: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:56.439193  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:56.442572  5945 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:56.382323 12887 consensus_queue.cc:260] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:56.447521  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:56.423229 12775 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.007s	sys 0.011s
I20260709 12:27:56.399369 12647 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:56.434674 12503 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.042s	sys 0.048s
I20260709 12:27:56.499493  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:56.454926 12775 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.007s	sys 0.011s
I20260709 12:27:56.541901  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:56.422705 12613 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:56.511514 13049 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 12:27:56.509882 13011 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 1.741s	user 0.008s	sys 0.007s
I20260709 12:27:56.438982 12892 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 12:27:56.510138 13053 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.575335  5949 raft_consensus.cc:2410] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:27:56.571058  5943 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:56.588268  5944 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
W20260709 12:27:56.583654  5990 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.696s (SENT) [suppressed 29 similar messages]
I20260709 12:27:56.569882  6017 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:56.638502  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:56.616065  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:56.642344  5949 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:56.646422  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:56.609526  5945 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:56.649770  6017 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 22. Current term is 23. Ops: [21.13-22.14]
I20260709 12:27:56.650115  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:56.650920  5945 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 23.
W20260709 12:27:56.652019  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68408.5us
I20260709 12:27:56.651830  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:56.654225  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:56.654379  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:56.655807  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:27:56.582911 13059 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.134s	user 0.000s	sys 0.007s
I20260709 12:27:56.658715  5945 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:56.609884 13058 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.218s	user 0.001s	sys 0.001s
W20260709 12:27:56.662226  6609 connection.cc:256] RPC call timeout handler was delayed by -1.9774414580001576s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
I20260709 12:27:56.579695 12887 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:56.681180  6762 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:56.681257  6609 connection.cc:256] RPC call timeout handler was delayed by -1.8676414580004348s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2049s for a response
I20260709 12:27:56.663564  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:56.591142 13046 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.007s	sys 0.023s
W20260709 12:27:56.610160 12542 consensus_meta.cc:287] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.649s	user 0.013s	sys 0.005s
W20260709 12:27:56.714051  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:56.714592  6609 connection.cc:256] RPC call timeout handler was delayed by -1.8136414580003475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2109s for a response
I20260709 12:27:56.721742  5943 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:27:56.721237  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:56.724292  6757 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 26: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:56.726986  5945 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
W20260709 12:27:56.694792 13046 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.008s	sys 0.023s
I20260709 12:27:56.730494  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:56.731159  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:56.736155  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:56.736480  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:56.741108  5943 raft_consensus.cc:2368] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21.
I20260709 12:27:56.742190  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:56.743355  5945 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:56.747565  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:56.737954  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:56.758687  6762 raft_consensus.cc:2410] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:27:56.763134  5943 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:56.748889  5940 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 26.
I20260709 12:27:56.772854  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:56.713114 12315 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:56.591750 13053 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:56.768468  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:56.787947  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:56.788960  6609 connection.cc:256] RPC call timeout handler was delayed by -1.5304364180001357s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
W20260709 12:27:56.789387  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:56.789569  6609 connection.cc:256] RPC call timeout handler was delayed by -1.5104695310001262s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
W20260709 12:27:56.789924  6609 connection.cc:256] RPC call timeout handler was delayed by -1.2682414580003751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2715s for a response
I20260709 12:27:56.790158  5943 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 8. Preceding OpId from leader: term: 21 index: 9. (index mismatch)
I20260709 12:27:56.701761 12949 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 12:27:56.792217  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:56.799932  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:56.800706  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:56.806452  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:56.796888  6017 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:56.796862  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1412) took 1082 ms. Trace:
I20260709 12:27:56.776851 12981 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.810349  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:56.810299  6017 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:56.720263 13054 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:27:56.814903  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:56.807938  6793 rpcz_store.cc:276] 0709 12:27:55.714899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.715062 (+   163us) service_pool.cc:224] Handling call
0709 12:27:55.715450 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:55.715922 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:55.715953 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:55.735783 (+ 19830us) log.cc:844] Serialized 26 byte log entry
0709 12:27:56.229411 (+493628us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:56.229479 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.229586 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:56.796542 (+566956us) raft_consensus.cc:1685] finished
0709 12:27:56.796579 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.796771 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":332,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":9617,"thread_start_us":399681,"threads_started":1,"wal-append.queue_time_us":634081}
I20260709 12:27:56.817873  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:56.822589  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:56.825657  6793 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:56.851873  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:56.853119  6778 raft_consensus.cc:2410] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 27 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:27:56.857048  6017 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:56.860474  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:56.807772 12916 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:56.703330 12542 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:56.847632 12920 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:27:56.886224  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:56.892020  6778 raft_consensus.cc:2410] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:27:56.892622  6757 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:56.893885  6762 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:56.793246 12315 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:56.851495 12923 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:56.918560 13073 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
W20260709 12:27:56.922505 13068 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:27:56.924732  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:56.926108  6762 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 20: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:56.928269  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:56.889761 13062 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:56.846230 12981 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:56.932936  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:56.891763 13067 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:56.941475  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:56.945580  6757 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:56.937681  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:56.948597  6756 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 20: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:56.949436  6778 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 20: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:56.950601  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:56.899881 12613 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:56.965894  6785 raft_consensus.cc:2393] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 16: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:27:56.957873 13063 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.352s	user 0.000s	sys 0.002s
W20260709 12:27:56.973191  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 164981us
I20260709 12:27:57.013294  6757 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
W20260709 12:27:56.963650 13060 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.325s	user 0.000s	sys 0.002s
I20260709 12:27:57.014411  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:56.916018 12923 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:57.018740 12719 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:57.021121 12930 consensus_meta.cc:287] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.809s	user 0.009s	sys 0.002s
W20260709 12:27:57.014853  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 208505us
I20260709 12:27:57.022303  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:57.023248  6756 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 24: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:57.023253  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:57.011515 12958 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.030239  5990 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:57.033317  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:57.033733  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:57.034175  6756 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 23: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:57.034658  6757 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:57.035902  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:57.031901 13065 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.124s	user 0.004s	sys 0.010s
I20260709 12:27:57.031278 13067 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:27:57.055158 13081 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:27:57.055800 12981 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:57.056527 12951 consensus_meta.cc:287] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.853s	user 0.015s	sys 0.018s
I20260709 12:27:57.057360 12542 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:57.059065  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1414) took 1339 ms. Trace:
I20260709 12:27:57.059602  6766 rpcz_store.cc:276] 0709 12:27:55.719707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.758089 (+ 38382us) service_pool.cc:224] Handling call
0709 12:27:55.758649 (+   560us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:55.759152 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:55.759194 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:55.764462 (+  5268us) log.cc:844] Serialized 26 byte log entry
0709 12:27:56.219746 (+455284us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:56.219789 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.219862 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:57.058749 (+838887us) raft_consensus.cc:1685] finished
0709 12:27:57.058779 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.058979 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1546,"raft.queue_time_us":1032,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":30565,"thread_start_us":387585,"threads_started":1,"wal-append.queue_time_us":592713}
I20260709 12:27:57.061034  6766 raft_consensus.cc:3055] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:57.061333  6766 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:57.034658 12647 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:57.051280  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:57.053843  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:57.066128  6609 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:57.027819 12930 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:57.071907 12315 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:57.072213 13053 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:57.034747 13052 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:27:57.074968 12930 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:57.075489 13018 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.695s	user 0.013s	sys 0.000s
I20260709 12:27:57.075981 13018 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:57.077306  5941 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 24: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:57.083560  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:57.051729 12923 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:57.086897 13077 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 12:27:57.087479 12760 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:57.088022 12760 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:27:57.072759 12874 consensus_meta.cc:287] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.670s	user 0.006s	sys 0.004s
I20260709 12:27:57.057183 12951 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:57.064179  6766 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:57.103875  6766 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:57.044770 12958 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:57.065986 12981 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:27:57.112243  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 82067.8us
I20260709 12:27:57.112658  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1652) took 1085 ms. Trace:
W20260709 12:27:57.099766 12943 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.152s	user 0.012s	sys 0.006s
I20260709 12:27:57.114832  6609 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:57.118145  6793 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:57.120000  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:57.127663 13080 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:27:57.101202 12930 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:57.132884 12951 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:57.134975 13083 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:27:57.137521 12930 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:57.137902 13018 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:57.138079  5987 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:57.099792 12874 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:57.140273 12958 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:57.096256 12760 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:57.140918 12874 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:57.141604 12760 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:57.143497 12959 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.728s	user 0.009s	sys 0.005s
I20260709 12:27:57.144207 12959 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:57.138113 12930 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:57.138144 12934 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:57.150823 12934 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.151350 12934 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:57.153882  6023 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75"
dest_uuid: "6add331326ca47038308a447bf798fa7"
 from {username='slave'} at 127.0.0.1:34888
I20260709 12:27:57.154788  6023 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:57.155136  6023 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:57.121483 12647 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:57.157511 12647 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:57.143623  6019 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:57.159192  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1654) took 1104 ms. Trace:
I20260709 12:27:57.159633  6019 rpcz_store.cc:276] 0709 12:27:56.055363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.142853 (+1087490us) service_pool.cc:224] Handling call
0709 12:27:57.143221 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.158788 (+ 15567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.159108 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:57.164899  6762 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 25.
I20260709 12:27:57.167403  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:57.168327  6762 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 23.
I20260709 12:27:57.167975  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:57.172094  6775 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:57.173811  5946 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:27:57.176141  5990 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:57.175642 12647 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:57.176931  5946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2133) took 1084 ms. Trace:
I20260709 12:27:57.178292  5946 rpcz_store.cc:276] 0709 12:27:56.092716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.094573 (+  1857us) service_pool.cc:224] Handling call
0709 12:27:57.169707 (+1075134us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b5800030398. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:27:57.169940 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.174285 (+  4345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.176855 (+  2570us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53634944}
I20260709 12:27:57.179690 12780 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:57.181384  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:57.182734  6775 raft_consensus.cc:2393] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 19: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:57.187194  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34888 (request call id 1726) took 1481 ms. Trace:
I20260709 12:27:57.187743  6021 rpcz_store.cc:276] 0709 12:27:55.705600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.792947 (+ 87347us) service_pool.cc:224] Handling call
0709 12:27:57.186973 (+1394026us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b5400661cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:57.187105 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":417952128}
I20260709 12:27:57.189502  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:57.191227 12959 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:27:57.192579 13047 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 1.165s	user 0.002s	sys 0.002s
I20260709 12:27:57.194466  5990 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:57.196077 12820 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:57.196657 12820 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 25 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:57.197773 12820 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:57.202692  6022 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:57.203334  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1655) took 1026 ms. Trace:
I20260709 12:27:57.203532  6022 rpcz_store.cc:276] 0709 12:27:56.176614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.183016 (+1006402us) service_pool.cc:224] Handling call
0709 12:27:57.183374 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.203085 (+ 19711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.203281 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:57.204252  5942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1772) took 2004 ms (client timeout 1696 ms). Trace:
W20260709 12:27:57.204566  5942 rpcz_store.cc:269] 0709 12:27:55.200087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.200202 (+   115us) service_pool.cc:224] Handling call
0709 12:27:57.188243 (+1988041us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540055ffd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:57.204180 (+ 15937us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2417412864}
I20260709 12:27:57.206893  6023 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:57.209185  6019 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:57.211716  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:57.201414 12989 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:57.215418 12951 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:57.216342 12959 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:57.216706 12959 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:57.219089  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:57.222339  6023 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:57.213362 12989 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.226596  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:57.227514  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: true dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:57.227216  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:34888 (request call id 1732) took 1098 ms. Trace:
I20260709 12:27:57.232504  6773 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:57.232537  6023 rpcz_store.cc:276] 0709 12:27:56.127506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.153674 (+1026168us) service_pool.cc:224] Handling call
0709 12:27:57.223136 (+ 69462us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:57.233462  6773 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:57.234634  6773 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:57.235067 13053 raft_consensus.cc:3055] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:57.235695  6773 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:57.235146  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1649) took 2259 ms. Trace:
I20260709 12:27:57.237275  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1357) took 2866 ms. Trace:
I20260709 12:27:57.213693 12947 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:57.238087  5947 rpcz_store.cc:276] 0709 12:27:54.370478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.371197 (+   719us) service_pool.cc:224] Handling call
0709 12:27:56.093955 (+1722758us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:56.167030 (+ 73075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.231308 (+1064278us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b5400530f50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:57.237187 (+  5879us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3912122240}
I20260709 12:27:57.238523  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:57.242905  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:57.222133 12951 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:57.248755 12947 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:57.249008 12951 raft_consensus.cc:697] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:57.236629  5868 rpcz_store.cc:276] 0709 12:27:54.975591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.491177 (+515586us) service_pool.cc:224] Handling call
0709 12:27:55.549696 (+ 58519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:55.550116 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:55.550153 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:55.550691 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:27:56.000375 (+449684us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:56.022298 (+ 21923us) log.cc:844] Serialized 14 byte log entry
0709 12:27:57.234344 (+1212046us) spinlock_profiling.cc:243] Waited 977 us on lock 0x7b18000026e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb10c0a 00007f39d9250219 00007f39cfb98a40 00007f39cfb934e4 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287 00007f39d2c9e208
0709 12:27:57.234668 (+   324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.234765 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:57.234815 (+    50us) raft_consensus.cc:1685] finished
0709 12:27:57.234912 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.235079 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1177,"prepare.queue_time_us":21399,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":14843,"raft.run_cpu_time_us":402,"raft.run_wall_time_us":19919,"spinlock_wait_cycles":2246912,"thread_start_us":448218,"threads_started":1,"wal-append.queue_time_us":648375,"wal-append.run_cpu_time_us":2489,"wal-append.run_wall_time_us":1002493}
W20260709 12:27:57.244861  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 1.12192e+06us
I20260709 12:27:57.251778  5838 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:57.245080  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1630) took 1386 ms. Trace:
I20260709 12:27:57.246897 13090 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:57.248445 12887 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.545s	user 0.014s	sys 0.000s
I20260709 12:27:57.252492  6016 rpcz_store.cc:276] 0709 12:27:55.858553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.093589 (+235036us) service_pool.cc:224] Handling call
0709 12:27:57.244852 (+1151263us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b5400764b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:57.245013 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30718848}
W20260709 12:27:57.253150  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.236932 13052 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:57.254384  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 1.13647e+06us
I20260709 12:27:57.239162 12989 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:57.249588 12708 consensus_queue.cc:1059] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.009s
W20260709 12:27:57.264760  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.235671 13053 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:57.243104 13054 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:57.259842  5840 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:57.285986 12966 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.310s	user 0.017s	sys 0.001s
I20260709 12:27:57.254458 13090 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.281786  6773 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
I20260709 12:27:57.217810 12959 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:57.271737 12824 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:57.295666  5990 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:57.290431  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 47602.3us
W20260709 12:27:57.304113  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 52427.4us
W20260709 12:27:57.291203 12966 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.316s	user 0.018s	sys 0.002s
I20260709 12:27:57.344578 12959 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:57.289819 12930 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:27:57.324103  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.338960 12686 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:57.331606 12310 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:57.299585 12542 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:57.334626 12708 raft_consensus.cc:3055] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:57.355492 12708 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:57.355352 12542 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:57.356011 12930 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:57.343674 12780 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:57.338116  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 78406.4us
I20260709 12:27:57.250198 12951 consensus_queue.cc:237] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:57.300511 13022 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:57.353600  6775 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:57.294412 13090 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:57.303102 12943 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.311057 12958 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:57.304311 12824 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.310209 12947 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:57.332261 12954 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:57.361850 12708 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:57.365010 12686 raft_consensus.cc:697] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:57.375485  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:57.365182 13022 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:57.376508  6023 raft_consensus.cc:2393] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:57.360827 12310 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:57.398722  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:57.382653  6022 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:57.362586 13015 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:27:57.383895  6021 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 12:27:57.376595 12686 consensus_queue.cc:237] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:57.402786  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:57.376241 12708 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:57.419397  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:57.420863  6015 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:57.429668 13102 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:57.431408 13102 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.350967 13053 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:57.408147  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 44784.3us
W20260709 12:27:57.376438 12783 consensus_queue.cc:1175] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1fc57407d56241b38a94d543e073f8af" responder_term: 16 status { last_received { term: 16 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 16 index: 10 } }
I20260709 12:27:57.403393 12958 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:57.434892  6022 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:57.365718 12943 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:57.371973 12824 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:57.454099  5913 leader_election.cc:336] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.551s (SENT)
I20260709 12:27:57.421911  6021 pending_rounds.cc:85] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 9
I20260709 12:27:57.419404 12542 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:57.471306  6021 pending_rounds.cc:107] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:27:57.472774 12958 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:57.478725 12310 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:57.479271 12934 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:57.469784  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:57.483580  6021 raft_consensus.cc:2889] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:57.489028  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:57.490082  5869 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:57.490012  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:57.498085  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:57.498230  5868 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:57.501523  5872 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:57.506086  6022 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:57.508289  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:57.511353  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:57.512089  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.446590 13102 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:57.516677  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 80350.3us
I20260709 12:27:57.519086  5874 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:57.499837 12820 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.522082 12887 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:57.524396 12820 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:57.525271 12887 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:57.525964  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:57.526753  6021 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:27:57.527794 12887 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:57.527935  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:57.528606 13022 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:57.521245  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:57.496464 12542 consensus_queue.cc:1059] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:27:57.523511  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:57.468142 12947 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:57.531234  5838 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:57.533993  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:57.534773  6768 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:27:57.539101  5915 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:57.541020  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:57.542418 12760 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:57.542936 12760 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.543208 13102 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:57.543854 13102 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.544474  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:57.545209  5873 raft_consensus.cc:2393] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 21: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:57.545189  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:57.546559 12943 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:57.546754 12954 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:57.547187 12943 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:57.547302 12954 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:57.547715 12943 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:57.548656 12989 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:57.549587 12954 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:57.546206  5840 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:57.554384 12515 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:57.555953 12923 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:57.556457 12923 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:57.543294 12760 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:57.546586 13015 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.557493 12923 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:57.558390 13015 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:57.561338 13022 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:57.561545 12542 raft_consensus.cc:3055] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:57.562005 12542 raft_consensus.cc:740] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:57.562692  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1733) took 1396 ms. Trace:
I20260709 12:27:57.562943 12542 consensus_queue.cc:260] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:57.564507 12542 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:57.566148 12708 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.544301 13102 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:57.563120  6019 rpcz_store.cc:276] 0709 12:27:56.166787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.160977 (+994190us) service_pool.cc:224] Handling call
0709 12:27:57.161780 (+   803us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.210987 (+ 49207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.562442 (+351455us) spinlock_profiling.cc:243] Waited 47.2 ms on lock 0x7b540026fa54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:27:57.562631 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108488576}
I20260709 12:27:57.578415  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:57.571938  5838 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:57.551455 12989 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:57.551847  6021 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:27:57.546783  6768 raft_consensus.cc:2393] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 24: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:57.576081  5865 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:27:57.571321  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:57.586673  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:57.588376  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:57.581024  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:57.579684  6015 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:57.580452  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:57.594244  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:57.595575  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:57.593698  6023 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
W20260709 12:27:57.600284  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.496313 12686 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 12:27:57.608687  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:57.554505 12934 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:57.614418  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:57.609725  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: true dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:57.609066  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:57.551987 13052 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:57.614006  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:57.618458  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:57.619405  5870 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 26.
I20260709 12:27:57.613994  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:57.560720 13015 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:57.616703  5949 raft_consensus.cc:2410] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:27:57.579334 12943 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:57.581233 12887 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:57.549896 13018 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 12:27:57.633798 12954 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:27:57.635303  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 48759.8us
I20260709 12:27:57.636864  5840 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:57.631484 12887 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:57.637943 12943 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:57.638200 12887 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:57.638362 12943 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.638782 12943 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:57.639146 12686 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:57.640022 12686 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:57.640448 12954 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:57.644620 13075 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.722s	user 0.000s	sys 0.005s
I20260709 12:27:57.647039 13102 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:57.580546 12708 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:57.647330 12824 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:57.641558 12954 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:57.649861  5869 raft_consensus.cc:2376] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18. Re-sending same reply.
I20260709 12:27:57.649431 12824 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:57.651712 12820 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:57.630749  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 49826.8us
W20260709 12:27:57.655136  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 540338us
I20260709 12:27:57.653292  5840 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:57.656535  6609 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:57.649456 12708 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:57.657006  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:57.641109 12686 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:57.658871  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:57.659452  5872 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:57.659662 12981 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:57.666990  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:57.667303  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:57.620184 13103 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:57.673703  5945 raft_consensus.cc:2393] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 21: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:27:57.668517  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:57.676839  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:57.678210  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:57.687793  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:57.688781  5940 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:27:57.690689  5943 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:57.685132  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:57.696298  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:57.697237  5949 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:27:57.615258 13109 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:57.669278 12824 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:57.673033 13102 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.672199 13015 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:57.677112 13103 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.724088  5838 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:57.728852 13109 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:57.729583 13109 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:57.730166 13109 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:57.704388 12909 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 12:27:57.672588 12783 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
W20260709 12:27:57.738795  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 164860us
I20260709 12:27:57.742451  5838 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:57.743993  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:57.709951 12712 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.684891 12708 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:27:57.727113 13103 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:57.684275 13108 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:27:57.686631 13090 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:57.756402  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.777740 13018 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:27:57.779655  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 123239us
I20260709 12:27:57.720563 12824 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:27:57.760102  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:57.783706  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.804365  6609 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:57.702337 12760 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:57.809507  5987 leader_election.cc:336] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.592s (SENT)
I20260709 12:27:57.810212  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:57.796428  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:57.806871  5913 leader_election.cc:336] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.848s (SENT)
I20260709 12:27:57.807337  5943 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:57.807943  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:57.813791  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:57.815532  6766 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:57.775630 13022 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:57.721704 13102 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:57.815775  6609 leader_election.cc:400] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21.
I20260709 12:27:57.853780  6609 leader_election.cc:403] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
W20260709 12:27:57.819094  5987 leader_election.cc:336] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.599s (SENT)
I20260709 12:27:57.854748  5987 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:57.774777 12930 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:27:57.857910  5987 leader_election.cc:336] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.619s (SENT)
I20260709 12:27:57.859319  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:57.858799  5947 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:57.859517  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:57.862468  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 48947.1us
I20260709 12:27:57.781796 12712 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:57.850359 12934 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:57.866204  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 50549us
I20260709 12:27:57.865170  6611 leader_election.cc:400] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21.
I20260709 12:27:57.866871  5874 raft_consensus.cc:2410] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:57.865907  5866 raft_consensus.cc:2368] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 24. Current term is 25.
I20260709 12:27:57.893190  6766 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:57.893515  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:57.893853  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 122209us
I20260709 12:27:57.897360  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:57.898954  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.885591 12930 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21.
I20260709 12:27:57.858799 13022 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.902411  6609 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:57.855882 13123 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:27:57.907560  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.880561 13015 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:57.891353 12686 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:57.863289 12951 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:57.917344  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:57.886057 13097 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:57.904572 13062 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:57.931779  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 80436.1us
I20260709 12:27:57.907830 12950 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:57.911459 13022 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:57.936172  5866 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 24: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:57.865896 13018 consensus_queue.cc:1059] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:27:57.860251 13105 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:27:57.922820 13115 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:57.904937 13067 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
W20260709 12:27:57.951082  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.928478 12930 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:57.928728 13123 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:57.956290 13090 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:57.950477 13117 consensus_queue.cc:1059] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:27:57.936378 13062 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:57.957494  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:57.970461  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:57.977855  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 60633.9us
I20260709 12:27:57.978170  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:57.912119 12824 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:57.969373  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:57.990002  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:57.994383  6015 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:57.954192 13115 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:57.987937  5866 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
I20260709 12:27:57.995538  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:57.926654 13015 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:57.982002  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:57.990918  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:57.932480 13122 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:27:57.939484 12966 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:27:57.990036 13062 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:58.018790 13114 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:58.008523 13119 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
W20260709 12:27:58.013005 13124 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:27:58.021919 13123 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:58.025846 12760 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:57.996832  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:58.031605  6019 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 19: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:27:57.974591 12930 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:58.032204 12930 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:58.034256  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:58.009783  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:58.010406 12712 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:58.038964  6021 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:58.010870 13128 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.004s
W20260709 12:27:58.011968 13120 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:27:58.012938  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:58.013340 13113 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 12:27:58.017424 13015 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:57.990736 13067 consensus_queue.cc:1243] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 12:27:58.019030 13111 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:27:57.934507 12686 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:58.049949  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 143739us
W20260709 12:27:58.050468  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 116876us
W20260709 12:27:58.010910 13121 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 12:27:58.053660 13116 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.188s	user 0.001s	sys 0.001s
I20260709 12:27:57.998440 13125 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.101s
I20260709 12:27:57.995862 12950 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.053s
W20260709 12:27:58.169694 13129 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:27:58.092418 13102 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:58.060601 13090 consensus_queue.cc:1059] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.114s
I20260709 12:27:58.185004 12951 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:58.185250 12686 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:58.164785 13067 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.186197  6015 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:58.186595 13067 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:58.116374 12887 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:58.189688 13067 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:58.191901 12887 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:58.136807 12945 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
W20260709 12:27:57.991588  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 54733.2us
W20260709 12:27:58.173583  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 113150us
I20260709 12:27:58.004405  5868 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:27:58.199162  5869 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:27:58.176878  6019 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:27:58.199803  6015 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 25.
I20260709 12:27:58.162488  6611 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:27:58.209247  5941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1780) took 1864 ms (client timeout 2063 ms). Trace:
W20260709 12:27:58.209964  5941 rpcz_store.cc:269] 0709 12:27:56.344819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.345141 (+   322us) service_pool.cc:224] Handling call
0709 12:27:58.208955 (+1863814us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b5400459cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:58.209155 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1593959680}
I20260709 12:27:57.116088  6017 rpcz_store.cc:276] 0709 12:27:56.027687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.858572 (+830885us) service_pool.cc:224] Handling call
0709 12:27:56.859439 (+   867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.859756 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:56.892199 (+ 32443us) log.cc:844] Serialized 14 byte log entry
0709 12:27:57.029084 (+136885us) log.cc:844] Serialized 14 byte log entry
0709 12:27:57.111744 (+ 82660us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b18001f7b60. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb10c0a 00007f39d9250219 00007f39cfb98a40 00007f39cfb934e4 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287 00007f39d2c9e208
0709 12:27:57.112039 (+   295us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:57.112127 (+    88us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:57.112206 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.112315 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.112541 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18194,"prepare.queue_time_us":35480,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":7088,"raft.queue_time_us":169333,"spinlock_wait_cycles":14015232,"thread_start_us":135961,"threads_started":1,"wal-append.queue_time_us":197958}
I20260709 12:27:58.222169  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:58.058079 12943 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:58.222970  5946 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:58.226191  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 63799.6us
I20260709 12:27:58.228824  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:58.224941  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:58.229559  6017 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:58.234691  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:58.238507  5913 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:58.237241  5868 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:58.244271  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:58.246937 13018 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:58.218900 13132 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:27:58.258968  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:58.262360  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:58.283041  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:58.261754  5873 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:58.270394  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 270068us
I20260709 12:27:58.290596  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:58.233340 13067 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:58.219535 13109 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:58.298846  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:58.328329  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.341010  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:58.259619 12943 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:58.333122  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:58.259619 12760 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:27:58.261976 13052 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.262542 12951 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:58.366060  5868 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:58.372155  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:58.373381  6793 raft_consensus.cc:2393] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 23: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:58.245157 13125 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.099s
W20260709 12:27:58.388485  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:58.334797 13126 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:27:58.288900 13018 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.281375 13117 consensus_queue.cc:1243] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:27:58.392017  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:58.395048  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:58.408332  5868 raft_consensus.cc:2376] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17. Re-sending same reply.
I20260709 12:27:58.409025  5873 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 25.
I20260709 12:27:58.409881  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:58.410862  6773 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:58.395520  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:58.398316  6775 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:58.260860 13131 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.137s
I20260709 12:27:58.404748  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:58.418035  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:58.419272  6773 raft_consensus.cc:2410] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:27:58.420791  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:58.423296  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:58.386004 13067 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 25 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:58.346062 12712 consensus_queue.cc:1059] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:27:58.425297  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:58.435220  6793 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 20: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:58.425513  6762 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:58.429287  5913 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:58.432814  6778 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 25: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:27:58.434104  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.347749 13022 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:58.437167  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:58.439288  6775 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:58.407279 13018 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:58.437942  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:58.443598  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:58.443842  6773 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
W20260709 12:27:58.446408  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 223816us
I20260709 12:27:58.386842 13052 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:58.391815 12951 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:58.444629  5868 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:27:58.452405  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:58.459226  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:58.459743  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:58.461225  5840 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:27:58.469305  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 221736us
I20260709 12:27:58.470624  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:58.475117  6023 raft_consensus.cc:1240] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 12:27:58.475306  5915 connection.cc:256] RPC call timeout handler was delayed by -1.1542296170000554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15580000000000002s for a response
I20260709 12:27:58.475525  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:58.477350  6024 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 23. Current term is 24. Ops: [23.15-23.15]
W20260709 12:27:58.475764  5915 connection.cc:256] RPC call timeout handler was delayed by -1.1479296170000453s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
W20260709 12:27:58.478628  5915 connection.cc:256] RPC call timeout handler was delayed by -1.1218296170000031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
W20260709 12:27:58.479001  5915 connection.cc:256] RPC call timeout handler was delayed by -1.0291296169998532s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16970000000000002s for a response
W20260709 12:27:58.480170  5915 connection.cc:256] RPC call timeout handler was delayed by -1.0212412730002143s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
I20260709 12:27:58.445562 13128 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:58.453259 13132 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:58.488808  6019 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:27:58.489737  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:58.493491  5868 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:58.484333  5941 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c"
dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
mode: GRACEFUL
new_leader_uuid: "7a0204b931394fc48ffc503716e0d111"
 from {username='slave'} at 127.0.0.1:48330
I20260709 12:27:58.490808  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:58.495023  5941 raft_consensus.cc:606] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Received request to transfer leadership to TS 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:58.495103  5873 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:58.496608  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:58.500903  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:27:58.501927  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:58.511165  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:58.511662  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:58.516356  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:58.518836  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:58.521147  6609 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:58.505569  6611 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:27:58.591363  5940 pending_rounds.cc:85] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 12
I20260709 12:27:58.592020  5940 pending_rounds.cc:107] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:27:58.592347  5940 raft_consensus.cc:2889] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:58.592752  5940 pending_rounds.cc:107] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 23.14
I20260709 12:27:58.593012  5940 raft_consensus.cc:2889] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:58.498922 12542 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.934s	user 0.015s	sys 0.012s
I20260709 12:27:58.495793 12310 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:58.609067  5946 raft_consensus.cc:2468] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:27:58.610532  5948 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: [19.10-22.11]
I20260709 12:27:58.588593 13127 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:58.602370  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 86073.3us
I20260709 12:27:58.506815 13139 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.498108 12503 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:58.636274  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 144774us
I20260709 12:27:58.524170 13132 raft_consensus.cc:697] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:58.594400 13141 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:27:58.625607 13140 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:27:58.623222  5868 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
W20260709 12:27:58.633316 13062 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.614s	user 0.010s	sys 0.002s
I20260709 12:27:58.618273 13108 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:58.663050  6024 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:27:58.523923 13131 consensus_queue.cc:1059] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.125s
I20260709 12:27:58.669803  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:58.678150  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:58.679597  5868 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:58.643330 12310 raft_consensus.cc:697] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:58.652801 13141 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.690516  5867 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 28.
I20260709 12:27:58.693192  5868 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:27:58.594133 13067 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:58.664016 13108 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.663161 12686 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:58.698859  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 177813us
I20260709 12:27:58.707647  6609 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:27:58.704028  6024 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:27:58.670511 13109 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:58.648667 12503 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:58.714231  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:58.679590 13138 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:58.723193  5868 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 18.
I20260709 12:27:58.663503 13105 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:27:58.648963 13139 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:58.730336  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:58.698635 13141 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:58.706374 13108 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:58.735945  5868 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 26.
I20260709 12:27:58.702575 13152 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:58.667147 13062 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:58.760515  5913 leader_election.cc:336] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.594s (SENT)
I20260709 12:27:58.754627  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2212) took 1091 ms. Trace:
I20260709 12:27:58.762636  5869 rpcz_store.cc:276] 0709 12:27:57.663458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.673594 (+ 10136us) service_pool.cc:224] Handling call
0709 12:27:58.198839 (+525245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:58.199554 (+   715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.754353 (+554799us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b540069e950. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:58.754552 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1206772096}
I20260709 12:27:58.766763  6775 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 25.
I20260709 12:27:58.769208  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:58.770406  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 50210.9us
I20260709 12:27:58.769420  5913 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:58.784062  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:58.757465 13133 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:58.802902  6756 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:58.756073 13152 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:58.757306 12951 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:58.770370 12981 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.089s	user 0.011s	sys 0.005s
I20260709 12:27:58.799475 13117 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:58.754428 13022 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:27:58.804474  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 173048us
W20260709 12:27:58.816644 13154 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:27:58.753906 13136 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:58.813884 13133 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:27:58.823295  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 153787us
I20260709 12:27:58.805209 13148 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.726806 13138 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:58.753598 12310 consensus_queue.cc:237] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:58.828083  5915 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:58.811859 13152 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:58.842481 12503 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:58.827874  5840 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:58.820684 12981 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:58.757323 13132 consensus_queue.cc:237] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:58.825668 12930 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.791s	user 0.006s	sys 0.010s
I20260709 12:27:58.837250 13133 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:58.876700  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 48916.5us
W20260709 12:27:58.845468 13103 consensus_meta.cc:287] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.050s	user 0.017s	sys 0.000s
W20260709 12:27:58.879076  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 51142.1us
W20260709 12:27:58.836477 13155 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:27:58.838953 13139 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:58.857280 13117 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:58.891023  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:58.847708 13138 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:58.896993  6022 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:58.828315 13131 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:58.902547  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:58.880465 12943 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:58.880409 13128 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:58.886556 13147 raft_consensus.cc:3055] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:58.887120 12914 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.840083 13165 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 12:27:58.911430  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:27:58.915472  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.915771  6022 raft_consensus.cc:2410] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:27:58.836993 13156 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 12:27:58.895697 13136 consensus_queue.cc:1059] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 12:27:58.877907 12930 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:58.904461 13131 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:58.887728 13103 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:58.925887  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:58.865131 13148 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:58.909819 12943 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:27:58.952319  6766 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:58.952914  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:58.953994  6773 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:58.954134  6762 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:27:58.955402  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:58.957593  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:58.957746  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:58.912143 13147 raft_consensus.cc:740] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:58.960457  5840 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:58.965013  6766 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:58.969357  5915 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:58.968155  6762 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:27:58.974337  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:58.978550  6785 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:58.971863  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:58.978530  6768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:59.069815  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 95578.6us
I20260709 12:27:59.070724  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:59.175645  5871 raft_consensus.cc:2393] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 20: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:27:59.176781  6768 raft_consensus.cc:2393] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 20: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:27:59.182171  6773 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
W20260709 12:27:59.079602  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 47908.7us
I20260709 12:27:58.942559 13131 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:58.979328  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:59.206610  6756 raft_consensus.cc:2410] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:27:58.938175 13174 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:59.201550  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:59.249581  6778 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:58.957911 12943 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:58.920832 12914 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:59.204561  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 232804us
I20260709 12:27:59.178251  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:27:59.039880  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.982908 13128 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:59.267699  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:59.263607  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:27:59.271392  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 200732us
I20260709 12:27:58.971446 13022 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:59.278002  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:58.967962 13169 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:59.177292 13136 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:58.978195 13173 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:58.971333 13168 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:59.284019  6756 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:59.296192  6756 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:27:58.978642 12953 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.294s
I20260709 12:27:59.248888 12983 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:58.978796 12503 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.303s
I20260709 12:27:59.250372 13175 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.282596  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:59.274919 13067 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:59.286509 13018 consensus_meta.cc:287] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.829s	user 0.011s	sys 0.003s
I20260709 12:27:59.274163 13062 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:59.262398 13171 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:59.285938 13156 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:59.329778  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.318359  6756 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:59.333405  6756 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:59.328675  6778 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
W20260709 12:27:59.331583  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 207932us
W20260709 12:27:59.324190  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.319917  6609 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:59.344552  5946 raft_consensus.cc:3055] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:59.345239  5946 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:59.179028 13117 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:59.318938 13067 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:59.329610 13018 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:59.302204 13128 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:59.320220 13169 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.330327 13136 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.318843 13146 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:59.334038 13114 raft_consensus.cc:3055] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:59.360335  6609 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:59.364861  5946 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:59.338572 13173 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:59.379029  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.352934 13165 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:59.386015  5946 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:59.318601 13175 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:59.363870 13158 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:59.363240 13174 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:27:59.405040  6756 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:27:59.377202 13146 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:59.378239 13114 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:59.379844 12923 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:59.406067  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 45870.6us
I20260709 12:27:59.384341 12712 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.297341 13147 consensus_queue.cc:260] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:59.361171 13022 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
W20260709 12:27:59.419932 13141 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.663s	user 0.009s	sys 0.001s
I20260709 12:27:59.391533 13165 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.395325 12930 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:59.420032 13149 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.724s	user 0.000s	sys 0.004s
I20260709 12:27:59.448551  5946 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:27:59.428263 12613 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 12:27:59.454497  5838 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:27:59.409466 13078 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:27:59.440477 13170 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.475s	user 0.001s	sys 0.000s
I20260709 12:27:59.452941  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:59.380702 13136 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:59.377912 13169 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:27:59.483254  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 172021us
I20260709 12:27:59.447705 13165 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:59.486742  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:59.453819 12503 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:27:59.481261 13178 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 12:27:59.502743  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 207416us
W20260709 12:27:59.501219  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 165684us
I20260709 12:27:59.466440 13148 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:59.526582  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:59.480350 13103 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:59.524443  5945 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:59.425490 13176 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.082s
I20260709 12:27:59.523305  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:59.526907  5987 connection.cc:256] RPC call timeout handler was delayed by -1.1039738089998536s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2219s for a response
I20260709 12:27:59.439999 12712 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:59.532641  5987 connection.cc:256] RPC call timeout handler was delayed by -1.0832738089998202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2242s for a response
W20260709 12:27:59.534305  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.534458 13078 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:27:59.535446 13078 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:59.535090 12989 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.534948  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:59.536942 12989 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:59.538843 13078 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:59.540772 12989 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:59.534693 12712 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:59.535216 12647 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.542467 12647 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:59.545398 12647 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:59.552712  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:27:59.553541  6023 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:27:59.558382  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:59.490381 12914 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:59.421752 13174 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:59.451115 13141 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:59.606117  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:59.607057  5944 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
W20260709 12:27:59.548694 13170 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:27:59.613519  5946 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:59.602752 13127 consensus_meta.cc:287] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.962s	user 0.009s	sys 0.005s
W20260709 12:27:59.609273 13149 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:27:59.527453 12503 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:59.548777 13148 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:27:59.605824 13181 raft_consensus.cc:2804] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:59.604727 13103 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:59.623431  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 136759us
I20260709 12:27:59.630198  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2145) took 1705 ms. Trace:
I20260709 12:27:59.627537  5840 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:59.630362 12503 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:59.631433  5943 rpcz_store.cc:276] 0709 12:27:57.924401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.924576 (+   175us) service_pool.cc:224] Handling call
0709 12:27:57.940655 (+ 16079us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:58.165542 (+224887us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:58.165576 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:58.208062 (+ 42486us) log.cc:844] Serialized 26 byte log entry
0709 12:27:58.438935 (+230873us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:58.463847 (+ 24912us) log.cc:844] Serialized 14 byte log entry
0709 12:27:58.464148 (+   301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.464243 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:59.419235 (+954992us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:27:59.629852 (+210617us) raft_consensus.cc:1685] finished
0709 12:27:59.629963 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:59.630123 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18168,"prepare.run_wall_time_us":3857,"raft.queue_time_us":82353,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":24067,"spinlock_wait_cycles":22752768,"thread_start_us":350221,"threads_started":2,"wal-append.queue_time_us":396998}
I20260709 12:27:59.620401 13127 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:59.615680  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 152566us
I20260709 12:27:59.635462  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:27:59.636993  5838 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:34714 dropped due to backpressure: service queue is full with 50 items
I20260709 12:27:59.637400  5838 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:34714 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1662)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2220)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1478)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2221)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1479)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2222)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2224)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1468)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1676)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1485)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2223)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1481)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1484)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1488)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1487)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1661)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1489)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1471)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1652)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1664)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1474)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1491)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2234)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1469)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1658)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1472)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1665)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1672)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1673)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1675)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2219)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1475)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1476)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1477)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2227)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1480)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1482)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1483)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2229)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1486)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2230)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2231)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2232)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2233)
I20260709 12:27:59.558375 13168 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:59.558888 13139 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:59.647825  5943 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
I20260709 12:27:59.648312 12310 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.649057 12310 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:59.650167 13181 raft_consensus.cc:697] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:59.650600 13103 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:27:59.614778  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 46886us
I20260709 12:27:59.622011  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:27:59.615391 13152 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.750s	user 0.002s	sys 0.012s
I20260709 12:27:59.654526 13103 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:59.655292 13152 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:59.656458 13169 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:59.622692 13067 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:59.658038  6793 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:59.658583 13176 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:59.490059 13114 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:59.659855 13139 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.660465 13139 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:59.660638 13114 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:59.661726  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:59.662441  5944 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:59.662676 13139 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:27:59.663233 13168 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.663463  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:59.652498 12914 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:59.665356  5944 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:59.663900 13168 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:59.651877 13181 consensus_queue.cc:237] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:59.643208  5838 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:54876 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1662)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2220)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1478)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2221)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1479)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2222)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2224)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1468)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1676)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1485)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2223)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1481)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1484)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1488)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1487)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1661)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1489)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1471)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1652)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1664)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1474)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1491)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2234)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1469)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1658)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1472)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1665)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1672)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1673)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1675)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2219)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1475)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1476)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1477)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2227)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1480)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1482)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1483)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2229)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1486)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2230)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2231)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2232)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2233)
I20260709 12:27:59.654148  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:27:59.680879  5838 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:54876 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1662)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2220)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1478)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2221)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1479)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2222)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2224)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1468)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1676)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1485)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2223)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1481)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1484)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1488)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1487)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1661)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1489)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1471)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1652)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1664)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1474)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1491)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2234)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1469)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1658)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1472)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1665)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1672)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1673)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1675)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2219)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1475)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1476)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1477)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2227)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1480)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1482)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1483)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2229)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1486)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2230)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2231)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2232)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2233)
I20260709 12:27:59.654505 13018 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:59.682263  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 146884us
I20260709 12:27:59.690044  5946 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:27:59.628943 13136 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:59.696702  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:27:59.698591  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:59.685468 13188 consensus_queue.cc:1059] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:27:59.695684 12914 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:27:59.711091 13169 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:59.711658 13169 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.711985 13169 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:59.704401  5987 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:27:59.681227 13185 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:59.695856 13197 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:59.721072 13197 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:59.722327 13197 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:59.723820 13197 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:59.728325 13175 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:59.729866  5872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2211) took 2097 ms. Trace:
I20260709 12:27:59.730438  5872 rpcz_store.cc:276] 0709 12:27:57.632069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.632292 (+   223us) service_pool.cc:224] Handling call
0709 12:27:57.654388 (+ 22096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.670559 (+ 16171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.729555 (+2058996us) spinlock_profiling.cc:243] Waited 8.32 ms on lock 0x7b54001d06d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:59.729754 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19139968}
I20260709 12:27:59.734304 13174 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:27:59.740140 12503 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:59.740808 12503 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:27:59.731992  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:59.742805  5872 raft_consensus.cc:2376] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19. Re-sending same reply.
W20260709 12:27:59.743814  5867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2217) took 1950 ms (client timeout 1594 ms). Trace:
I20260709 12:27:59.744392  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:59.715337 12989 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:59.744778 13173 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:59.745573 13173 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:59.746469 13173 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:27:59.744318  5867 rpcz_store.cc:269] 0709 12:27:57.791953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.012585 (+220632us) service_pool.cc:224] Handling call
0709 12:27:59.741582 (+1728997us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b54004a0454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:59.741777 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1273946880}
I20260709 12:27:59.718309  5944 raft_consensus.cc:2468] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:27:59.741729 12503 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:59.750653  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:59.745246  5865 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2220) took 1473 ms (client timeout 1837 ms). Trace:
I20260709 12:27:59.754007  5838 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:27:59.754369  5865 rpcz_store.cc:269] 0709 12:27:58.271951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.744056 (+1472105us) service_pool.cc:224] Handling call
0709 12:27:59.745180 (+  1124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.750869  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:59.756456  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:59.757529  5872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1662) took 2035 ms (client timeout 2242 ms). Trace:
W20260709 12:27:59.758080  5872 rpcz_store.cc:269] 0709 12:27:57.721833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.731594 (+2009761us) service_pool.cc:224] Handling call
0709 12:27:59.743386 (+ 11792us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.760340  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:59.716547  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:27:59.761044  5867 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:59.762544  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1729) took 4052 ms. Trace:
W20260709 12:27:59.763991  5865 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2221) took 1490 ms (client timeout 1857 ms). Trace:
W20260709 12:27:59.748759 13108 consensus_meta.cc:287] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.986s	user 0.013s	sys 0.000s
I20260709 12:27:59.762948  6018 rpcz_store.cc:276] 0709 12:27:55.709730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.236045 (+526315us) service_pool.cc:224] Handling call
0709 12:27:57.245539 (+1009494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:57.350042 (+104503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:57.350092 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:57.358691 (+  8599us) log.cc:844] Serialized 26 byte log entry
0709 12:27:57.438530 (+ 79839us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:57.438715 (+   185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.438961 (+   246us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:57.552049 (+113088us) raft_consensus.cc:1685] finished
0709 12:27:57.552070 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:59.752925 (+2200855us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400764b50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:59.762457 (+  9532us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2482,"raft.queue_time_us":22928,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":16848,"spinlock_wait_cycles":2320276224,"thread_start_us":163475,"threads_started":2,"wal-append.queue_time_us":157999,"wal-append.run_cpu_time_us":9545,"wal-append.run_wall_time_us":1074811}
W20260709 12:27:59.767663  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51222.5us
W20260709 12:27:59.764467  5865 rpcz_store.cc:269] 0709 12:27:58.273585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.755275 (+1481690us) service_pool.cc:224] Handling call
0709 12:27:59.763922 (+  8647us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.768766  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:59.772024  5865 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2222) took 1498 ms (client timeout 1921 ms). Trace:
W20260709 12:27:59.772581  5865 rpcz_store.cc:269] 0709 12:27:58.273989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.768467 (+1494478us) service_pool.cc:224] Handling call
0709 12:27:59.771956 (+  3489us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.771944  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:59.774735  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:59.761808 12315 raft_consensus.cc:3055] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:59.765002 13102 raft_consensus.cc:2804] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:59.531080 13146 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:27:59.785548  5872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1479) took 1422 ms (client timeout 1821 ms). Trace:
I20260709 12:27:59.766672 13053 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:59.788072 13198 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:59.788246  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:27:59.789220  5944 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:59.784277  5865 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2224) took 1474 ms (client timeout 1899 ms). Trace:
I20260709 12:27:59.788239  5913 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:59.790155  5865 rpcz_store.cc:269] 0709 12:27:58.310264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.774463 (+1464199us) service_pool.cc:224] Handling call
0709 12:27:59.784157 (+  9694us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.790652 13198 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:59.791209  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:59.791630  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:27:59.791410  5872 rpcz_store.cc:269] 0709 12:27:58.362951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.761008 (+1398057us) service_pool.cc:224] Handling call
0709 12:27:59.785466 (+ 24458us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.793004  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:59.793540  5867 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
W20260709 12:27:59.794785  5867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1478) took 1446 ms (client timeout 1765 ms). Trace:
I20260709 12:27:59.793574  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:27:59.795739  5867 rpcz_store.cc:269] 0709 12:27:58.348290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.760026 (+1411736us) service_pool.cc:224] Handling call
0709 12:27:59.794697 (+ 34671us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:59.792078  5865 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1468) took 2115 ms (client timeout 2557 ms). Trace:
I20260709 12:27:59.792433 13198 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:59.798444 13141 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:27:59.800290  5865 rpcz_store.cc:269] 0709 12:27:57.676915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.790957 (+2114042us) service_pool.cc:224] Handling call
0709 12:27:59.792020 (+  1063us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:59.807226 12983 consensus_meta.cc:287] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.002s
I20260709 12:27:59.807847 12983 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:59.769713 13108 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:59.815143  5868 raft_consensus.cc:2368] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21.
W20260709 12:27:59.815783 12943 consensus_meta.cc:287] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.537s	user 0.013s	sys 0.000s
I20260709 12:27:59.815997  5942 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 12:27:59.816560  5942 pending_rounds.cc:85] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 13
I20260709 12:27:59.816884  5942 pending_rounds.cc:107] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:27:59.785072 12315 raft_consensus.cc:740] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:27:59.818102 13146 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:59.817132  5942 raft_consensus.cc:2889] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:59.816362 12943 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:59.820751 12943 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:27:59.821818 13108 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:27:59.824043 13136 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.818159 12315 consensus_queue.cc:260] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:59.827855 13168 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:27:59.828153  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.783838 13173 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:59.830034 13191 log.cc:927] Time spent T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.144s	user 0.000s	sys 0.002s
W20260709 12:27:59.840687  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.842669  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1385) took 1510 ms. Trace:
I20260709 12:27:59.843374  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:59.843760  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:59.844915  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:27:59.848587  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:27:59.833915  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:59.834600 13102 raft_consensus.cc:697] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 22 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:27:59.831617 13136 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:59.835512 12315 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:59.853294 13102 consensus_queue.cc:237] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.8, Last appended by leader: 8, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:27:59.865198  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:27:59.869525  5867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1485) took 1449 ms (client timeout 1899 ms). Trace:
W20260709 12:27:59.872478  5867 rpcz_store.cc:269] 0709 12:27:58.420636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.800854 (+1380218us) service_pool.cc:224] Handling call
0709 12:27:59.869426 (+ 68572us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:59.888499  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 95587.5us
W20260709 12:27:59.895900  5865 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2223) took 1586 ms (client timeout 2031 ms). Trace:
W20260709 12:27:59.896376  5865 rpcz_store.cc:269] 0709 12:27:58.309340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.832805 (+1523465us) service_pool.cc:224] Handling call
0709 12:27:59.895826 (+ 63021us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.843207  5940 rpcz_store.cc:276] 0709 12:27:58.332322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.499552 (+167230us) service_pool.cc:224] Handling call
0709 12:27:58.591056 (+ 91504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:58.595953 (+  4897us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:58.694641 (+ 98688us) log.cc:844] Serialized 14 byte log entry
0709 12:27:58.822456 (+127815us) log.cc:844] Serialized 14 byte log entry
0709 12:27:59.249746 (+427290us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:59.250380 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 12:27:59.250578 (+   198us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:59.250878 (+   300us) log.cc:844] Serialized 14 byte log entry
0709 12:27:59.251201 (+   323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.251313 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:59.842385 (+591072us) raft_consensus.cc:1685] finished
0709 12:27:59.842410 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:59.842575 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":225301,"prepare.run_cpu_time_us":153,"prepare.run_wall_time_us":1944,"raft.queue_time_us":66081,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":155,"spinlock_wait_cycles":27593472,"thread_start_us":580520,"threads_started":3,"wal-append.queue_time_us":172967}
I20260709 12:27:59.897677  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:27:59.898742  5865 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:27:59.899458  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1484) took 1484 ms. Trace:
I20260709 12:27:59.899916  5865 rpcz_store.cc:276] 0709 12:27:58.415572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.897336 (+1481764us) service_pool.cc:224] Handling call
0709 12:27:59.899373 (+  2037us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:59.898768  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:59.901954  5940 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:27:59.906625  5913 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:27:59.907205  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:59.895748 13197 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:59.848619 13022 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:59.943112  5987 leader_election.cc:336] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.737s (SENT)
I20260709 12:27:59.859400 12824 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:27:59.916023 13201 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:27:59.967242  5944 raft_consensus.cc:2468] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:27:59.919662 13176 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:59.974232  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:27:59.972750 13195 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.982393 13195 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:27:59.955193 13108 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:27:59.992200 13108 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:27:59.996299 13108 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:27:59.974927 13131 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.704s	user 0.003s	sys 0.011s
I20260709 12:28:00.001103  5870 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
W20260709 12:28:00.002281  5870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1481) took 1615 ms (client timeout 1995 ms). Trace:
I20260709 12:28:00.002058 13131 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:00.002763  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:00.002851  5870 rpcz_store.cc:269] 0709 12:27:58.386598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.848320 (+1461722us) service_pool.cc:224] Handling call
0709 12:28:00.002214 (+153894us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.004189  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.005091  5870 raft_consensus.cc:2410] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:28:00.005517  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1487) took 1181 ms. Trace:
I20260709 12:28:00.005901  5870 rpcz_store.cc:276] 0709 12:27:58.824221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.003904 (+1179683us) service_pool.cc:224] Handling call
0709 12:28:00.005450 (+  1546us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.006565  6019 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:59.910080  5840 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34730 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1488)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1487)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1661)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1489)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1471)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1652)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1664)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1474)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1491)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2234)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1678)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1685)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2236)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2238)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2237)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1490)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1679)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1681)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1682)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1683)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1680)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1469)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1658)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1472)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1665)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1672)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1673)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1675)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2219)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1475)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1476)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1477)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2227)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1480)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1482)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1483)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2229)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1486)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2230)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2231)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2232)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2233)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2239)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2240)
I20260709 12:28:00.007139  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.007992  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:00.008638  6016 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:00.009543  5840 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:34714 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1489)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1471)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1652)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1664)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1474)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1491)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2234)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1678)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1685)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2236)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1687)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2238)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2237)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1686)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1490)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1688)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1689)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1679)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1681)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1682)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1683)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1680)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1469)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1658)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1472)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1665)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1672)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1673)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1675)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2219)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1475)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1476)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1477)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2227)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1480)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1482)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1483)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2229)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1486)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2230)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2231)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2232)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2233)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2239)
W20260709 12:28:00.013504  5913 consensus_peers.cc:597] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:34714 dropped due to backpressure: service queue is full with 50 items. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:00.015115  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1382) took 1728 ms. Trace:
I20260709 12:28:00.015554  5948 rpcz_store.cc:276] 0709 12:27:58.287113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.287231 (+   118us) service_pool.cc:224] Handling call
0709 12:27:58.610259 (+323028us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:58.610964 (+   705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.014888 (+1403924us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54003084d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:00.015046 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741856000}
I20260709 12:28:00.016812  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:00.013864  5840 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34730 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1489)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1471)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1652)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1664)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1474)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1491)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2234)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1678)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1685)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2236)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1687)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2238)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2237)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1686)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1490)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1688)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1689)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1679)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1681)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1682)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1683)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1680)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1469)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1658)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1472)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1665)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1672)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1673)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1675)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2219)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1475)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1476)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1477)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2227)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1480)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1482)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1483)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2229)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1486)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2230)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2231)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2232)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2233)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2239)
I20260709 12:28:00.018369 13115 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:00.019507  6016 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:00.018612  5840 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34730 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1489)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1471)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1652)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1664)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1474)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1491)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2234)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1678)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1685)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2236)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1687)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2238)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2237)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1686)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1490)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1688)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1689)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1679)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1681)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1682)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1683)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1680)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1469)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1658)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1472)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1665)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1672)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1673)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1675)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2219)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1475)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1476)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1477)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2227)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1480)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1482)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1483)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2229)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1486)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2230)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2231)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2232)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2233)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2239)
I20260709 12:27:59.973372 13169 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:27:59.968565 12824 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:00.024449  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:00.035693  5940 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 23.
I20260709 12:28:00.032213  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:00.007966  5870 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1661) took 2289 ms (client timeout 2980 ms). Trace:
I20260709 12:28:00.017460  5948 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
W20260709 12:28:00.039567  5870 rpcz_store.cc:269] 0709 12:27:57.718231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.006755 (+2288524us) service_pool.cc:224] Handling call
0709 12:28:00.007889 (+  1134us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:59.993074  5874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1466) took 2329 ms (client timeout 1841 ms). Trace:
W20260709 12:28:00.042243  5874 rpcz_store.cc:269] 0709 12:27:57.663783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.668084 (+  4301us) service_pool.cc:224] Handling call
0709 12:27:59.992789 (+2324705us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54005d6bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:27:59.992949 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434878208}
I20260709 12:28:00.032423  5840 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34730 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1489)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1471)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1652)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1664)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1474)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1491)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2234)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1678)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1685)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2236)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1687)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2238)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2237)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1686)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1490)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1688)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1689)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1679)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1681)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1682)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1683)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1680)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1469)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1658)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1472)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1663)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1665)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1672)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1673)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1675)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2219)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1475)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1476)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1477)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2227)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1480)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1482)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1483)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2229)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1486)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2230)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2231)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2232)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2233)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2239)
I20260709 12:28:00.043416  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.043761  5915 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:00.044875  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:00.045161  5874 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
W20260709 12:28:00.045889  5874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1667) took 2251 ms (client timeout 2937 ms). Trace:
W20260709 12:28:00.046283  5874 rpcz_store.cc:269] 0709 12:27:57.794439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.043103 (+2248664us) service_pool.cc:224] Handling call
0709 12:28:00.045829 (+  2726us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.031623  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:00.048141  5944 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
I20260709 12:28:00.048219  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:00.045887  6023 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:27:59.980969 13165 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:00.050595  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:00.055352  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1489) took 1108 ms. Trace:
I20260709 12:28:00.056017  5874 rpcz_store.cc:276] 0709 12:27:58.947468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.047820 (+1100352us) service_pool.cc:224] Handling call
0709 12:28:00.055260 (+  7440us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.063812  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:00.064203  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:00.064615  6778 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:00.058001  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:00.067408  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:27:59.957248 13022 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:00.079598  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:00.062114  5915 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:28:00.080302  5987 leader_election.cc:336] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34730 dropped due to backpressure: service queue is full with 50 items
I20260709 12:28:00.079118  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:00.081804  6021 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:28:00.085602  6024 raft_consensus.cc:2410] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 21 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:28:00.021593 12310 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:00.028268 12824 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:00.087702  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:00.078996  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.123143  5874 raft_consensus.cc:2410] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 12:28:00.078198 12953 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.090680  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:00.094565 13150 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.049499 12503 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:00.095027 13119 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:00.095618 13165 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:00.025857 13195 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:00.028199 13052 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:00.025699 13169 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:00.081179  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:00.095652  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.102051  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:00.102403  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2137) took 2437 ms. Trace:
I20260709 12:28:00.132357  5947 rpcz_store.cc:276] 0709 12:27:57.665110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.665304 (+   194us) service_pool.cc:224] Handling call
0709 12:27:57.678051 (+ 12747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.860526 (+182475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.102110 (+2241584us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54004682d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:00.102333 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415190400}
I20260709 12:28:00.105614  5945 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:00.135810  5945 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:28:00.108346 13022 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:00.108963  5873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1670) took 2243 ms (client timeout 1795 ms). Trace:
I20260709 12:28:00.139784  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:28:00.139889  5873 rpcz_store.cc:269] 0709 12:27:57.865195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.496301 (+631106us) service_pool.cc:224] Handling call
0709 12:28:00.108722 (+1612421us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54003890d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:00.108878 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2409265920}
I20260709 12:28:00.112545 13130 raft_consensus.cc:3055] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:28:00.148762  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 68542.5us
I20260709 12:28:00.113788 12951 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:28:00.124979  5874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1471) took 2428 ms (client timeout 3118 ms). Trace:
I20260709 12:28:00.125803  5944 raft_consensus.cc:2410] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 24 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
W20260709 12:28:00.127285  6015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1648) took 2158 ms (client timeout 2458 ms). Trace:
W20260709 12:28:00.159982  5874 rpcz_store.cc:269] 0709 12:27:57.696907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.078498 (+2381591us) service_pool.cc:224] Handling call
0709 12:28:00.124902 (+ 46404us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.127300  6756 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:00.056841 13176 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:00.131381  6773 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:28:00.162047  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:00.137935 13123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.113s	user 0.023s	sys 0.031s
W20260709 12:28:00.162942 13123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.138s	user 0.023s	sys 0.032s
I20260709 12:28:00.129333  6778 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:28:00.141789  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.163159  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:00.164198  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1664) took 2423 ms. Trace:
I20260709 12:28:00.164445  6773 raft_consensus.cc:2393] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 26: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:28:00.164672  5873 rpcz_store.cc:276] 0709 12:27:57.741120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.141390 (+2400270us) service_pool.cc:224] Handling call
0709 12:28:00.164122 (+ 22732us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.139065  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2226) took 1805 ms. Trace:
I20260709 12:28:00.149581  5866 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 20: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:00.166148  5870 rpcz_store.cc:276] 0709 12:27:58.334567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.054991 (+1720424us) service_pool.cc:224] Handling call
0709 12:28:00.138975 (+ 83984us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:00.167516  5866 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1652) took 2489 ms (client timeout 3211 ms). Trace:
I20260709 12:28:00.167871  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:00.168133  5866 rpcz_store.cc:269] 0709 12:27:57.678170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.124606 (+2446436us) service_pool.cc:224] Handling call
0709 12:28:00.167397 (+ 42791us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.169462  5870 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 25: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:28:00.160705  6015 rpcz_store.cc:269] 0709 12:27:57.971235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.995249 (+ 24014us) service_pool.cc:224] Handling call
0709 12:28:00.127103 (+2131854us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b540041da54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:00.127220 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436836736}
I20260709 12:28:00.170125  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2234) took 1298 ms. Trace:
I20260709 12:28:00.170579  5870 rpcz_store.cc:276] 0709 12:27:58.871735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.167545 (+1295810us) service_pool.cc:224] Handling call
0709 12:28:00.170041 (+  2496us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.170895  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:00.171886  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.159942  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:00.171886  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.173004  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.174099  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:00.162956  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1474) took 2440 ms. Trace:
I20260709 12:28:00.174939  5870 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:00.175518  5874 rpcz_store.cc:276] 0709 12:27:57.722788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.161750 (+2438962us) service_pool.cc:224] Handling call
0709 12:28:00.162890 (+  1140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.176597  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:00.177119  6768 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:00.177838  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.166306  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.179829  5874 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:00.180006  5873 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
I20260709 12:28:00.175876  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:00.178594  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:00.181545  5945 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: [21.9-21.9]
I20260709 12:28:00.174962  6609 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:28:00.129803 12909 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:00.183466  5913 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:00.183789 12909 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:00.147934 13119 raft_consensus.cc:697] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:00.186173 13119 consensus_queue.cc:237] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:00.186553 12909 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:00.189204 13206 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:28:00.183254  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.179176  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.128651 12950 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:00.130414 13158 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.193176  5873 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:28:00.194042  6609 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:00.131330 12958 raft_consensus.cc:3055] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:00.193976 12951 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:00.201488  6609 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:00.202749 12951 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:00.203321  5945 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:28:00.196414 13188 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:00.157780 13195 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:00.207240  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:00.208110  6773 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:00.182307 13176 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:00.211766  5870 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
I20260709 12:28:00.212783  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.200301 12958 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:00.213732  5913 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:00.197749  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:00.217341  6778 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:28:00.217931  5940 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:28:00.219349  6778 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:00.231889  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:00.237227  6773 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:00.226613  6756 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
W20260709 12:28:00.238868  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.245493  6015 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:28:00.131588 13150 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:00.131990 12953 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:00.211634 13067 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:00.149114 13130 raft_consensus.cc:740] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:00.215888 13176 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:00.238698  6778 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 12:28:00.222016 13117 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:00.260442  6024 raft_consensus.cc:1240] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:00.247460 13168 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:00.216305 12950 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:00.222492 13158 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:00.252063 13195 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.266263  6020 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
W20260709 12:28:00.272207  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 70792.4us
I20260709 12:28:00.273257  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:00.265156 13168 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:00.275274  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:00.272823 12945 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:00.278822 13168 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:00.271092 13195 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:00.314059 13209 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 12:28:00.321393 13210 log.cc:927] Time spent T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 12:28:00.332489 13216 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:28:00.334434 13195 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:00.379021  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 312631us
I20260709 12:28:00.380625  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.382783  5866 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:28:00.384491  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:00.384905  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.387159  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.388587  5866 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 25: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:00.389875  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.391057  5866 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 24: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:00.392396  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:00.393316  5866 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:00.220988  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:00.399257  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:00.400084  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 165505us
I20260709 12:28:00.400405  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.402280  5870 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:28:00.405491  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1666) took 1930 ms. Trace:
W20260709 12:28:00.406883  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.413584  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:00.414275  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:00.422740  5945 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:28:00.428889  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.406236  6017 rpcz_store.cc:276] 0709 12:27:58.475051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.477586 (+  2535us) service_pool.cc:224] Handling call
0709 12:27:58.477918 (+   332us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:58.490708 (+ 12790us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:58.663099 (+172391us) log.cc:844] Serialized 14 byte log entry
0709 12:27:58.821678 (+158579us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:58.825851 (+  4173us) log.cc:844] Serialized 52 byte log entry
0709 12:27:58.826002 (+   151us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:58.826038 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.404215 (+1578177us) spinlock_profiling.cc:243] Waited 2.56 ms on lock 0x7b54005659d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:00.404342 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:00.404446 (+   104us) raft_consensus.cc:1685] finished
0709 12:28:00.404654 (+   208us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:00.405090 (+   436us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2161,"prepare.queue_time_us":232474,"prepare.run_cpu_time_us":408,"prepare.run_wall_time_us":1336,"raft.queue_time_us":4926,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":24526,"spinlock_wait_cycles":5884672,"thread_start_us":274166,"threads_started":2,"wal-append.queue_time_us":199202,"wal-append.run_cpu_time_us":4292,"wal-append.run_wall_time_us":1430118}
I20260709 12:28:00.424652  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:00.418615  5944 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:00.429940 12783 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:00.431151 12874 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 12:28:00.432126 12914 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:00.400733  5873 raft_consensus.cc:2376] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19. Re-sending same reply.
I20260709 12:28:00.431176 12909 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:00.443813 12989 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:00.282873 13130 consensus_queue.cc:260] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:00.268782 12958 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:00.452210 13130 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:00.453500 12958 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:00.453783  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:00.317493 13150 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:00.480854  6775 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 17.
I20260709 12:28:00.483327  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.484846 13067 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:00.432284 12887 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:28:00.457978 12783 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:00.485862  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:00.495644  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:00.496901 13123 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:00.497740 13123 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:00.498243 13123 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:00.497284  6019 raft_consensus.cc:2410] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:28:00.500424 12914 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:00.502737 12914 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:00.512070 12950 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:00.513034 13158 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:00.513962 13173 consensus_meta.cc:287] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.676s	user 0.021s	sys 0.004s
I20260709 12:28:00.514493 13219 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:00.514775 13223 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:00.515484 13223 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:00.515712  5872 raft_consensus.cc:2410] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:28:00.515990 13223 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:00.518841  6024 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:00.521999 12953 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:00.533304  5873 raft_consensus.cc:1275] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:28:00.541450 12783 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:00.543210  5868 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1653) took 2864 ms (client timeout 2219 ms). Trace:
W20260709 12:28:00.543872  5868 rpcz_store.cc:269] 0709 12:27:57.678475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.759930 (+1081455us) service_pool.cc:224] Handling call
0709 12:28:00.542668 (+1782738us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54004f5bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:00.542934 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2399378816}
W20260709 12:28:00.553975  5871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2213) took 2877 ms (client timeout 1987 ms). Trace:
I20260709 12:28:00.551972  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1658) took 2854 ms. Trace:
W20260709 12:28:00.554586  5871 rpcz_store.cc:269] 0709 12:27:57.676435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.676585 (+   150us) service_pool.cc:224] Handling call
0709 12:28:00.553212 (+2876627us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400026a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:00.553562 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3200984448}
I20260709 12:28:00.557888  5873 rpcz_store.cc:276] 0709 12:27:57.685191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.484843 (+2799652us) service_pool.cc:224] Handling call
0709 12:28:00.485535 (+   692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.537989 (+ 52454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.539253 (+  1264us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:00.567255  6020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1644) took 2599 ms (client timeout 1628 ms). Trace:
W20260709 12:28:00.568171  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 292803us
W20260709 12:28:00.568055  6020 rpcz_store.cc:269] 0709 12:27:57.967638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.970130 (+  2492us) service_pool.cc:224] Handling call
0709 12:28:00.566641 (+2596511us) spinlock_profiling.cc:243] Waited 2.28 s on lock 0x7b540065b3d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:00.566941 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5237636480}
I20260709 12:28:00.569814  5865 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:00.573231  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2215) took 2870 ms. Trace:
I20260709 12:28:00.575671  5871 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:00.576946  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1665) took 2800 ms. Trace:
I20260709 12:28:00.577484  5871 rpcz_store.cc:276] 0709 12:27:57.776471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.572822 (+2796351us) service_pool.cc:224] Handling call
0709 12:28:00.575180 (+  2358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.576537 (+  1357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.576844 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.582401  5873 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:00.583508  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1663) took 2855 ms. Trace:
I20260709 12:28:00.583930  5873 rpcz_store.cc:276] 0709 12:27:57.727956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.563861 (+2835905us) service_pool.cc:224] Handling call
0709 12:28:00.581861 (+ 18000us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.583098 (+  1237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.583392 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.573488  5865 rpcz_store.cc:276] 0709 12:27:57.703055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.512848 (+2809793us) service_pool.cc:224] Handling call
0709 12:28:00.513548 (+   700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.571770 (+ 58222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.573169 (+  1399us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.603246  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1668) took 1051 ms. Trace:
I20260709 12:28:00.603789  6022 rpcz_store.cc:276] 0709 12:27:59.551536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.552040 (+   504us) service_pool.cc:224] Handling call
0709 12:27:59.604592 (+ 52552us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:59.605062 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:59.618474 (+ 13412us) log.cc:844] Serialized 14 byte log entry
0709 12:28:00.415010 (+796536us) log.cc:844] Serialized 14 byte log entry
0709 12:28:00.415383 (+   373us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:00.416467 (+  1084us) log.cc:844] Serialized 26 byte log entry
0709 12:28:00.416662 (+   195us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:00.416693 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.416803 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:00.602768 (+185965us) raft_consensus.cc:1685] finished
0709 12:28:00.602792 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:00.603068 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":39148,"prepare.run_cpu_time_us":180,"prepare.run_wall_time_us":17871,"raft.queue_time_us":85345,"raft.run_cpu_time_us":845,"raft.run_wall_time_us":68301,"thread_start_us":730751,"threads_started":2,"wal-append.queue_time_us":515920}
I20260709 12:28:00.609835 13067 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:00.615532  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:00.752307  6024 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:28:00.725605  5868 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:00.733349  5866 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:28:00.768206  5873 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:28:00.773330  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.774706 12783 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.775588 12783 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:00.776755  5865 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:28:00.777673  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1675) took 2819 ms. Trace:
I20260709 12:28:00.778065  5865 rpcz_store.cc:276] 0709 12:27:57.958636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.622188 (+2663552us) service_pool.cc:224] Handling call
0709 12:28:00.776134 (+153946us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.777273 (+  1139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.777558 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.784289 12783 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:00.789978  6793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1870) took 1826 ms. Trace:
I20260709 12:28:00.790159 13156 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:28:00.791236  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:00.791994  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.795693 13018 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.790709  6793 rpcz_store.cc:276] 0709 12:27:58.963338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.264501 (+301163us) service_pool.cc:224] Handling call
0709 12:27:59.281307 (+ 16806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:59.658585 (+377278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.789308 (+1130723us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b5400008554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:00.789675 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865719552}
I20260709 12:28:00.800268  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1673) took 2872 ms. Trace:
I20260709 12:28:00.805476  5870 raft_consensus.cc:1240] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:00.785244  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48330 (request call id 12) took 2450 ms. Trace:
I20260709 12:28:00.806805  5871 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:00.815903  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1672) took 2913 ms. Trace:
I20260709 12:28:00.815488 13219 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:00.809751  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1472) took 3067 ms. Trace:
I20260709 12:28:00.819660  5868 rpcz_store.cc:276] 0709 12:27:57.703401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.544977 (+2841576us) service_pool.cc:224] Handling call
0709 12:28:00.545593 (+   616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.767306 (+221713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.770657 (+  3351us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:00.805789  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 190802us
I20260709 12:28:00.816380  5871 rpcz_store.cc:276] 0709 12:27:57.902976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.585260 (+2682284us) service_pool.cc:224] Handling call
0709 12:28:00.806336 (+221076us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.815540 (+  9204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.815831 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.800961  5873 rpcz_store.cc:276] 0709 12:27:57.928019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.590147 (+2662128us) service_pool.cc:224] Handling call
0709 12:28:00.767628 (+177481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.768829 (+  1201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.799875 (+ 31046us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54001446d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:00.800153 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380316928}
I20260709 12:28:00.814329  5941 rpcz_store.cc:276] 0709 12:27:58.347345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.448585 (+101240us) service_pool.cc:224] Handling call
0709 12:28:00.784607 (+2336022us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b4c0031ec40. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d0f4fa20 00007f39d0f60a29 00007f39d0f26644 00007f39d0f4df2d 00007f39d0f4db93 00007f39d2c7dc92 00007f39d2c7d84a 00007f39d7c4b60a 00007f39d11d17d4 00007f39d11d1762 00007f39d11d1691
0709 12:28:00.784912 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3716480}
I20260709 12:28:00.817783  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1469) took 3140 ms. Trace:
I20260709 12:28:00.822178  5870 rpcz_store.cc:276] 0709 12:27:57.677202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.804409 (+3127207us) service_pool.cc:224] Handling call
0709 12:28:00.805062 (+   653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.817317 (+ 12255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.817692 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.823241 13221 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.834987 13221 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:00.842347 12824 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:00.843758 12824 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:00.850368  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.843791  5871 raft_consensus.cc:1240] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 22 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:00.858011  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1477) took 2526 ms. Trace:
I20260709 12:28:00.858745  5871 rpcz_store.cc:276] 0709 12:27:58.331517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.823248 (+2491731us) service_pool.cc:224] Handling call
0709 12:28:00.843432 (+ 20184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.856074 (+ 12642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.857917 (+  1843us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:00.859236  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.862414  5865 raft_consensus.cc:1240] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:00.864212  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2219) took 2592 ms. Trace:
I20260709 12:28:00.864431  5865 rpcz_store.cc:276] 0709 12:27:58.271618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.830933 (+2559315us) service_pool.cc:224] Handling call
0709 12:28:00.862152 (+ 31219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.863953 (+  1801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.864160 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.808709 13018 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:00.883159 12923 consensus_queue.cc:1059] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:00.887892  6609 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:00.892334  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1482) took 2488 ms. Trace:
I20260709 12:28:00.896258  5865 rpcz_store.cc:276] 0709 12:27:58.403617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.874126 (+2470509us) service_pool.cc:224] Handling call
0709 12:28:00.890661 (+ 16535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.891270 (+   609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.891319 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:00.891403 (+    84us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:00.891525 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.891642 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:00.892214 (+   572us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.889873  5871 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:00.898619  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1480) took 2522 ms. Trace:
I20260709 12:28:00.898999  5871 rpcz_store.cc:276] 0709 12:27:58.376454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.860393 (+2483939us) service_pool.cc:224] Handling call
0709 12:28:00.885998 (+ 25605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.898276 (+ 12278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.898549 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.894349 12783 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.900453 12783 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:00.899221  5868 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:00.905599  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2228) took 2564 ms. Trace:
I20260709 12:28:00.908758  5870 rpcz_store.cc:276] 0709 12:27:58.341581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.828514 (+2486933us) service_pool.cc:224] Handling call
0709 12:28:00.885270 (+ 56756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.904946 (+ 19676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.904993 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:00.905090 (+    97us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:00.905179 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.905268 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:00.905438 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.910626  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1476) took 2613 ms. Trace:
I20260709 12:28:00.910969  5868 rpcz_store.cc:276] 0709 12:27:58.297643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.828034 (+2530391us) service_pool.cc:224] Handling call
0709 12:28:00.875745 (+ 47711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.903988 (+ 28243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.910557 (+  6569us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:00.914453  6757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1419) took 2671 ms (client timeout 1898 ms). Trace:
W20260709 12:28:00.914987  6757 rpcz_store.cc:269] 0709 12:27:58.243376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.243884 (+   508us) service_pool.cc:224] Handling call
0709 12:28:00.913889 (+2670005us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b5400008f54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:00.914118 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2216057216}
I20260709 12:28:00.919314 12783 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:00.924114 13223 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:00.910193 13147 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:00.926750 13176 consensus_meta.cc:287] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.658s	user 0.015s	sys 0.007s
I20260709 12:28:00.927328 13176 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:00.929608 13115 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:00.893002 13131 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:00.934716  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:00.934581 13123 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:00.935638  5869 raft_consensus.cc:2376] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 25. Re-sending same reply.
W20260709 12:28:00.936336  5869 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1677) took 2732 ms (client timeout 1737 ms). Trace:
I20260709 12:28:00.912231 13221 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:00.936705  5869 rpcz_store.cc:269] 0709 12:27:58.204144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.764206 (+560062us) service_pool.cc:224] Handling call
0709 12:28:00.934389 (+2170183us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b58002e2f98. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:28:00.936256 (+  1867us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6572288}
I20260709 12:28:00.938330  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2231) took 2164 ms. Trace:
I20260709 12:28:00.938946  5874 rpcz_store.cc:276] 0709 12:27:58.773753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.932492 (+2158739us) service_pool.cc:224] Handling call
0709 12:28:00.932980 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.933267 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.933296 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:00.933358 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:00.933435 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.933492 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:00.938058 (+  4566us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.940779  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.942266  6609 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:00.942961  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.945040  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.947702  5871 raft_consensus.cc:1240] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:28:00.942343  5874 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 28: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:00.947829  5869 raft_consensus.cc:2410] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 12:28:00.929483  6611 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:00.951732 13208 log.cc:927] Time spent T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.749s	user 0.009s	sys 0.007s
I20260709 12:28:00.951947 13162 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:00.952029  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2229) took 2454 ms. Trace:
I20260709 12:28:00.953773  5871 rpcz_store.cc:276] 0709 12:27:58.498052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.913091 (+2415039us) service_pool.cc:224] Handling call
0709 12:28:00.947226 (+ 34135us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:00.951663 (+  4437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.951962 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:00.953131 12950 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:28:00.954555  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.956985  5865 raft_consensus.cc:1240] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:00.957293 13219 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:00.958233 13219 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:00.958909 13018 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:00.960882  6611 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:00.963205 13123 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:00.963372 13134 consensus_queue.cc:1059] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:00.964762 13134 raft_consensus.cc:3055] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:00.961429  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1483) took 2553 ms. Trace:
I20260709 12:28:00.970034 13223 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:00.970201  5865 rpcz_store.cc:276] 0709 12:27:58.408237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.910443 (+2502206us) service_pool.cc:224] Handling call
0709 12:28:00.956579 (+ 46136us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.957572 (+   993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.961360 (+  3788us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160512}
I20260709 12:28:00.965411 13134 raft_consensus.cc:740] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:00.971848  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:00.972891 13174 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.973662 13174 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:00.975090 13136 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:00.975520  5948 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:28:00.976328 13168 consensus_meta.cc:287] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.664s	user 0.018s	sys 0.000s
I20260709 12:28:00.976848 13168 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:00.980126 13168 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:00.981307 12824 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:00.981832  5946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42740 (request call id 2156) took 1627 ms (client timeout 1938 ms). Trace:
I20260709 12:28:00.982291  5871 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:00.982154  5947 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 17. Current term is 18. Ops: []
W20260709 12:28:01.019608  5946 rpcz_store.cc:269] 0709 12:27:59.353772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.452625 (+ 98853us) service_pool.cc:224] Handling call
0709 12:28:00.981250 (+1528625us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b5400308ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:00.981464 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329394944}
I20260709 12:28:01.020825  5871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2233) took 2223 ms. Trace:
I20260709 12:28:01.021248  5871 rpcz_store.cc:276] 0709 12:27:58.797535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.955188 (+2157653us) service_pool.cc:224] Handling call
0709 12:28:00.982051 (+ 26863us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.020468 (+ 38417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.020750 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:01.022071  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:01.022831  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:01.026023  6022 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:28:01.026291 12824 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.026926 12824 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:01.030618  6019 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:28:01.031028 12824 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:00.990172 12989 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:00.994506  6762 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:00.994750  6793 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23. Ops: []
W20260709 12:28:00.994860  5872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1676) took 2791 ms (client timeout 2078 ms). Trace:
I20260709 12:28:00.995282 13223 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:00.996886  6757 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:00.995138  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:00.972764 13134 consensus_queue.cc:260] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:00.999254  5867 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1488) took 2148 ms (client timeout 1629 ms). Trace:
W20260709 12:28:01.001565  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:01.003500  6775 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1443) took 1159 ms (client timeout 1533 ms). Trace:
I20260709 12:28:01.004120  6756 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:28:01.009846 13152 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:01.004922 13108 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:01.044919  6019 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:28:01.045267 13108 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:01.046739 13108 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:01.048128 13108 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:01.048918 13174 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:01.005065 12315 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.050293 13223 raft_consensus.cc:697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:00.983222  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2149) took 2936 ms. Trace:
I20260709 12:28:01.051704 13223 consensus_queue.cc:237] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:01.018941 13130 consensus_meta.cc:287] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.566s	user 0.019s	sys 0.004s
I20260709 12:28:01.022197  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:28:01.038839  5867 rpcz_store.cc:269] 0709 12:27:58.850611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.998843 (+2148232us) service_pool.cc:211] Skipping call since client already timed out
0709 12:28:00.999040 (+   197us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:28:01.039773  6775 rpcz_store.cc:269] 0709 12:27:59.844214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.846002 (+  1788us) service_pool.cc:224] Handling call
0709 12:28:01.003131 (+1157129us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b5400003554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:01.003390 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1264390528}
I20260709 12:28:01.032939 13067 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:28:01.035004  5872 rpcz_store.cc:269] 0709 12:27:58.203821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.793257 (+1589436us) service_pool.cc:224] Handling call
0709 12:28:00.994537 (+1201280us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b54002f62d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:00.994768 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1652105472}
I20260709 12:28:01.051235  5942 rpcz_store.cc:276] 0709 12:27:58.046541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.047006 (+   465us) service_pool.cc:224] Handling call
0709 12:27:58.047365 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:59.818931 (+1771566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.982693 (+1163762us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400633154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:00.983010 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4066371456}
I20260709 12:28:00.961390 13219 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:01.050132 12315 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:01.036193 12989 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:01.033972 13156 raft_consensus.cc:3055] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:01.056803  6766 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 23.
I20260709 12:28:01.056214  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:01.074213  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:01.074807 13134 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:01.080219  5872 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:28:01.076014  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:01.083171  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:28:01.077229  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:01.077983 13156 raft_consensus.cc:740] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:28:01.087459  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 141611us
I20260709 12:28:01.090780  5865 raft_consensus.cc:1240] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:28:01.094359  5865 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2239) took 1203 ms. Trace:
I20260709 12:28:01.094870  5865 rpcz_store.cc:276] 0709 12:27:59.890750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.973284 (+1082534us) service_pool.cc:224] Handling call
0709 12:28:01.036221 (+ 62937us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:01.092874 (+ 56653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.094255 (+  1381us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:01.095314  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:01.087778 13067 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 25 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:01.090399 12824 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:01.105896  5870 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2230) took 2333 ms. Trace:
I20260709 12:28:01.108816  5870 rpcz_store.cc:276] 0709 12:27:58.773036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.950264 (+2177228us) service_pool.cc:224] Handling call
0709 12:28:01.059262 (+108998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.096576 (+ 37314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:01.096618 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:01.096667 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:01.105478 (+  8811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.105586 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.105801 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:01.116142  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:01.117163  5872 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 28.
I20260709 12:28:01.121080  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:01.121687  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:01.112344 12824 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:01.132381 13067 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:01.139356  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 144398us
W20260709 12:28:01.146944  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 49946.6us
I20260709 12:28:01.149405  5872 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:28:01.131201 12824 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:01.145916  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:01.151144  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:01.157614  6762 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
W20260709 12:28:01.163576  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.164316  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:01.165653  6773 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 25.
I20260709 12:28:01.166043  6757 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:28:01.166582 12647 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.167292 12647 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:01.171442 12647 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:01.172648 12647 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.174160 12647 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:01.177281 12647 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:01.178285 12647 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.179000 12647 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:01.182180 13232 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:28:01.182214 12647 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:01.189811 13197 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:01.190907 13197 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:01.201424  5874 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:01.202863  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2232) took 2428 ms. Trace:
I20260709 12:28:01.203256  5874 rpcz_store.cc:276] 0709 12:27:58.774075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.977811 (+2203736us) service_pool.cc:224] Handling call
0709 12:28:01.200780 (+222969us) spinlock_profiling.cc:243] Waited 58.2 ms on lock 0x7b58002e2f98. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:28:01.201089 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.202322 (+  1233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.202749 (+   427us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133961472}
W20260709 12:28:01.212132  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 53086.6us
I20260709 12:28:01.216450 13176 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:01.226357  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:01.226567 12989 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:01.228094  5871 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:28:01.208976 13197 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:01.217536  5868 raft_consensus.cc:1240] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 19. Current term is 20. Ops: []
W20260709 12:28:01.227982  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 62673.7us
I20260709 12:28:01.232820  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1486) took 2512 ms. Trace:
I20260709 12:28:01.233152  5868 rpcz_store.cc:276] 0709 12:27:58.718698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.915374 (+2196676us) service_pool.cc:224] Handling call
0709 12:28:01.217117 (+301743us) spinlock_profiling.cc:243] Waited 62.8 ms on lock 0x7b58002e2f98. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:28:01.217312 (+   195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.230604 (+ 13292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.230798 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144470784}
I20260709 12:28:01.155200 13174 consensus_queue.cc:1059] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:28:01.238685  5838 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:01.237116 13115 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.245250  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:01.244910 12954 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.240024 13136 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:01.250964  5942 raft_consensus.cc:2410] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:28:01.160512 13156 consensus_queue.cc:260] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:01.261176  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 100018us
I20260709 12:28:01.245361 12647 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:28:01.262316  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:01.263202 12958 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:01.268297 12954 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:01.271788 12542 raft_consensus.cc:2804] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:01.272727 12542 raft_consensus.cc:697] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:01.273540 12954 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:01.273861  5990 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:01.273912 12542 consensus_queue.cc:237] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:01.277311  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:01.277709 12909 consensus_queue.cc:1059] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:28:01.278316  5943 raft_consensus.cc:2393] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 28: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:01.278901 12874 raft_consensus.cc:3055] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:01.275508  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:01.256392 12989 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:01.279381 12874 raft_consensus.cc:740] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:01.276394  5840 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:01.284157  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:01.285260 12958 consensus_queue.cc:1059] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:01.285866  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2227) took 2944 ms. Trace:
I20260709 12:28:01.286368  6756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43760 (request call id 1584) took 1131 ms. Trace:
I20260709 12:28:01.285969 13136 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:01.289255 12909 consensus_queue.cc:1243] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 22.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:28:01.289763  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.243805 13130 consensus_queue.cc:1059] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:28:01.290948  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1475) took 3005 ms. Trace:
I20260709 12:28:01.307003  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:01.285712 13018 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:01.295504 12647 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:01.293123 12315 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:01.295789  5949 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:28:01.302809  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:01.309862  5866 rpcz_store.cc:276] 0709 12:27:58.285584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.827150 (+2541566us) service_pool.cc:224] Handling call
0709 12:28:00.873202 (+ 46052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:00.955955 (+ 82753us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:00.996099 (+ 40144us) log.cc:844] Serialized 14 byte log entry
0709 12:28:01.230809 (+234710us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:01.244439 (+ 13630us) log.cc:844] Serialized 26 byte log entry
0709 12:28:01.244571 (+   132us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:01.244601 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.244658 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.290618 (+ 45960us) raft_consensus.cc:1685] finished
0709 12:28:01.290648 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.290857 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39405,"prepare.queue_time_us":39652,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":3091,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":9332,"thread_start_us":234179,"threads_started":1,"wal-append.queue_time_us":279583}
I20260709 12:28:01.257592 13115 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:01.294710  6756 rpcz_store.cc:276] 0709 12:28:00.154852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.241287 (+ 86435us) service_pool.cc:224] Handling call
0709 12:28:01.003419 (+762132us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.039982 (+ 36563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.286118 (+246136us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b5400003550. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:01.286300 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1751337728}
I20260709 12:28:01.312536  5943 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 25: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:01.293367  5873 rpcz_store.cc:276] 0709 12:27:58.341253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.831407 (+2490154us) service_pool.cc:224] Handling call
0709 12:28:00.874446 (+ 43039us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:00.903916 (+ 29470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.903960 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:00.928949 (+ 24989us) log.cc:844] Serialized 52 byte log entry
0709 12:28:01.091925 (+162976us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:01.091981 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.092064 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.285517 (+193453us) raft_consensus.cc:1685] finished
0709 12:28:01.285537 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.285790 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17138,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":2177,"thread_start_us":157407,"threads_started":1,"wal-append.queue_time_us":320982}
W20260709 12:28:01.320701  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 72223us
I20260709 12:28:01.326268  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:01.329737  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:01.329430  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:01.337327  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 214285us
I20260709 12:28:01.338645  5949 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:28:01.315711 13138 raft_consensus.cc:3055] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:01.316488 13018 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.350183  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:01.352658 13105 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:28:01.354203  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.351881  5867 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 26.
W20260709 12:28:01.358865  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 85124.7us
I20260709 12:28:01.333668 13067 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.362603 13067 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:01.363091  5990 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:01.365368  5990 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:01.359563  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:01.357820  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:01.358412  6773 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:01.353363  5840 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:01.358840 13223 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:01.328248 12874 consensus_queue.cc:260] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:01.351040 13018 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:01.349644 13138 raft_consensus.cc:740] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:01.357616 13128 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:28:01.338635 13219 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.365311 13067 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:01.383813  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1387) took 2001 ms. Trace:
I20260709 12:28:01.384334  5947 rpcz_store.cc:276] 0709 12:27:59.382841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.133809 (+750968us) service_pool.cc:224] Handling call
0709 12:28:00.981524 (+847715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.020057 (+ 38533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.383530 (+363473us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b5400308ed0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:01.383725 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1839859584}
I20260709 12:28:01.385103  6762 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:28:01.387512 13223 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:01.388200 13223 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:01.366544 12958 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:01.365809 12647 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:01.389811 12954 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:01.390801 12954 raft_consensus.cc:697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:01.391191  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:01.392093  5873 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:28:01.392221 12954 consensus_queue.cc:237] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:01.393172  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:28:01.381278  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.382812 13182 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.383273 12874 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:01.394949  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2157) took 2036 ms. Trace:
I20260709 12:28:01.394639 13182 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:01.356570 13105 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:01.369498 13136 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:01.379588  5941 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 26.
I20260709 12:28:01.405529  5948 rpcz_store.cc:276] 0709 12:27:59.358300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.086822 (+728522us) service_pool.cc:224] Handling call
0709 12:28:00.087226 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.975985 (+888759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.372669 (+396684us) spinlock_profiling.cc:243] Waited 887 ms on lock 0x7b54003089d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:01.394875 (+ 22206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2041113856}
I20260709 12:28:01.379791  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:01.412418  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.414076 13175 consensus_queue.cc:1059] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:01.415589 13175 raft_consensus.cc:3055] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:01.416008 13175 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:01.409334 13115 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:01.419191  5987 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34730 dropped due to backpressure: service queue is full with 50 items
I20260709 12:28:01.419775  5987 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:01.421334  5987 leader_election.cc:336] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34730 dropped due to backpressure: service queue is full with 50 items
I20260709 12:28:01.365583 13188 consensus_queue.cc:1059] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 12:28:01.410748  5946 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:28:01.433956  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:01.434854  6793 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 27.
I20260709 12:28:01.435997  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:01.436739  6762 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:01.439277  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:01.423388  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:01.442176  5947 raft_consensus.cc:2376] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18. Re-sending same reply.
I20260709 12:28:01.442790 13174 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.443548 13174 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:01.391266 13219 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:01.445868 13221 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:01.446310 13221 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:01.447517  5915 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:01.447366 13221 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:01.448045 13219 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:01.448863  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:01.449747  6773 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 26.
I20260709 12:28:01.450484 13134 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:01.450671 13197 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:01.451009 13134 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:01.451092 13197 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.451447 13134 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:01.430989  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:01.453766  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:01.451531 13197 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:01.410205 13105 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:01.457825  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 90041.8us
I20260709 12:28:01.455097  5915 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:01.368278 12989 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.058s
W20260709 12:28:01.460932  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51049.6us
I20260709 12:28:01.462738  5873 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 25: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:01.464284  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:01.408577 12958 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:01.471616  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:01.469115  5913 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:01.474911  5944 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 8. Preceding OpId from leader: term: 22 index: 9. (index mismatch)
I20260709 12:28:01.448911 13221 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:01.469614  6766 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:28:01.417181 13175 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:01.389356 13223 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:01.481221  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:01.484218  6775 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:28:01.480144  5840 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:01.495669  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:01.478494 12989 consensus_queue.cc:1243] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 12:28:01.510635  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 98714us
I20260709 12:28:01.383843 13138 consensus_queue.cc:260] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 21.9, Last appended by leader: 9, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:01.519155  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 91967.2us
I20260709 12:28:01.518563  6762 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:28:01.524190  5913 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:28:01.526854  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 76059.5us
I20260709 12:28:01.527334 13138 raft_consensus.cc:3060] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:01.528530 13239 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:28:01.530133 13246 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:01.532075 12983 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:01.532783 12983 raft_consensus.cc:697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:01.533258 13128 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:01.534026 13128 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:01.534546 13128 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:01.537365  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:01.358429  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:01.502512 13176 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:01.549417 13176 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:01.550323  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:01.553963 13115 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:01.554464 13115 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.554872 13115 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:01.556666 13175 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:01.557785  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.558174 13176 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:01.560124 13156 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:01.562454 12983 consensus_queue.cc:237] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:01.562795 13197 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:01.564741 12958 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:01.565439 13197 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:01.565886 13137 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:01.568138  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 210869us
I20260709 12:28:01.570827 13223 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:01.571862  6785 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: []
W20260709 12:28:01.570827  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.544104  6766 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:28:01.576097  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:01.517187 13188 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:01.520190 12989 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:01.601861  6775 raft_consensus.cc:2393] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 27: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:28:01.590775  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 169513us
I20260709 12:28:01.582527  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:01.602346  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:01.599491  6757 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
W20260709 12:28:01.603021  5987 leader_election.cc:336] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34730 dropped due to backpressure: service queue is full with 50 items
I20260709 12:28:01.603533  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:01.603982 13197 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:01.604334  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:01.604525 13197 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.561456 13228 raft_consensus.cc:3055] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:01.605330  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:01.605218 13197 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:01.609247 13228 raft_consensus.cc:740] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:01.611860 12989 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:01.633658  5940 raft_consensus.cc:2410] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
W20260709 12:28:01.633898  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 161288us
I20260709 12:28:01.655982  6793 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:28:01.617060 13188 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:01.654611  5915 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:01.675706  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:01.676504  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:01.680927  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:01.677217  6022 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
W20260709 12:28:01.686105  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 210444us
I20260709 12:28:01.690879  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:01.691406  6020 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:28:01.634812 12954 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:28:01.657018 12542 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:01.660848 12647 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:01.698354  6019 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
W20260709 12:28:01.703411  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.707195  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:01.711778  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:01.686164 12783 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:01.687996 13176 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:01.730055  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:28:01.725486  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 120251us
I20260709 12:28:01.745997  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:01.747354 13067 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 12:28:01.716213 13219 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
W20260709 12:28:01.753851 13108 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.705s	user 0.004s	sys 0.012s
I20260709 12:28:01.754977 13253 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 12:28:01.756449 13253 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:01.756963 13253 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:01.761776 13223 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:01.764407 13188 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:01.770272  5869 raft_consensus.cc:2410] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:28:01.776584  5990 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:01.690296 12989 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:28:01.709025 13174 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:01.780515  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.786816  6017 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 24: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:01.788432  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:01.789265  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:01.790293  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:01.792023  6022 raft_consensus.cc:1275] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 12:28:01.794011  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:01.797490  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:01.797866  6756 raft_consensus.cc:1275] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 21 index: 9. Preceding OpId from leader: term: 22 index: 9. (term mismatch)
I20260709 12:28:01.781239 12315 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 12:28:01.716773 13182 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:01.800134  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:01.797930  6015 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:01.804484  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:01.802191  6020 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:28:01.803860  6756 pending_rounds.cc:85] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 8
I20260709 12:28:01.806833  6018 raft_consensus.cc:3060] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:01.811661  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 241990us
I20260709 12:28:01.802819  6022 pending_rounds.cc:85] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 10
I20260709 12:28:01.820405  5990 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:01.803381  6016 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:01.744447 13176 raft_consensus.cc:697] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:28:01.832456  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 228171us
I20260709 12:28:01.834039 13182 consensus_queue.cc:1059] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:01.834213  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:01.835016 13182 raft_consensus.cc:3055] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:01.835433 13182 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:01.836282 13182 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:01.837535 13182 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:01.840518 13108 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:01.841078 13108 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.841439 13108 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:01.824872  6022 pending_rounds.cc:107] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:28:01.850719  6022 raft_consensus.cc:2889] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:01.824178  6756 pending_rounds.cc:107] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 21.9
I20260709 12:28:01.852908  6756 raft_consensus.cc:2889] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:01.855273  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.854768  6611 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:28:01.863976  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.740077 12542 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.114s
W20260709 12:28:01.866343  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:01.866132  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 177647us
W20260709 12:28:01.875049  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 75040.2us
W20260709 12:28:01.881835  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 77578us
I20260709 12:28:01.883559  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:01.861747 13260 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.862015 12958 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:01.828413 13128 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:01.789408 13228 consensus_queue.cc:260] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:01.895574  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:01.908533  5940 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:01.875696 13174 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:01.862656 13138 consensus_queue.cc:1059] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:28:01.880749 13136 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:28:01.922150  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 67446.2us
I20260709 12:28:01.924686  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:01.926134  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:01.875183 12954 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:28:01.934307  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:01.936075  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:01.891733 12958 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:01.947358  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 67318.9us
W20260709 12:28:01.946192  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 141045us
I20260709 12:28:01.946497 13182 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.886834 13134 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:01.937402 13067 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:01.921275 13188 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:01.922093 13174 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.947237 12958 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:01.947027 12989 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:01.970507  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:01.971202 13254 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:01.971756 13254 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.971953 12954 consensus_queue.cc:1243] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:28:01.972184 13254 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:01.971366  6021 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:01.950783 13138 consensus_queue.cc:1243] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 23.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:28:01.890736 13253 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:01.955789 13173 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:28:01.991815 13253 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:01.993178 13269 raft_consensus.cc:3055] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:01.993742 13269 raft_consensus.cc:740] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:01.994634 13269 consensus_queue.cc:260] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:01.996093 13269 raft_consensus.cc:3060] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:01.998656 12315 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.999223 13246 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:01.999696 13246 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:01.999392 12315 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:02.000115 13246 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:02.002848 12315 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:01.913933 13260 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:01.957492 13067 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:02.006615  5840 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.196:34835 timed out after 1.532s (SENT) [suppressed 39 similar messages]
I20260709 12:28:01.947938 13136 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:02.071450  6018 raft_consensus.cc:2468] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 25.
I20260709 12:28:01.958423 13182 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:01.969905 13174 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:02.080170  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:02.082314  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:02.066634  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 130691us
W20260709 12:28:02.067431  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 186737us
W20260709 12:28:02.082649  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 199073us
W20260709 12:28:02.020865 13018 consensus_meta.cc:287] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.637s	user 0.016s	sys 0.000s
I20260709 12:28:01.990723 13138 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:02.097779  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 71082.8us
I20260709 12:28:01.948432 13176 consensus_queue.cc:237] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:02.006073 13156 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:02.103466  5947 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:28:02.104984  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.070086 13067 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:02.105435  5946 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:28:02.098896  5990 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:02.085988 13271 raft_consensus.cc:3055] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:28:02.116559  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 168341us
W20260709 12:28:02.115029  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.117388  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:02.118722  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:02.097199 12989 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:28:02.119913  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 181420us
W20260709 12:28:02.135751 13252 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.342s	user 0.002s	sys 0.000s
I20260709 12:28:02.142406  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:02.145748 13257 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 12:28:02.125926 12989 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:28:02.148319 13266 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
W20260709 12:28:02.150110 13267 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 12:28:02.100476 13018 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:02.120208 13271 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:02.113282 13138 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:02.156737 13134 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:02.128224 13188 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:02.120463 13156 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:02.126902 13276 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:28:02.097078 13273 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:28:02.146945 13268 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:02.136368  6023 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 12:28:02.276995  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 270383us
I20260709 12:28:02.278380  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1498) took 1345 ms. Trace:
I20260709 12:28:02.339991  5869 rpcz_store.cc:276] 0709 12:28:00.933287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.956199 (+ 22912us) service_pool.cc:224] Handling call
0709 12:28:01.226158 (+269959us) spinlock_profiling.cc:243] Waited 4.29 ms on lock 0x7b58002e2f98. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:28:02.164068 (+937910us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b54001be754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:02.278287 (+114219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1257924736}
I20260709 12:28:02.280035  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:28:02.282998  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 157762us
W20260709 12:28:02.297268  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 169821us
I20260709 12:28:02.292233  5940 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:28:02.304436  6021 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
W20260709 12:28:02.298362 13175 consensus_meta.cc:287] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.741s	user 0.015s	sys 0.010s
I20260709 12:28:02.285264 13280 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:02.275908 13283 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:02.314615  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:02.328598  6015 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:02.351393  6015 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:02.296870 12989 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:02.345641  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:02.312613 12783 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:28:02.308084 13221 consensus_meta.cc:287] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.818s	user 0.017s	sys 0.000s
I20260709 12:28:02.314615  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:28:02.309652 13115 consensus_meta.cc:287] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.754s	user 0.009s	sys 0.015s
I20260709 12:28:02.364799 13275 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:02.350060 13280 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.365121 13271 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:02.365908  6757 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:28:02.365774 13283 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:02.408540  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.323035 13188 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.411391  6785 raft_consensus.cc:2410] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:28:02.415160  6019 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 27.
W20260709 12:28:02.416014  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 48078.7us
I20260709 12:28:02.407526  6015 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:02.424196  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56305.8us
I20260709 12:28:02.426350  6015 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:02.426417  5915 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:02.427235  5840 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Timed out: RequestConsensusVote RPC to 127.2.4.196:34835 timed out after 1.628s (SENT)
I20260709 12:28:02.427742  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:02.430378 13271 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:02.430770  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 150912us
I20260709 12:28:02.431087 13247 raft_consensus.cc:476] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Not starting pre-election -- already a leader
I20260709 12:28:02.432302 13247 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:02.433893 13277 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:02.433755 13261 consensus_queue.cc:1048] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.092s
I20260709 12:28:02.434440 13277 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.434592 13261 consensus_queue.cc:1243] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:28:02.434823 13277 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:02.355585 13219 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:02.429306  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2251) took 1024 ms. Trace:
I20260709 12:28:02.357383 13134 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:02.445035  5867 rpcz_store.cc:276] 0709 12:28:01.396152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.408325 (+ 12173us) service_pool.cc:224] Handling call
0709 12:28:01.410256 (+  1931us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:01.410594 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:01.410634 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:01.445724 (+ 35090us) log.cc:844] Serialized 26 byte log entry
0709 12:28:01.580009 (+134285us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:01.580076 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.580193 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:02.419851 (+839658us) raft_consensus.cc:1685] finished
0709 12:28:02.419878 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.420070 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":133849,"threads_started":1,"wal-append.queue_time_us":246130}
I20260709 12:28:02.450042  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:02.453748  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:02.428843 13264 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:02.420218 13280 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:02.438488 12783 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:02.438087 13188 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:02.489514  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 63249.9us
I20260709 12:28:02.485157  6023 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 25: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:02.445657 13128 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:02.445005 13219 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:02.443944 13147 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:02.466333 13228 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:02.506237  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 62266.9us
I20260709 12:28:02.424798 13275 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:02.513860  5915 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:02.423322 13115 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:02.464529 13284 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:02.526142  6023 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:28:02.369441 13268 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:02.491247 13168 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:02.491886 13182 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:02.531065 13272 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.465413 13134 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:02.510098 13136 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:02.573302  6756 raft_consensus.cc:2410] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:28:02.516176 13219 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:02.538918  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:02.587972  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 160756us
I20260709 12:28:02.513813 13273 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:28:02.523916 13260 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:02.598438  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 264676us
I20260709 12:28:02.505887 13282 consensus_queue.cc:1059] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 12:28:02.575505 13138 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:02.593035 13108 consensus_meta.cc:287] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.748s	user 0.015s	sys 0.008s
W20260709 12:28:02.591864 12824 consensus_meta.cc:287] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.432s	user 0.021s	sys 0.000s
I20260709 12:28:02.550060 13128 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.585060 13272 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:02.533600 13147 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:02.606984 13105 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 1.119s	user 0.003s	sys 0.017s
W20260709 12:28:02.601812 13197 consensus_meta.cc:287] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.950s	user 0.016s	sys 0.003s
W20260709 12:28:02.627091  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.603081 13287 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:02.549970 13284 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:02.590695 13018 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:02.683491  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:02.610668 13176 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:02.562875 13168 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:02.718066  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:02.719244  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:02.617509  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 78716.2us
I20260709 12:28:02.738329  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:02.707527 13269 consensus_meta.cc:287] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.711s	user 0.012s	sys 0.011s
I20260709 12:28:02.733054  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:02.713197 13286 raft_consensus.cc:2804] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:28:02.734246  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 251124us
I20260709 12:28:02.626886 13108 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:02.650753 12824 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:02.730691 13261 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.726555 13156 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:02.749308  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:02.760788  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:02.761668  5873 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:02.770627  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:02.716859 13128 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:02.726981 13197 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:02.752506 13182 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.784301 13138 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.786196 13108 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:02.770143 12983 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:02.787333  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:02.798990 13301 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:02.798969 12983 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:02.800361 12983 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:02.801404  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 91312.6us
I20260709 12:28:02.802906  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:28:02.764631 13249 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:28:02.799439  6017 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 24: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:02.723174 13105 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:02.756955 13270 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.756800 13286 raft_consensus.cc:697] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:02.758929 13176 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:28:02.846413  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 100342us
I20260709 12:28:02.730882 13287 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:02.801837 13289 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:02.844350  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 82413.2us
W20260709 12:28:02.877980  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 75217.6us
I20260709 12:28:02.842398  6015 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:28:02.876632  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:28:02.835574  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 66604.2us
I20260709 12:28:02.887142  5866 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:02.880968  5944 raft_consensus.cc:2393] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 25: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:28:02.889139 13274 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.345s	user 0.000s	sys 0.002s
I20260709 12:28:02.889662  6768 raft_consensus.cc:1275] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 12:28:02.890110 13309 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:28:02.890342 13307 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 12:28:02.890586 13264 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:02.891434 13290 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:02.895264 13197 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:02.896148  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:02.839306 13276 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.961536 13276 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:02.962639 13067 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.842s	user 0.013s	sys 0.000s
I20260709 12:28:02.963111 13067 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:02.964067 13276 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:02.964147 13310 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.900559  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:02.965077  6018 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:28:02.964915 13310 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:02.967172 13300 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 12:28:02.969110 13310 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:02.969813  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 231599us
I20260709 12:28:02.970559  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 1407) took 1069 ms. Trace:
I20260709 12:28:02.970775  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:28:02.971118  5944 rpcz_store.cc:276] 0709 12:28:01.901262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.901425 (+   163us) service_pool.cc:224] Handling call
0709 12:28:02.970254 (+1068829us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b54004678d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:02.970474 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2123666816}
I20260709 12:28:02.798875 13278 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:02.972537  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:02.770747 13261 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:02.972985 13286 consensus_queue.cc:237] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 23.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:02.829126 13182 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:02.908546 13137 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.830605 13138 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:02.886302 13301 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:02.926244  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:02.982046  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 46019.8us
I20260709 12:28:02.927508  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:02.929278  6756 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
W20260709 12:28:02.909104 12958 consensus_meta.cc:287] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.919s	user 0.012s	sys 0.002s
I20260709 12:28:02.906775 13134 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:02.939428  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.880247 12954 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:28:02.945345  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:02.948565 13246 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.948s	user 0.017s	sys 0.000s
W20260709 12:28:02.948751 12989 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.529s	user 0.001s	sys 0.010s
I20260709 12:28:02.949044 13312 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:02.950111 13168 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:02.951447  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:02.953239 13290 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:02.977166 13136 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.955615 12824 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:02.959110  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:03.015789  6017 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:02.959686  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:02.960917 13284 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:02.897786 13174 consensus_meta.cc:287] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.796s	user 0.015s	sys 0.000s
W20260709 12:28:02.966114  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.966780  6024 pending_rounds.cc:85] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 9
I20260709 12:28:02.880487 13115 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:03.023146 13182 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:03.005904 13254 consensus_meta.cc:287] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.033s	user 0.013s	sys 0.003s
I20260709 12:28:03.024715  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.025103 13254 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:03.025470  5865 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:03.025782 13304 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.026429 13304 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:03.027400 13254 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:03.029000 13304 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:03.029415  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.029891 13128 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:03.031172 12783 consensus_meta.cc:287] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.003s
W20260709 12:28:03.005606 13253 consensus_meta.cc:287] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.010s	user 0.017s	sys 0.000s
W20260709 12:28:02.928853 12647 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 25 current term: 25 is higher.
I20260709 12:28:02.965266 13067 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:03.083050 13315 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:28:03.084225 13277 consensus_meta.cc:287] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.649s	user 0.003s	sys 0.012s
I20260709 12:28:03.003152 12989 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:03.084661 13277 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:03.085489  6023 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:03.086287 13289 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:02.977245  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:02.979380  5873 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 29.
I20260709 12:28:03.088915  6762 raft_consensus.cc:3060] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:02.914624 13275 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:02.933736 13271 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.005s
I20260709 12:28:02.957329 13272 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:02.988211 13147 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:02.881278 13270 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:02.995743  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:02.998363  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:02.978025 13273 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:03.005293  6778 raft_consensus.cc:2368] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19.
I20260709 12:28:02.933542 13268 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:02.954897 13105 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:02.982669 13175 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:03.008841  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.012919  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.014389  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:02.985486 13301 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:03.019116  6024 pending_rounds.cc:107] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:28:03.021324  6756 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:28:03.001830 13307 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.121523  6024 raft_consensus.cc:2889] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:03.022804 13313 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:02.908354  5871 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:28:03.023437 13290 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:02.990571 12958 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:03.138943  5868 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:03.033044  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.035154 13138 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:02.973121 13278 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:03.040593 13247 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.608s	user 0.006s	sys 0.004s
I20260709 12:28:02.984838 13137 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:03.001221 13246 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:02.972558 13176 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:02.982026 13264 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:28:02.975026  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:28:03.045682  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 54694.6us
I20260709 12:28:03.078029  6015 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 25.
I20260709 12:28:03.049883  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:03.089777  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.050619  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:03.049857 13174 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:03.175091  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:03.095409 13277 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:03.189476  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:03.031706 12783 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:03.029618 13293 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.324s	user 0.004s	sys 0.000s
I20260709 12:28:03.119871 13272 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:03.021996  5838 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:03.137427  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:03.129317 13175 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:28:03.125938 13228 consensus_meta.cc:287] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.599s	user 0.005s	sys 0.008s
I20260709 12:28:03.127300 13301 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:03.043278 13261 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:03.218820  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.172835  6793 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:28:03.174145  6762 raft_consensus.cc:2468] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 29.
W20260709 12:28:03.201027  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 179086us
I20260709 12:28:03.182708  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:03.184089  6766 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:03.183476  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:28:03.190562  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 140051us
I20260709 12:28:03.221961  6775 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:03.191208  6021 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 25: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:03.119001  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.193106  6018 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 25: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:03.018922 13136 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:03.199868  6757 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:28:03.144364  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:03.265800  6756 raft_consensus.cc:3060] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:03.254927  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 117617us
I20260709 12:28:03.255057  5869 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:28:03.222376  5838 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:03.178686 13246 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:03.263397  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.176975 13247 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:03.279470 13174 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:03.223265  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:03.198529 13290 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:03.280308 13136 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:03.223129 13264 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:03.282323  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.283005 12958 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:03.176877 13278 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:03.176738 13137 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:03.288306  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:03.290575  6017 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
W20260709 12:28:03.281392 13318 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:28:03.293504  5913 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:03.251804 13175 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:03.184742 12983 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:03.302438  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 119099us
I20260709 12:28:03.132856 13307 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:03.300747  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:28:03.302417  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 79219.9us
I20260709 12:28:03.303620  6757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.268157 13261 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:03.319624  6021 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 19: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:28:03.191545 13280 consensus_meta.cc:287] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.676s	user 0.011s	sys 0.010s
I20260709 12:28:03.280386 13258 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:28:03.310096  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 87809.6us
W20260709 12:28:03.296801 13293 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:28:03.312678  6020 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:03.313515  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.224561 13272 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:03.314363  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.311936 13287 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:03.346959  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.350554 13276 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:03.345038 13261 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:03.327550  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.377226  6762 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 22: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:28:03.377310  6023 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:03.307957 13286 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:03.310362  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:03.347983 13137 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:03.329977  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2258) took 1451 ms. Trace:
W20260709 12:28:03.350353  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 405119us
I20260709 12:28:03.307639 13264 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:03.302006 13290 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:03.306458 13282 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:03.384394  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.347329  6609 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:03.357271  6022 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
I20260709 12:28:03.351473 12983 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:03.397168 13247 consensus_queue.cc:1048] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:03.398139 13247 consensus_queue.cc:1243] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:28:03.397840 13276 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:28:03.399603 13322 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.051s	user 0.002s	sys 0.000s
I20260709 12:28:03.400665  6773 raft_consensus.cc:3055] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:03.401171  6773 raft_consensus.cc:740] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:03.348855 13307 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:03.359342  5915 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:03.402139  6773 consensus_queue.cc:260] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:03.402544  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 82330us
I20260709 12:28:03.403784 13272 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:03.349475  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1773) took 1062 ms. Trace:
I20260709 12:28:03.406055  6773 raft_consensus.cc:3060] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:03.383004 13188 consensus_meta.cc:287] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.868s	user 0.004s	sys 0.009s
I20260709 12:28:03.407055 13188 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:03.352937 13270 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:03.367995  6019 raft_consensus.cc:2410] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:28:03.365877  5913 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:28:03.367211  6766 raft_consensus.cc:2468] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 27.
I20260709 12:28:03.380857 13176 raft_consensus.cc:2804] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:03.385012  5866 rpcz_store.cc:276] 0709 12:28:01.878143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.878344 (+   201us) service_pool.cc:224] Handling call
0709 12:28:01.892165 (+ 13821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.916566 (+1024401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.329586 (+413020us) spinlock_profiling.cc:243] Waited 995 ms on lock 0x7b54001bd354. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:03.329870 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2287592832}
I20260709 12:28:03.309244 13173 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:03.419713  5943 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:28:03.420063  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:03.420935  6019 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 19: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:03.421015  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:03.374527 13280 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:03.388149  5840 leader_election.cc:336] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Timed out: RequestConsensusVote RPC to 127.2.4.196:34835 timed out after 2.458s (SENT)
I20260709 12:28:03.382759 13261 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:03.388664 13328 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 12:28:03.381950 13219 consensus_meta.cc:287] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.779s	user 0.010s	sys 0.003s
I20260709 12:28:03.390268  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.438144  6017 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:28:03.382143 13271 consensus_queue.cc:1048] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:28:03.406270 13115 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:03.365099  5865 raft_consensus.cc:1240] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:03.439512  5941 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:28:03.440163  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:03.409286 13188 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:03.386202  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:03.407078 13323 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:28:03.321862 12989 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:03.413745  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:03.413771  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:28:03.414639  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 48855.3us
I20260709 12:28:03.448763  5947 raft_consensus.cc:2410] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:28:03.417066  5946 raft_consensus.cc:1275] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:28:03.405830  6016 rpcz_store.cc:276] 0709 12:28:02.286929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.287054 (+   125us) service_pool.cc:224] Handling call
0709 12:28:02.322846 (+ 35792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.323176 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:02.368327 (+ 45151us) log.cc:844] Serialized 14 byte log entry
0709 12:28:02.680969 (+312642us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b18002d3e80. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:28:02.991023 (+310054us) log.cc:844] Serialized 14 byte log entry
0709 12:28:03.348840 (+357817us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b18002d3e80. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb10c0a 00007f39d9250219 00007f39cfb98a40 00007f39cfb934e4 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287 00007f39d2c9e208
0709 12:28:03.349074 (+   234us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:03.349124 (+    50us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:03.349180 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.349264 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.349404 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":345654,"prepare.queue_time_us":33724,"prepare.run_cpu_time_us":170,"prepare.run_wall_time_us":9346,"raft.queue_time_us":359202,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":3396,"spinlock_wait_cycles":44637312,"thread_start_us":309161,"threads_started":1,"wal-append.queue_time_us":732470}
I20260709 12:28:03.419116  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.450044  6015 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 25: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:03.387143  6021 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
I20260709 12:28:03.422361  6775 raft_consensus.cc:2468] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 25.
I20260709 12:28:03.424431  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:03.430610  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:03.451135  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.451927 13228 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:03.452260  5867 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:28:03.383358 13286 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:03.435165  5944 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:28:03.453946 13286 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:03.435725  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:03.454900  5940 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:28:03.401217 13175 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:03.456691 13175 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:03.412343  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.410827 13290 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:03.444180  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 56076.9us
I20260709 12:28:03.445905  6766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:03.412879 13264 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:03.459259  5913 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:03.465371  6756 raft_consensus.cc:2468] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 25.
I20260709 12:28:03.467550  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.468304  6775 raft_consensus.cc:2393] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 26: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:28:03.424273 13270 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:03.468555  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:03.469453  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.471871  5949 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:28:03.473106  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.423523 13282 raft_consensus.cc:697] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:03.476806  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:03.478021  6021 raft_consensus.cc:2393] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 25: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:03.478248  6756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.447703 13173 raft_consensus.cc:697] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:03.520880  5915 leader_election.cc:400] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19.
I20260709 12:28:03.521070  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:28:03.524943  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 63990.2us
I20260709 12:28:03.525825  5915 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:03.440305 13176 raft_consensus.cc:697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 29 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:03.529829  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:03.522141  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 44406.5us
W20260709 12:28:03.472841 13221 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.011s	sys 0.010s
I20260709 12:28:03.466183 13324 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.447482 13272 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.526336  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:03.471290 13197 consensus_queue.cc:1059] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 12:28:03.523432 13269 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:28:03.545614 13176 consensus_queue.cc:237] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:03.478385 13108 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.549973 13108 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:03.526798 13277 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:28:03.527349  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.441586 13219 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:03.447554 13312 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.528713  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.530695  6017 raft_consensus.cc:2410] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:28:03.444796 13105 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.473272 13304 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:03.534518  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:03.475278 13175 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.536733  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:03.576978  6021 raft_consensus.cc:2368] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 25. Current term is 26.
I20260709 12:28:03.538556  5867 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:03.539502 13321 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.539995  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:03.540505 13173 consensus_queue.cc:237] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:03.543733 13288 log.cc:927] Time spent T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.794s	user 0.005s	sys 0.000s
I20260709 12:28:03.558861  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:03.527503  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 50016.8us
I20260709 12:28:03.558251  6023 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 25.
I20260709 12:28:03.520563 13287 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.568320  6020 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
W20260709 12:28:03.535115 13221 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.011s	sys 0.010s
I20260709 12:28:03.602176  6757 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 22: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:28:03.573855 13105 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:03.569739 12989 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.530318 13270 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:03.537917 13115 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:03.540030 13306 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:03.599462  5866 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:28:03.384817 13258 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:03.635362 13258 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:03.635552 13286 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:03.636389  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:03.585305 13134 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:28:03.637061  5944 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:28:03.569885 13324 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:03.581800 13312 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:03.590945  6015 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:28:03.640918 13271 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:28:03.646919  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.602449 13276 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:03.647819  6015 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
W20260709 12:28:03.648357 13302 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.916s	user 0.000s	sys 0.002s
I20260709 12:28:03.647899 13276 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:03.640558  6773 raft_consensus.cc:1275] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:28:03.622311 13326 log.cc:927] Time spent T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:28:03.607213  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.653854  5869 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 23: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:03.616919 12989 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:03.656131 12989 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:03.602286 13287 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:03.619558 13105 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:03.639340 13219 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:03.658339 12783 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:03.658922 13287 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:03.664383 13283 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:03.622141  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.666075  6017 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:03.667232  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.668229  6017 raft_consensus.cc:2410] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 26 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:28:03.667563 13286 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:03.583925  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 49453.9us
I20260709 12:28:03.533080 13313 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:28:03.669381 13108 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:03.670477  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.555963 13282 consensus_queue.cc:237] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:03.616396 13306 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:03.586947 13175 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:03.631660  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.718057  6016 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
I20260709 12:28:03.608191 13304 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:03.563508 13278 consensus_queue.cc:1059] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:28:03.562649 13272 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:03.647315 13271 consensus_queue.cc:1243] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:28:03.622543 13280 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:03.649787 12909 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
W20260709 12:28:03.637115  5987 connection.cc:256] RPC call timeout handler was delayed by -1.907271228999889s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19130000000000003s for a response
I20260709 12:28:03.731101  6756 raft_consensus.cc:2410] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 20 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
W20260709 12:28:03.731969  5987 connection.cc:256] RPC call timeout handler was delayed by -1.8537064480001391s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24740000000000004s for a response
W20260709 12:28:03.732434  5987 connection.cc:256] RPC call timeout handler was delayed by -1.7799064480000197s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2556s for a response
I20260709 12:28:03.650344 13276 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:03.626255 13270 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:03.588339 13321 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:03.650807 13324 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:03.639832  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:03.652810 13312 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:03.678452  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1699) took 1127 ms. Trace:
I20260709 12:28:03.679080  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:03.743358  5945 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:03.684247  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.614692  5872 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:28:03.687377 13290 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:03.688980  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.689198  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1663) took 1126 ms. Trace:
I20260709 12:28:03.742136  6024 rpcz_store.cc:276] 0709 12:28:02.550768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.960661 (+409893us) service_pool.cc:224] Handling call
0709 12:28:02.961069 (+   408us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:03.351648 (+390579us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:03.351699 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:03.352522 (+   823us) log.cc:844] Serialized 52 byte log entry
0709 12:28:03.379442 (+ 26920us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:03.379507 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.677886 (+298379us) spinlock_profiling.cc:243] Waited 5.39 ms on lock 0x7b54001720d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:03.677934 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:03.677983 (+    49us) raft_consensus.cc:1685] finished
0709 12:28:03.677997 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.678371 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":228930,"raft.queue_time_us":229551,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":145,"spinlock_wait_cycles":12395776,"thread_start_us":26405,"threads_started":1,"wal-append.queue_time_us":78913}
I20260709 12:28:03.756297  6022 rpcz_store.cc:276] 0709 12:28:02.563038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.905902 (+342864us) service_pool.cc:224] Handling call
0709 12:28:03.689013 (+783111us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b5400364cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:03.689129 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":862321408}
I20260709 12:28:03.678999  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1665) took 1062 ms. Trace:
I20260709 12:28:03.689845 13134 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:28:03.691088  6609 leader_election.cc:336] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.629s (SENT)
I20260709 12:28:03.695778  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.700465  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:03.760047  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:03.760697  5940 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:03.702279  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:03.704070  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:03.704409  5867 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 26.
I20260709 12:28:03.763525  5867 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:03.705755  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:03.764667  5947 raft_consensus.cc:2368] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23.
I20260709 12:28:03.674324 12989 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.648924  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:03.768110  5946 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:03.768378  6015 raft_consensus.cc:2376] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21. Re-sending same reply.
I20260709 12:28:03.708943  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.715714 13306 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:03.724366  5915 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:28:03.589318  5840 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:28:03.772716  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 183474us
I20260709 12:28:03.590493  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:03.732924  5987 connection.cc:256] RPC call timeout handler was delayed by -1.2448712289997275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2649s for a response
I20260709 12:28:03.738775  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.739975  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.757277  6017 rpcz_store.cc:276] 0709 12:28:02.616204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.676412 (+1060208us) service_pool.cc:224] Handling call
0709 12:28:03.678386 (+  1974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:03.678663 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:03.678690 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:03.678732 (+    42us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:03.678761 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.678820 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.678918 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:03.767541  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:03.769798  5867 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
W20260709 12:28:03.779488  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 55509.9us
I20260709 12:28:03.782519  6793 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:03.782119  6020 raft_consensus.cc:2410] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:28:03.783246  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.648945 13301 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:03.783948  5867 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:28:03.730643 13286 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:28:03.771878  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:03.789512  5948 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:03.789414  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.785395  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.786959  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.783120  5915 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:03.792572  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:03.791080  6778 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 26.
I20260709 12:28:03.796545  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:03.796968  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.797610  5865 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 25.
I20260709 12:28:03.798579  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:03.799556  5915 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:03.799635  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.801239  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.765859 13175 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:03.778481 13312 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:03.788086 13271 raft_consensus.cc:2804] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:03.821072  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:03.674057 13264 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:03.817862  6019 raft_consensus.cc:2393] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 23: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:03.805480  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:03.821803  6775 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:28:03.822278  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.789026 13278 raft_consensus.cc:2804] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:03.815279  5915 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:03.823822  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:28:03.845510  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 154444us
I20260709 12:28:03.846858  5867 raft_consensus.cc:2393] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 19: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:03.845688  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.847786  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:03.848136  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:03.851541  6773 raft_consensus.cc:2410] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:28:03.813496  5940 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:28:03.852257  5947 raft_consensus.cc:2468] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:28:03.832532  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
W20260709 12:28:03.862360  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 58024.7us
I20260709 12:28:03.863368  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.866550  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:03.867969  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:03.875948  5865 raft_consensus.cc:2368] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 24. Current term is 25.
I20260709 12:28:03.876165  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:03.877449  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.887081  6775 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:28:03.815904 13018 raft_consensus.cc:2804] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:03.844324 13313 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.798179 12989 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:03.893005  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:03.824796 13115 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:03.824867 13136 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.901245  6785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:03.902029  6785 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:03.895542  6016 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 19.
I20260709 12:28:03.847062 13271 raft_consensus.cc:697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 27 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:03.898058  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:03.896612  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.820607 13261 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:03.907299  6768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1459) took 1607 ms. Trace:
I20260709 12:28:03.908560  6775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:03.891674 13260 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:03.816802 13321 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:03.913029  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.896380  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
I20260709 12:28:03.890237 13280 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:03.896490  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:03.919309  5947 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
I20260709 12:28:03.907776  6768 rpcz_store.cc:276] 0709 12:28:02.299933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.300108 (+   175us) service_pool.cc:224] Handling call
0709 12:28:02.315388 (+ 15280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.918625 (+603237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.905008 (+986383us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b5400470c54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:03.907216 (+  2208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1005837696}
I20260709 12:28:03.917428  6778 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 25: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:03.910437  6775 raft_consensus.cc:2368] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19.
I20260709 12:28:03.902613 13108 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.922119  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:03.902184 13018 raft_consensus.cc:697] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:03.899798 13115 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:03.925900  6016 raft_consensus.cc:3060] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:03.928939  6773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:03.931375  6773 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:28:03.932513  6773 pending_rounds.cc:85] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Aborting all ops after (but not including) 13
I20260709 12:28:03.932705  6773 pending_rounds.cc:107] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:28:03.932852  6773 raft_consensus.cc:2889] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:03.926275  5942 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:28:03.934312  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:03.938234  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:03.946605  6778 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:28:03.949195  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:03.950279 13105 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:03.951129 13105 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:03.951372  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:03.952603 13105 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:03.953110  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:03.960690  6775 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 12:28:03.906910 12824 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:28:03.965605  6785 raft_consensus.cc:2468] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 26.
I20260709 12:28:03.934892 13115 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:03.902994 13313 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:04.016258  5872 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:28:03.972033 13105 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:04.018080  6775 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:04.020846  6775 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 26. Current term is 27. Ops: [25.13-26.14]
I20260709 12:28:03.973016 12989 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:03.916162 13136 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:04.022703 13168 consensus_queue.cc:1059] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:04.024240 13136 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:04.024389 13168 raft_consensus.cc:3055] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:04.024645  5946 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
I20260709 12:28:04.025049 13168 raft_consensus.cc:740] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:04.025714  5945 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:28:04.026437  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:04.026199 13168 consensus_queue.cc:260] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:03.929718 13108 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:04.027989  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:04.028666  5873 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:28:04.029249 13108 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:04.030331  5874 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:28:04.031935  6016 raft_consensus.cc:2468] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:28:04.031968  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:04.033000  5869 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:28:03.982599 13301 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:03.983872 13264 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:03.938547 13294 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:03.984804 13290 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:03.986564  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:04.056209  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 103169us
I20260709 12:28:04.056811  6019 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:28:04.056727 13294 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:04.057756  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:04.058427  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:04.059334  6019 raft_consensus.cc:2368] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19.
I20260709 12:28:04.059495  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:03.992071 13261 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:04.060524 13301 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:04.060631  6019 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:04.062680  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:04.061770  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1776) took 1226 ms. Trace:
I20260709 12:28:04.063107 12315 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:04.063486  6023 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:28:04.063608  6019 rpcz_store.cc:276] 0709 12:28:02.835121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.060026 (+1224905us) service_pool.cc:224] Handling call
0709 12:28:04.060426 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.061439 (+  1013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.061708 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:03.990078  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:04.065606  6017 raft_consensus.cc:2410] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 26 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:28:03.993451  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:04.070519  5873 raft_consensus.cc:1240] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:03.990094 13018 consensus_queue.cc:237] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:04.072369 13264 raft_consensus.cc:2764] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:28:03.996878  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1664) took 1408 ms. Trace:
I20260709 12:28:04.000957  6762 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22. Ops: []
W20260709 12:28:04.002054  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 55556.9us
I20260709 12:28:03.925948 13280 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:04.002576  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:04.003682  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:04.008834 12983 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:28:04.011299  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 368315us
I20260709 12:28:04.012374  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:04.015021  5865 raft_consensus.cc:1240] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:04.016927 13313 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:03.935729 13276 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:03.979539 13272 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:04.026844  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:03.980162 13271 consensus_queue.cc:237] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:04.034191  5948 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:28:04.035538  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:04.083592  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:04.032231  6793 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
W20260709 12:28:04.050740  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 200408us
I20260709 12:28:03.988256  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:04.095044  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:04.061347 13301 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:04.001876 12647 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.074451  6021 rpcz_store.cc:276] 0709 12:28:02.588873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.675017 (+1086144us) service_pool.cc:224] Handling call
0709 12:28:03.675426 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:03.675702 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:03.675733 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:03.676289 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 12:28:03.877331 (+201042us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:03.877400 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.877521 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:03.996613 (+119092us) raft_consensus.cc:1685] finished
0709 12:28:03.996638 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.996808 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":200504,"threads_started":1,"wal-append.queue_time_us":316216}
I20260709 12:28:04.078033  6015 raft_consensus.cc:2368] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23.
I20260709 12:28:04.077986  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:04.069387  6024 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:04.078791  5944 raft_consensus.cc:3060] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:04.079504  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1779) took 1115 ms. Trace:
I20260709 12:28:04.079514  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:04.088678  5838 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:04.095721  6020 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:28:04.108680  6762 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:04.059725 13294 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:04.097220  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:04.095505  5915 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:04.124142  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1701) took 1132 ms. Trace:
I20260709 12:28:04.124771  6024 rpcz_store.cc:276] 0709 12:28:02.991842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.068667 (+1076825us) service_pool.cc:224] Handling call
0709 12:28:04.069113 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.123735 (+ 54622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.124052 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:04.127619  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:04.104887  5871 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:04.076548 13278 raft_consensus.cc:697] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:04.087319 13276 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:04.112433 12647 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:04.114727  5840 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:04.115495 13182 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:04.080161 13221 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:04.168655  6015 raft_consensus.cc:1240] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:04.109984 13105 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:04.083786 13168 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:04.130174  6762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af"
W20260709 12:28:04.131202  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 284284us
I20260709 12:28:04.087723 13173 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:04.108520  6019 rpcz_store.cc:276] 0709 12:28:02.960821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.065027 (+1104206us) service_pool.cc:224] Handling call
0709 12:28:04.065337 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.065518 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:04.065541 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:04.065585 (+    44us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:04.065617 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.076209 (+ 10592us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.076392 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:04.153051  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:04.187384  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 107968us
I20260709 12:28:04.187922  5945 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:28:04.188555  6015 raft_consensus.cc:1275] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:28:04.075582 13115 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:04.157614  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 62195.2us
I20260709 12:28:04.159852  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:04.073468 13306 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:04.080605 12983 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:04.090575 13264 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 12:28:04.215811  6019 raft_consensus.cc:1275] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:28:04.194448  6793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:04.196067  5915 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:04.095228 13313 consensus_queue.cc:1048] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 12:28:04.160578 13276 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:04.161933 12783 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:04.228425  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:04.167879 13294 raft_consensus.cc:2764] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:28:04.233095  5947 raft_consensus.cc:2368] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20.
I20260709 12:28:04.192845  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:28:04.154349 13108 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:04.157303 13182 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:04.222961  6775 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:04.226086  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:04.187516 13105 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:04.230052  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:04.168320 13260 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.153833 12824 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 12:28:04.190996 13261 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:04.214941 13321 consensus_queue.cc:1059] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:28:04.237596  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:04.229334  5945 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:28:04.234695 13348 raft_consensus.cc:2804] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:04.188143 13221 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:04.257108  5915 leader_election.cc:400] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19.
I20260709 12:28:04.247998 13105 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:04.243065 13182 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:04.260133  6611 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:04.265460  5915 leader_election.cc:403] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 12:28:04.236864  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 44127.9us
I20260709 12:28:04.249202 12783 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:04.252333 13197 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:04.277336  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:04.272688  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:04.253655 13247 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:04.262864 12874 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.494s	user 0.015s	sys 0.012s
I20260709 12:28:04.278988  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:04.257594 13260 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:04.311280  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:04.319721  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:28:04.295161 12874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.532s	user 0.015s	sys 0.012s
I20260709 12:28:04.272089 13348 raft_consensus.cc:697] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:04.294731 13261 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19.
I20260709 12:28:04.328949  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:04.279103 13197 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:04.332209  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:04.299410 13018 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:04.338011  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:04.338001  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 65388.9us
I20260709 12:28:04.297863 13312 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:04.279258 13294 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:28:04.358470  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:04.366981  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:04.369917 13247 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:04.370620 13247 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:04.371098 13247 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:04.376845  5948 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 20.
I20260709 12:28:04.327296 13321 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:04.379292  6609 leader_election.cc:400] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23.
I20260709 12:28:04.256842 13278 consensus_queue.cc:237] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:04.390313  5942 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 24: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:04.388559  6609 leader_election.cc:403] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:04.337555 13197 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:04.320429 12989 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:28:04.400023  5943 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:04.346946 13280 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:04.413363  5948 raft_consensus.cc:1240] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 24. Current term is 25. Ops: []
W20260709 12:28:04.411583 13345 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.170s	user 0.000s	sys 0.002s
W20260709 12:28:04.442803  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 328151us
I20260709 12:28:04.403105 12647 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:04.428727 13115 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:04.539741 13282 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:04.649932 13282 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:04.650549 13282 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:04.652621  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 311578us
W20260709 12:28:04.542292 13353 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
W20260709 12:28:04.543555 13356 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:28:04.428118 13306 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:04.537786 13348 consensus_queue.cc:237] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:04.500197 13156 raft_consensus.cc:3055] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:28:04.630280  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 370213us
I20260709 12:28:04.684978  6611 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:04.662508  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:04.546641 13321 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:04.685137  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:28:04.698773  5944 consensus_meta.cc:287] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.016s
I20260709 12:28:04.699596  5944 raft_consensus.cc:2468] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
I20260709 12:28:04.704227  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:04.705695  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:04.711906  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:04.697815 13351 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.322s	user 0.000s	sys 0.009s
I20260709 12:28:04.722108  6609 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:28:04.706557 13350 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.303s	user 0.003s	sys 0.006s
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:493: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
I20260709 12:28:04.693284 13156 raft_consensus.cc:740] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:04.729107  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:28:04.731509  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 420301us
I20260709 12:28:04.742445  6778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1890) took 1051 ms. Trace:
I20260709 12:28:04.713454 12824 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:04.713461 13307 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:04.743029  6778 rpcz_store.cc:276] 0709 12:28:03.691484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.962558 (+271074us) service_pool.cc:224] Handling call
0709 12:28:04.034282 (+ 71724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.052190 (+ 17908us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:04.073503 (+ 21313us) log.cc:844] Serialized 14 byte log entry
0709 12:28:04.741992 (+668489us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:04.742055 (+    63us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:04.742129 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.742211 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.742375 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18196,"prepare.queue_time_us":21699,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"thread_start_us":666313,"threads_started":2}
I20260709 12:28:04.721472 13221 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:04.726392 13173 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:04.718286 13373 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23.
I20260709 12:28:04.726889 13260 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:04.735658 13264 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:04.808386  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2192) took 1358 ms. Trace:
I20260709 12:28:04.721267 13182 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:04.814877  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:04.783156 12954 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:04.819561  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 195430us
I20260709 12:28:04.807922  2067 tablet_server.cc:179] TabletServer@127.2.4.193:38701 shutting down...
I20260709 12:28:04.808817  5941 rpcz_store.cc:276] 0709 12:28:03.449842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.535380 (+ 85538us) service_pool.cc:224] Handling call
0709 12:28:04.029904 (+494524us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b5800030398. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:28:04.030105 (+   201us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:04.030480 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:04.030504 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:04.031019 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 12:28:04.220243 (+189224us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:04.220290 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.220373 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:04.793604 (+573231us) raft_consensus.cc:1685] finished
0709 12:28:04.793743 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.808305 (+ 14562us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":451,"raft.queue_time_us":4221,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":2546,"spinlock_wait_cycles":50061056,"thread_start_us":188681,"threads_started":1,"wal-append.queue_time_us":325405}
I20260709 12:28:04.796480 13173 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:04.837251  6778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1fc57407d56241b38a94d543e073f8af" is_pre_election: true
I20260709 12:28:04.822324 13377 raft_consensus.cc:2804] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:28:04.845448 13352 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.145s	user 0.001s	sys 0.001s
I20260709 12:28:04.840108 13371 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:04.850993 13174 raft_consensus.cc:3055] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:28:04.814571 13350 log.cc:927] Time spent T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 12:28:04.861389 13375 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:28:04.764235 13321 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:04.848178 13264 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:04.856292 13173 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:04.872299  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 150346us
I20260709 12:28:04.900856  6609 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:28:04.845199 13271 consensus_queue.cc:1048] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:28:04.840552 13272 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:28:04.862854 13377 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:04.903620  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:28:04.905270  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 90513us
I20260709 12:28:04.898298  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1786) took 1215 ms. Trace:
I20260709 12:28:04.855731 13373 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:04.873275 13174 raft_consensus.cc:740] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:04.841549 13156 consensus_queue.cc:260] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:04.907868  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2191) took 1476 ms. Trace:
W20260709 12:28:04.904341 13105 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.623s	user 0.014s	sys 0.000s
I20260709 12:28:04.909578  6015 rpcz_store.cc:276] 0709 12:28:03.682849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.667840 (+984991us) service_pool.cc:224] Handling call
0709 12:28:04.684352 (+ 16512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.684705 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:04.706849 (+ 22144us) log.cc:844] Serialized 14 byte log entry
0709 12:28:04.897891 (+191042us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:04.897955 (+    64us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:04.898004 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.898081 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.898219 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28193,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":177,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":4609,"thread_start_us":178872,"threads_started":1}
I20260709 12:28:04.916189  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:28:04.909487 13168 consensus_meta.cc:287] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.717s	user 0.014s	sys 0.005s
I20260709 12:28:04.818889 13307 consensus_queue.cc:1048] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 12:28:04.922910  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:04.909273 12783 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.630s	user 0.009s	sys 0.009s
I20260709 12:28:04.922149  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1788) took 1238 ms. Trace:
I20260709 12:28:04.925859 13156 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:04.926435 13174 consensus_queue.cc:260] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:04.911154  5949 rpcz_store.cc:276] 0709 12:28:03.431122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.524027 (+ 92905us) service_pool.cc:224] Handling call
0709 12:28:03.982239 (+458212us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b5800030398. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:28:03.982446 (+   207us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:28:03.982944 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:03.982973 (+    29us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:28:03.983669 (+   696us) log.cc:844] Serialized 78 byte log entry
0709 12:28:04.198094 (+214425us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:04.456009 (+257915us) log.cc:844] Serialized 14 byte log entry
0709 12:28:04.632973 (+176964us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.633080 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:04.903735 (+270655us) raft_consensus.cc:1685] finished
0709 12:28:04.907540 (+  3805us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.907786 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":166758,"prepare.queue_time_us":673059,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":193018,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":245,"spinlock_wait_cycles":52147968,"thread_start_us":441752,"threads_started":2,"wal-append.queue_time_us":317365}
I20260709 12:28:04.934404 13382 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:04.936254  6611 leader_election.cc:400] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23.
I20260709 12:28:04.923110 13105 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:04.912870 13368 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:28:04.922065 13197 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.506s	user 0.009s	sys 0.008s
I20260709 12:28:04.941187  6015 rpcz_store.cc:276] 0709 12:28:03.683349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.921050 (+1237701us) service_pool.cc:224] Handling call
0709 12:28:04.921481 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.921730 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:04.921768 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:04.921815 (+    47us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:04.921848 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.921926 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.922070 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:04.940760  6609 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:28:04.938015  5915 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:04.914332 13374 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:04.971218 13354 log.cc:927] Time spent T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
W20260709 12:28:04.947955 13359 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 12:28:04.975099  5869 raft_consensus.cc:1240] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:04.975535  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:04.973533 12783 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:04.946892 13373 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:04.978142 13168 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:04.953545 13382 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:04.972863 13197 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:05.008984 13367 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:28:04.923929 12954 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.057s
W20260709 12:28:05.030438 13351 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:28:04.999384 13105 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:05.014048  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2196) took 1379 ms. Trace:
I20260709 12:28:05.050372  5944 rpcz_store.cc:276] 0709 12:28:03.634992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.707148 (+1072156us) service_pool.cc:224] Handling call
0709 12:28:04.708119 (+   971us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:04.726732 (+ 18613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:04.726786 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:04.758506 (+ 31720us) log.cc:844] Serialized 26 byte log entry
0709 12:28:04.902533 (+144027us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:04.902590 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.902664 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:05.013843 (+111179us) raft_consensus.cc:1685] finished
0709 12:28:05.013863 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.013982 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8935,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":10220,"thread_start_us":79360,"threads_started":1,"wal-append.queue_time_us":215148}
I20260709 12:28:04.949987 13307 consensus_queue.cc:1243] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer 1fc57407d56241b38a94d543e073f8af log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:28:05.017174  6773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1889) took 1326 ms. Trace:
W20260709 12:28:05.019678  6018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37100 (request call id 1700) took 2033 ms (client timeout 1809 ms). Trace:
I20260709 12:28:04.922205 13369 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:05.019958 13282 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:04.997960  5690 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40812 (request call id 144) took 65890 ms (client timeout 15000 ms). Trace:
W20260709 12:28:04.954365 13247 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.583s	user 0.015s	sys 0.000s
W20260709 12:28:05.001794 13364 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 12:28:05.002797 13182 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:05.001073 12983 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:04.912705 13377 raft_consensus.cc:3060] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:05.021773  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 81127us
W20260709 12:28:04.952505 13349 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
W20260709 12:28:04.981016  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 58201.9us
I20260709 12:28:05.071808  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:04.979344  5949 raft_consensus.cc:1240] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:05.027661  6015 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
I20260709 12:28:05.021392 13374 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:04.975032 13313 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:28:05.055295  6018 rpcz_store.cc:269] 0709 12:28:02.986399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.000280 (+ 13881us) service_pool.cc:224] Handling call
0709 12:28:05.019388 (+2019108us) spinlock_profiling.cc:243] Waited 71.5 ms on lock 0x7b5400160b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:05.019556 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164389760}
I20260709 12:28:05.054162  6773 rpcz_store.cc:276] 0709 12:28:03.691167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.931910 (+240743us) service_pool.cc:224] Handling call
0709 12:28:03.932284 (+   374us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:03.936807 (+  4523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:03.936861 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:03.937691 (+   830us) log.cc:844] Serialized 52 byte log entry
0709 12:28:04.294536 (+356845us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:04.294599 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.294691 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:05.016920 (+722229us) raft_consensus.cc:1685] finished
0709 12:28:05.016944 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.017107 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":318,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":18437,"thread_start_us":307874,"threads_started":1,"wal-append.queue_time_us":876606}
I20260709 12:28:05.058389  5948 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:05.043905 13168 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.081694  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:05.082556  6018 raft_consensus.cc:2393] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 25: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:05.083163 12909 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:05.083482  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:05.084187  6018 raft_consensus.cc:2393] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 24: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:05.084971  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:05.043278 13382 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:05.085526  6018 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:05.064790 13369 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:05.044937 13383 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:05.032954 13286 consensus_queue.cc:1059] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:28:05.074893  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:04.979554 13174 raft_consensus.cc:3060] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:05.078646  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1807) took 1228 ms. Trace:
I20260709 12:28:05.079263  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1785) took 1396 ms. Trace:
I20260709 12:28:05.081717 13168 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:05.094188 13276 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.858s	user 0.014s	sys 0.000s
I20260709 12:28:05.092023  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:05.062592  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:05.094686  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1777) took 2259 ms. Trace:
I20260709 12:28:05.100751  5949 rpcz_store.cc:276] 0709 12:28:03.850472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.949049 (+1098577us) service_pool.cc:224] Handling call
0709 12:28:04.979108 (+ 30059us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.078365 (+ 99257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.078594 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:05.066551  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 116150us
I20260709 12:28:05.102576  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:05.085425 13197 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:05.078127 12954 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:05.067742 13247 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:05.110939  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:05.117340  6016 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 27.
I20260709 12:28:05.100862  6019 rpcz_store.cc:276] 0709 12:28:03.682601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.231257 (+548656us) service_pool.cc:224] Handling call
0709 12:28:04.232707 (+  1450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:04.233144 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:04.233179 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:04.240721 (+  7542us) log.cc:844] Serialized 26 byte log entry
0709 12:28:04.627091 (+386370us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:04.627190 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.627314 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:05.079034 (+451720us) raft_consensus.cc:1685] finished
0709 12:28:05.079062 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.079201 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3075,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":2320,"thread_start_us":328617,"threads_started":1,"wal-append.queue_time_us":560673}
I20260709 12:28:05.107110  6023 rpcz_store.cc:276] 0709 12:28:02.835507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.064422 (+1228915us) service_pool.cc:224] Handling call
0709 12:28:04.064837 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:04.065196 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:04.065223 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:04.074310 (+  9087us) log.cc:844] Serialized 26 byte log entry
0709 12:28:04.281110 (+206800us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:04.281179 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.281299 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:05.094450 (+813151us) raft_consensus.cc:1685] finished
0709 12:28:05.094473 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.094623 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4670,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":188,"thread_start_us":173988,"threads_started":1,"wal-append.queue_time_us":459144}
I20260709 12:28:05.120066  5944 raft_consensus.cc:1240] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:05.120819  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1810) took 1269 ms. Trace:
I20260709 12:28:05.121188  5944 rpcz_store.cc:276] 0709 12:28:03.888784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.106624 (+1217840us) service_pool.cc:224] Handling call
0709 12:28:05.119884 (+ 13260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.120550 (+   666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.120755 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:05.122756  6018 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:28:05.125588  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:05.126776  5949 raft_consensus.cc:1240] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:05.127414  6016 raft_consensus.cc:2468] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:28:05.096935 12909 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:28:05.130227  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1811) took 1240 ms. Trace:
I20260709 12:28:05.133158  5949 rpcz_store.cc:276] 0709 12:28:03.890240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.109683 (+1219443us) service_pool.cc:224] Handling call
0709 12:28:05.126474 (+ 16791us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.129860 (+  3386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.130143 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:05.102598 13383 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 26 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:05.142233  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:05.143823  6015 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 25: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:05.146708 13304 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:05.147230 13304 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:05.148078 13304 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:05.149243 13304 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:05.152498  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1706) took 1467 ms. Trace:
I20260709 12:28:05.155257 13321 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:05.158293 13321 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.161051 13321 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:05.167308 13372 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.168064 13372 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:05.152954  6015 rpcz_store.cc:276] 0709 12:28:03.684854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.151647 (+1466793us) service_pool.cc:224] Handling call
0709 12:28:05.151969 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.152153 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:05.152179 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:05.152224 (+    45us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:05.152254 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.152304 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.152412 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:05.173660  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:05.174248  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1809) took 1323 ms. Trace:
I20260709 12:28:05.169210  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1420) took 1524 ms. Trace:
I20260709 12:28:05.171290  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:05.112005 13276 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:05.156113 13366 log.cc:927] Time spent T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 12:28:05.158944 13247 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:05.174391  6015 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:05.175477  5941 rpcz_store.cc:276] 0709 12:28:03.644988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.855704 (+1210716us) service_pool.cc:224] Handling call
0709 12:28:04.862601 (+  6897us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:04.868465 (+  5864us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:04.868519 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:04.900071 (+ 31552us) log.cc:844] Serialized 26 byte log entry
0709 12:28:05.061580 (+161509us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:05.061664 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.061746 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:05.168969 (+107223us) raft_consensus.cc:1685] finished
0709 12:28:05.168992 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.169148 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5787,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":6292,"thread_start_us":139817,"threads_started":1,"wal-append.queue_time_us":236750}
I20260709 12:28:05.175203  6020 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:28:05.119349 13173 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:05.174732  5940 rpcz_store.cc:276] 0709 12:28:03.851114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.875844 (+ 24730us) service_pool.cc:224] Handling call
0709 12:28:05.174005 (+1298161us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b5400632c54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:05.174175 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":903774976}
I20260709 12:28:05.158556 13301 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:05.097543 13373 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:05.195950  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 86144.5us
I20260709 12:28:05.204835  6021 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:05.195963 13368 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 25 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:05.240945  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:05.173148 13369 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:05.215054 13372 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:05.058117  5690 rpcz_store.cc:269] 0709 12:26:59.017130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.017531 (+   401us) service_pool.cc:224] Handling call
0709 12:28:04.903361 (+65885830us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:54.718724 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:54.908021 (+189297us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:54.908087 (+    66us) write_op.cc:435] Acquired schema lock
0709 12:27:54.908110 (+    23us) tablet.cc:662] Decoding operations
0709 12:27:54.909023 (+   913us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:54.909152 (+   129us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:54.909184 (+    32us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:27:54.910016 (+   832us) tablet.cc:701] Row locks acquired
0709 12:27:54.910078 (+    62us) write_op.cc:260] PREPARE: finished
0709 12:27:54.910271 (+   193us) write_op.cc:270] Start()
0709 12:27:54.910453 (+   182us) write_op.cc:276] Timestamp: P: 1783600074910201 usec, L: 0
0709 12:27:54.910491 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:27:55.020530 (+110039us) log.cc:844] Serialized 2827 byte log entry
0709 12:27:55.937885 (+917355us) op_driver.cc:464] REPLICATION: finished
0709 12:27:57.013289 (+1075404us) write_op.cc:301] APPLY: starting
0709 12:27:57.022209 (+  8920us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:57.125616 (+103407us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:57.125807 (+   191us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:57.134941 (+  9134us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:57.211409 (+ 76468us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=0,mrs_lookups=0
0709 12:27:57.222054 (+ 10645us) write_op.cc:312] APPLY: finished
0709 12:27:57.249562 (+ 27508us) log.cc:844] Serialized 72 byte log entry
0709 12:27:57.400644 (+151082us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:57.400774 (+   130us) write_op.cc:454] Released schema lock
0709 12:27:57.400846 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:57.401746 (+   900us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:57.401781 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":210666,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":936925,"apply.run_cpu_time_us":35884,"apply.run_wall_time_us":444719,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33415,"num_ops":7,"prepare.queue_time_us":210165,"prepare.run_cpu_time_us":53410,"prepare.run_wall_time_us":772691,"raft.queue_time_us":348635,"raft.run_cpu_time_us":33601,"raft.run_wall_time_us":904623,"replication_time_us":997313,"thread_start_us":1312714,"threads_started":5,"wal-append.queue_time_us":628985,"wal-append.run_cpu_time_us":39631,"wal-append.run_wall_time_us":2276247}]]}
I20260709 12:28:05.217597 13173 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:05.219193 13276 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:05.200284 13374 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:05.221783  6762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1483) took 1006 ms. Trace:
I20260709 12:28:05.227264  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:05.227968 13247 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:05.227269 13321 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:05.230152 13377 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:05.232223 13383 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:05.232167  6015 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 25.
W20260709 12:28:05.253954  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 162013us
I20260709 12:28:05.254881  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:05.194942 13307 consensus_queue.cc:1059] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 12:28:05.197438 13264 consensus_queue.cc:1048] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:28:05.203487  6611 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:05.201314 13271 consensus_queue.cc:1048] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:28:05.206223  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:05.156471 13391 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 12:28:05.241817  5866 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:28:05.242362  6766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1472) took 1837 ms. Trace:
W20260709 12:28:05.270015  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 63854.4us
I20260709 12:28:05.271242  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:28:05.269464  5866 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1511) took 1651 ms (client timeout 1910 ms). Trace:
W20260709 12:28:05.272351  5866 rpcz_store.cc:269] 0709 12:28:03.617898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.618071 (+   173us) service_pool.cc:224] Handling call
0709 12:28:05.240725 (+1622654us) spinlock_profiling.cc:243] Waited 35.7 ms on lock 0x7b58002e2f98. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5eb71 00007f39cfb5ea22 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c490a0 00007f39d7c48984 00007f39d11c36d4 00007f39d11c3662
0709 12:28:05.269395 (+ 28670us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82198912}
I20260709 12:28:05.243176  6016 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 27.
I20260709 12:28:05.221004  5940 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:28:05.246407  6762 rpcz_store.cc:276] 0709 12:28:04.215418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.217795 (+  2377us) service_pool.cc:224] Handling call
0709 12:28:04.218214 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:04.218653 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:04.218692 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:04.224253 (+  5561us) log.cc:844] Serialized 26 byte log entry
0709 12:28:04.569878 (+345625us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:04.569942 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.570168 (+   226us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:05.155343 (+585175us) spinlock_profiling.cc:243] Waited 7.93 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:28:05.221403 (+ 66060us) raft_consensus.cc:1685] finished
0709 12:28:05.221427 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.221713 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":600,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":10012,"spinlock_wait_cycles":18233088,"thread_start_us":308728,"threads_started":1,"wal-append.queue_time_us":622074}
I20260709 12:28:05.257033  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:05.262838  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 59498.6us
I20260709 12:28:05.276867  6611 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:05.279862  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:05.280680  6015 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:28:05.281692  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1713) took 1551 ms. Trace:
I20260709 12:28:05.282172  6015 rpcz_store.cc:276] 0709 12:28:03.730386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.279742 (+1549356us) service_pool.cc:224] Handling call
0709 12:28:05.280144 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.281347 (+  1203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.281597 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:05.269435  6766 rpcz_store.cc:276] 0709 12:28:03.405213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.533543 (+128330us) service_pool.cc:224] Handling call
0709 12:28:03.544631 (+ 11088us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:03.651827 (+107196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:03.651866 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:03.652401 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 12:28:03.862167 (+209766us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:03.862232 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.239001 (+1376769us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540004bfd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:05.239048 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:05.239104 (+    56us) raft_consensus.cc:1685] finished
0709 12:28:05.239116 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.242278 (+  3162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5149,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"spinlock_wait_cycles":245501696,"thread_start_us":209259,"threads_started":1,"wal-append.queue_time_us":311703,"wal-append.run_cpu_time_us":1770,"wal-append.run_wall_time_us":1043675}
I20260709 12:28:05.242406 13168 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:05.269338 13374 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:05.291460  6015 raft_consensus.cc:3055] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:05.293267  6015 raft_consensus.cc:740] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:05.271539 13173 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:05.286543 13307 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:05.251482 13247 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:05.286788 13362 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:05.299394  5990 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:05.297758  6015 consensus_queue.cc:260] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:05.307032  5990 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:05.305944  5915 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:05.294903 13374 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.317764  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:05.320889  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:05.322435  5915 leader_election.cc:400] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Vote denied by peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701) with higher term. Message: Invalid argument: T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 18. Current term is 19.
I20260709 12:28:05.286048 13398 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:28:05.327405  6020 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:05.328380  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:05.323926  5915 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:05.295644 13173 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.335973 13173 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:05.314065 13391 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:05.342413 13391 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.343006 13391 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:05.312312 13301 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:05.345860 13301 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.346413 13301 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:05.351613 13278 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:05.352247 13278 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.352703 13278 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:05.368468 13278 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:05.372828 13278 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:05.379897  5915 leader_election.cc:336] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:05.380570  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:05.326038 13286 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:05.319149 13174 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:05.396374  6020 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:28:05.325471 13221 raft_consensus.cc:2804] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:05.324581 13374 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:05.402227  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:05.319736 13307 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:28:05.405566  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 127458us
I20260709 12:28:05.330008 13373 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:05.303175 13368 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:05.381477 13278 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:05.412286  5915 leader_election.cc:336] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:05.319826 13276 consensus_queue.cc:1059] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 12:28:05.420593  6022 raft_consensus.cc:1275] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:28:05.314940 13372 consensus_queue.cc:1059] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 12:28:05.423364  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:05.430914  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:05.433022  5941 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 19: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:28:05.430809  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 112424us
I20260709 12:28:05.435700 13321 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:05.436295  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:05.422484 13373 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:05.437238  5945 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 27.
I20260709 12:28:05.438189 13307 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:05.451089  5915 leader_election.cc:336] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:05.452263  5915 leader_election.cc:336] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:05.453058  5915 leader_election.cc:336] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:05.460093  6775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1478) took 1366 ms. Trace:
I20260709 12:28:05.460784  6775 rpcz_store.cc:276] 0709 12:28:04.093495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.093717 (+   222us) service_pool.cc:224] Handling call
0709 12:28:04.222683 (+128966us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.242317 (+ 19634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.459744 (+1217427us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54005294d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:05.459988 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295688448}
I20260709 12:28:05.459308  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:28:05.464555  5915 consensus_peers.cc:597] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.465102  5915 leader_election.cc:336] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:05.436340 13168 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:05.478869  5945 raft_consensus.cc:1275] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 14. (term mismatch)
I20260709 12:28:05.479424  5945 pending_rounds.cc:85] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 13
I20260709 12:28:05.479609  5945 pending_rounds.cc:107] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 26.14
I20260709 12:28:05.479754  5945 raft_consensus.cc:2889] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:05.476918  5915 leader_election.cc:336] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:05.480906  5915 consensus_peers.cc:597] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.481823  5915 consensus_peers.cc:597] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.481972  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:05.482129  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:05.482705  5871 raft_consensus.cc:3060] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:05.454197 13321 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:05.490741  5868 raft_consensus.cc:1240] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 27 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 26. Current term is 27. Ops: [26.14-26.14]
I20260709 12:28:05.473662 13377 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:05.502691  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:05.474967 13278 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:05.501184  5866 raft_consensus.cc:1275] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 29 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:28:05.503468  5865 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
W20260709 12:28:05.492923 13396 log.cc:927] Time spent T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
W20260709 12:28:05.492978 13402 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:28:05.511883  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:05.514011  5871 raft_consensus.cc:2468] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:28:05.492630 13404 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.515846  5874 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:05.519426 13136 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.514582 13410 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.560513 13410 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:05.562026  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:05.562865  5873 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:05.541256  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:28:05.567159  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 91387.1us
I20260709 12:28:05.500777 13168 consensus_queue.cc:1059] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 12:28:05.543695  5945 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:05.547647  5915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.493821 13391 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:05.550148 13221 raft_consensus.cc:697] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 26 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:05.501547 13321 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:05.558120 13400 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:28:05.558444 13404 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:05.577003  5915 leader_election.cc:336] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:05.559510 13136 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:05.573926  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 84860.1us
I20260709 12:28:05.566411 13247 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:05.486896 13406 consensus_queue.cc:1059] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.091s
I20260709 12:28:05.506574 13173 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:05.549819 13372 consensus_queue.cc:1059] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:05.566347 13362 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.569103  5990 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:05.596817 13363 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:05.598640 13363 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:28:05.578653  5915 leader_election.cc:336] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:05.599119  5945 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:28:05.589775  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:05.600370  6022 raft_consensus.cc:3060] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:28:05.599175 13363 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:05.602733 13321 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:05.603680 13363 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:05.604740 13404 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:05.605576 13363 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:05.607002 13136 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:05.607189 13173 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:05.607645 13136 raft_consensus.cc:3055] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:05.608139 13136 raft_consensus.cc:740] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:28:05.610778 13361 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.880s	user 0.003s	sys 0.000s
I20260709 12:28:05.586315 13377 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:05.612756 13377 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:05.613904 13417 raft_consensus.cc:3055] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:05.614208 13406 raft_consensus.cc:2804] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:05.614485 13417 raft_consensus.cc:740] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:05.614740 13406 raft_consensus.cc:697] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:05.615020 13377 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:05.615541  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46531.3us
I20260709 12:28:05.588371  5873 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:05.619405  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:05.609112 13136 consensus_queue.cc:260] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:05.592902 13362 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:05.627849  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:28:05.621964 13338 log.cc:927] Time spent T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.704s	user 0.006s	sys 0.000s
I20260709 12:28:05.630764 13406 consensus_queue.cc:237] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:05.633211 13404 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:05.634016 13404 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:05.636463 13404 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:05.636871 13377 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:05.637518 13377 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:05.627228 13362 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:05.638921 13399 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 12:28:05.645484 13301 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:05.653748 13221 consensus_queue.cc:237] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:05.622372 13372 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:05.684091 13301 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:05.687901  5915 leader_election.cc:336] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:05.667936 13382 consensus_meta.cc:287] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.554s	user 0.008s	sys 0.005s
I20260709 12:28:05.691965  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2204) took 1365 ms. Trace:
W20260709 12:28:05.690250  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 48474.9us
W20260709 12:28:05.680275 13400 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:28:05.690891  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:05.656524 13276 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.692515  5949 rpcz_store.cc:276] 0709 12:28:04.326705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.145250 (+818545us) service_pool.cc:224] Handling call
0709 12:28:05.187893 (+ 42643us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:05.188481 (+   588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:05.188525 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:05.197695 (+  9170us) log.cc:844] Serialized 52 byte log entry
0709 12:28:05.285501 (+ 87806us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:05.287041 (+  1540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.287139 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:05.691588 (+404449us) raft_consensus.cc:1685] finished
0709 12:28:05.691621 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.691878 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":879,"raft.queue_time_us":161,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":5660,"thread_start_us":62926,"threads_started":1,"wal-append.queue_time_us":165685}
I20260709 12:28:05.696424  5945 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:28:05.697717  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:05.698067  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 220686us
I20260709 12:28:05.699070  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:05.699148  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:05.701596  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:05.704365 13282 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:05.707705  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1671) took 2423 ms. Trace:
I20260709 12:28:05.708139  6021 rpcz_store.cc:276] 0709 12:28:03.284193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.167819 (+883626us) service_pool.cc:224] Handling call
0709 12:28:04.198960 (+ 31141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.209367 (+1010407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.707371 (+498004us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54003638d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:05.707631 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2309177600}
I20260709 12:28:05.709532  5945 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:05.700641 13276 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:05.691191 13410 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:05.615419 13417 consensus_queue.cc:260] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:05.716771  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:05.686096 13374 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:05.719020  6021 raft_consensus.cc:2468] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:28:05.712343 13282 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:05.720371  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:05.720465 13282 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:05.722432  5945 raft_consensus.cc:1240] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:28:05.722970 13282 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:05.723919 13175 raft_consensus.cc:3055] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:05.724411 13175 raft_consensus.cc:740] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:05.726475 13312 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:05.727049 13312 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:05.730903 13417 raft_consensus.cc:3060] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:05.731228  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1808) took 1880 ms. Trace:
I20260709 12:28:05.731850  5942 rpcz_store.cc:276] 0709 12:28:03.850834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.366515 (+515681us) service_pool.cc:224] Handling call
0709 12:28:05.730752 (+1364237us) spinlock_profiling.cc:243] Waited 7.27 ms on lock 0x7b5400633154. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:05.731107 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16723328}
I20260709 12:28:05.734076  5940 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
W20260709 12:28:05.740612 13304 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.568s	user 0.013s	sys 0.000s
I20260709 12:28:05.694531 13382 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:05.743860 13360 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.818s	user 0.003s	sys 0.000s
I20260709 12:28:05.744727 13382 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:05.746415  5873 pending_rounds.cc:85] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 9
I20260709 12:28:05.746902  5873 pending_rounds.cc:107] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:28:05.747361  5873 raft_consensus.cc:2889] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:05.691421 13372 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:05.752403  5990 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:05.756470  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:05.756788  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:05.758260  5990 leader_election.cc:336] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:05.746868 13418 raft_consensus.cc:2764] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election decision vote started in defunct term 22: won
W20260709 12:28:05.760267  5990 leader_election.cc:336] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:05.760097  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:05.761235  5990 leader_election.cc:336] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:05.761377  5872 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 26.
W20260709 12:28:05.762100  5990 leader_election.cc:336] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:05.770613  5990 leader_election.cc:336] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:05.714375 13264 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:28:05.750579 13421 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:28:05.771040  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:05.773169  5990 consensus_peers.cc:597] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.707206 13377 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:05.725458 13175 consensus_queue.cc:260] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:05.783713  5872 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:28:05.784075  5990 leader_election.cc:336] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:05.785799  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 85143.7us
I20260709 12:28:05.782238  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:28:05.785411  5990 consensus_peers.cc:597] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.787269  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:05.791567  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:05.795222  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:28:05.796015  5990 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.807359  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:05.808314  5871 raft_consensus.cc:2393] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 23: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:05.789371 13301 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:05.805584  5913 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:05.796900  5872 raft_consensus.cc:2393] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 24: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:05.793488 13175 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:05.814730  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1518) took 1490 ms. Trace:
I20260709 12:28:05.815135  5869 rpcz_store.cc:276] 0709 12:28:04.324207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.324663 (+   456us) service_pool.cc:224] Handling call
0709 12:28:04.336986 (+ 12323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.030579 (+693593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.814479 (+783900us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b5400118ad4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:05.814671 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1453531008}
I20260709 12:28:05.817981 13374 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:05.799324 13391 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 28 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:05.819224  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:05.820053  6023 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:05.821507  5947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1814) took 1910 ms (client timeout 2090 ms). Trace:
W20260709 12:28:05.821974  5947 rpcz_store.cc:269] 0709 12:28:03.911522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.920038 (+  8516us) service_pool.cc:224] Handling call
0709 12:28:05.821287 (+1901249us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b54008b01d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:05.821448 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":685115776}
I20260709 12:28:05.779409 13421 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:05.810222  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:05.811688  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:05.812983  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:05.821991  6022 raft_consensus.cc:2468] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:05.821187 13188 consensus_queue.cc:1048] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:05.828606  5874 raft_consensus.cc:2368] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22.
I20260709 12:28:05.827718  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:05.829696  6022 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:05.834074 13406 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:05.834708 13406 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:05.835134 13406 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:05.846448  5987 leader_election.cc:400] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 24. Current term is 25.
I20260709 12:28:05.875295  5987 leader_election.cc:403] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 12:28:05.810997 13301 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:05.877871  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:05.883162  6024 raft_consensus.cc:2410] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 26 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:28:05.888984 13221 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 24. Current term is 25.
I20260709 12:28:05.877233  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:05.861017 13421 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:05.876493 13404 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.899765  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:05.900643  5869 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:05.902701  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.902216 13404 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:05.905529 13404 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:05.915570  5990 consensus_peers.cc:597] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.917331  6024 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:05.918272 13372 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:05.918987  6024 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:05.883230 13301 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:05.930086  5990 leader_election.cc:336] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:05.930991  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 125538us
I20260709 12:28:05.931527  6022 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 28.
I20260709 12:28:05.853413 13304 raft_consensus.cc:2804] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:05.824877 13423 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:05.856734 13428 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:05.857275 13412 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.858319 13276 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:05.860992  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:05.864784  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:05.947600  5872 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:28:05.866207  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:05.873001  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.948298  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:05.873355  6023 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
I20260709 12:28:05.950353  6023 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:28:05.851656 13406 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:05.951509  5874 raft_consensus.cc:2368] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22.
I20260709 12:28:05.957392  6024 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:05.956796  5913 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:05.950559 13304 raft_consensus.cc:697] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:05.954022 13188 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:28:05.959326  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:05.963719 13304 consensus_queue.cc:237] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:05.969306 13385 raft_consensus.cc:3055] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:05.958854 13175 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.983347  5869 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 27.
I20260709 12:28:05.962407 13406 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:05.943989 13428 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.984174  6024 raft_consensus.cc:1275] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:05.960978 13418 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:05.983233  5871 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 20: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:05.987848  6024 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:05.950371 13412 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:05.988420 13175 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:05.992846 12983 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:05.993918  6023 raft_consensus.cc:3055] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:06.001070  6024 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 26. Current term is 27. Ops: []
W20260709 12:28:05.987172 13383 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.001s	sys 0.006s
I20260709 12:28:06.012123  6023 raft_consensus.cc:740] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:06.013054  5944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2203) took 1687 ms. Trace:
I20260709 12:28:06.013517  5944 rpcz_store.cc:276] 0709 12:28:04.326066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.143276 (+817210us) service_pool.cc:224] Handling call
0709 12:28:05.184274 (+ 40998us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:05.184801 (+   527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:05.184843 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:05.194696 (+  9853us) log.cc:844] Serialized 26 byte log entry
0709 12:28:05.865471 (+670775us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b1800383b60. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d28230d1 00007f39d2c3ea0f
0709 12:28:05.954368 (+ 88897us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:05.954413 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.954487 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:06.000560 (+ 46073us) raft_consensus.cc:1685] finished
0709 12:28:06.000582 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:06.012979 (+ 12397us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":180,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":10492,"spinlock_wait_cycles":5261184,"thread_start_us":88317,"threads_started":1,"wal-append.queue_time_us":787616}
I20260709 12:28:05.983887 13385 raft_consensus.cc:740] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:06.013247  6023 consensus_queue.cc:260] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:05.989236 13432 raft_consensus.cc:2804] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:05.986397 13428 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:06.014433 13383 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.003s	sys 0.008s
I20260709 12:28:06.023294  6023 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:06.023463  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 177214us
I20260709 12:28:06.028350  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:06.030876  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:06.033023  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:06.034267 13433 log.cc:927] Time spent T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:28:06.037345  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:06.027580 13432 raft_consensus.cc:697] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:06.044912  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:06.045171  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:06.066366 13385 consensus_queue.cc:260] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:06.089699  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.062019 13441 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:06.037949 13406 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:06.163965  5865 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 25.
I20260709 12:28:06.165683  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:06.164317 13383 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.040575 13411 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:06.172856  6021 raft_consensus.cc:2410] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:28:06.166268  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:06.183632  5871 raft_consensus.cc:2393] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 23: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:06.176192  6023 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:28:06.179844  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 134753us
I20260709 12:28:06.185196 13417 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:06.177847 13108 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:28:06.187821  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 231084us
I20260709 12:28:06.188005  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:06.189772 13284 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:06.190261 13284 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:06.190331 13411 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:28:06.190918 13355 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 1.532s	user 0.005s	sys 0.000s
I20260709 12:28:06.191323 13284 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:06.192039  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:06.192512 13284 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:06.192873 13411 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:06.193488 13108 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:06.193683  5867 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:28:06.194199  5867 pending_rounds.cc:85] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 10
I20260709 12:28:06.194466  5867 pending_rounds.cc:107] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:28:06.194732  5867 raft_consensus.cc:2889] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:06.194635 13108 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:06.196108 13108 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:06.199143 13363 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.199803 13363 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:06.195896  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:06.205189  5990 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:06.206220  5990 leader_election.cc:336] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:06.206286  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:06.208076  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:06.208698  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:06.209542  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:06.209879  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:06.185231 13383 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:06.210390  5942 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 27.
I20260709 12:28:06.211231  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:06.210457  5947 raft_consensus.cc:1240] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:06.208277  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:06.212034  5944 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 21: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:06.209546  5871 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:06.177713 13312 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:06.212795 13417 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:06.214339 13363 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:06.214536 13417 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:06.215123 13417 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:06.215528  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1781) took 3254 ms. Trace:
I20260709 12:28:06.216955  6015 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:06.219492  6024 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:06.181547 13441 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:06.220993 13441 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:06.227579  6023 raft_consensus.cc:3055] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:06.228066  6023 raft_consensus.cc:740] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:06.229017  5944 raft_consensus.cc:1240] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22. Ops: [19.12-21.13]
I20260709 12:28:06.222322 13418 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.215961  6017 rpcz_store.cc:276] 0709 12:28:02.961347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.066382 (+1105035us) service_pool.cc:224] Handling call
0709 12:28:04.066795 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:04.067224 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:04.067523 (+   299us) log.cc:844] Serialized 14 byte log entry
0709 12:28:04.224415 (+156892us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:04.232513 (+  8098us) log.cc:844] Serialized 26 byte log entry
0709 12:28:04.232738 (+   225us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:04.232789 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.232895 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:06.190185 (+1957290us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:28:06.202697 (+ 12512us) raft_consensus.cc:1685] finished
0709 12:28:06.215225 (+ 12528us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:06.215426 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6897,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":3733,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":7737,"spinlock_wait_cycles":8681216,"thread_start_us":155620,"threads_started":1,"wal-append.queue_time_us":332294}
I20260709 12:28:06.230196  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:28:06.230897  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.221516 13371 raft_consensus.cc:3055] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:28:06.224596 13401 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.761s	user 0.002s	sys 0.000s
I20260709 12:28:06.235456  5913 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:06.228120  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:06.238183  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:06.231412  6611 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:06.241371  6611 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:06.241364 13404 consensus_queue.cc:1059] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:06.242363 13398 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:06.243026 13398 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 28 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:06.243220 13404 raft_consensus.cc:3055] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:06.243783  6015 raft_consensus.cc:1275] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 12:28:06.244270  6015 pending_rounds.cc:85] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 11
I20260709 12:28:06.244556  6015 pending_rounds.cc:107] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:28:06.244135 13398 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:06.232391 13418 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:06.245496  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:06.246500 13410 raft_consensus.cc:2804] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:06.246939 13410 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:06.247282 13410 raft_consensus.cc:3060] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:06.247323 13418 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:06.244781  6015 raft_consensus.cc:2889] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:06.250018  6023 consensus_queue.cc:260] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:06.254360  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:06.254750  6023 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:06.254442  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:06.255856  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1714) took 1319 ms. Trace:
I20260709 12:28:06.256424  6015 rpcz_store.cc:276] 0709 12:28:04.936032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.283209 (+347177us) service_pool.cc:224] Handling call
0709 12:28:05.283619 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:06.216521 (+932902us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54000e40a0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bf8a1d 00007f39d2bdf1a4 00007f39d2c61447 00007f39d2c79fe7 00007f39d2c7b7de 00007f39d2c85b49 00007f39d2c86416 00007f39d2c82d8f 00007f39d2c8283e 00007f39d7c47419
0709 12:28:06.255490 (+ 38969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.255766 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538528128}
I20260709 12:28:06.252391  6611 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:06.239657 13371 raft_consensus.cc:740] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:06.267113 13374 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:06.271427 13374 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 25 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:06.272603 13374 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:06.267293 13407 consensus_queue.cc:1059] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:06.269016 13188 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:06.275043 13188 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:06.275784 13176 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:06.278316  6017 raft_consensus.cc:1275] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:28:06.260516 13418 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:06.279663 13418 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:06.281203 13278 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:06.281698 13278 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:06.282053 13278 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:06.252583 13403 raft_consensus.cc:2804] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:06.263919  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:06.258200 13447 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:06.270619 13444 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:28:06.243851 13404 raft_consensus.cc:740] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:06.280228 13348 raft_consensus.cc:3055] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:06.292178 13348 raft_consensus.cc:740] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:06.293027  6023 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 13. (term mismatch)
I20260709 12:28:06.293437 13348 consensus_queue.cc:260] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:06.294828 13348 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:06.277405 13188 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:06.297694 13447 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:06.301851  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:06.303443  6021 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:06.304728  5987 leader_election.cc:400] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20.
I20260709 12:28:06.326817  5987 leader_election.cc:403] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:28:06.304853 13412 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:06.313512  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.330519 13406 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20.
I20260709 12:28:06.336563  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:06.293506  6023 pending_rounds.cc:85] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 12
I20260709 12:28:06.338933  6023 pending_rounds.cc:107] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 24.13
I20260709 12:28:06.288715 13403 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:06.340409  6023 raft_consensus.cc:2889] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:06.343454  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:28:06.345247  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 109864us
I20260709 12:28:06.347955  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:06.353437  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:06.352212  6023 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:28:06.392436 13312 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:06.656574  5694 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40830 (request call id 151) took 79620 ms (client timeout 15000 ms). Trace:
W20260709 12:28:06.727146  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 499093us
I20260709 12:28:06.369761  5869 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 22: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:28:06.426154  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:06.794670  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.359581  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:06.395066  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:06.411669 13136 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:06.374689 13447 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:06.354635 13412 raft_consensus.cc:2764] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 12:28:06.702201 13407 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.799803 13411 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:06.800243 13407 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:06.345014 13403 raft_consensus.cc:3060] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:06.799276  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 439735us
I20260709 12:28:06.423378 13371 consensus_queue.cc:260] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:06.354240 13418 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:06.511979 13385 raft_consensus.cc:3060] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:06.464205  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.808070  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:06.258538 13432 consensus_queue.cc:237] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:06.810288  5871 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:06.393832 12783 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.791726 13454 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:06.485049 13278 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:06.415437 13247 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
I20260709 12:28:06.457609  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2207) took 2038 ms. Trace:
I20260709 12:28:06.397245  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1413) took 2852 ms. Trace:
I20260709 12:28:06.451844  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:06.446075  6016 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 19. Current term is 20. Ops: []
W20260709 12:28:06.454937 13358 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 1.796s	user 0.004s	sys 0.000s
I20260709 12:28:06.427708 13383 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:06.354143 13406 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:06.449222 13373 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:06.439098  6020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1706) took 1265 ms (client timeout 1503 ms). Trace:
I20260709 12:28:06.349066 13417 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:06.450122  6785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1892) took 2758 ms. Trace:
I20260709 12:28:06.476852  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:28:06.860236  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 383461us
I20260709 12:28:06.387588 13428 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:06.436151  6021 raft_consensus.cc:2410] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:28:06.400082 13437 raft_consensus.cc:3055] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:28:06.483692  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.867267  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1710) took 3176 ms. Trace:
W20260709 12:28:06.701500  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 435680us
I20260709 12:28:06.867780  6016 rpcz_store.cc:276] 0709 12:28:03.690538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.286465 (+1595927us) service_pool.cc:224] Handling call
0709 12:28:06.445769 (+1159304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:06.828029 (+382260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.867063 (+ 39034us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400763750. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:06.867208 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2665036160}
I20260709 12:28:06.480545  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1418) took 2908 ms. Trace:
I20260709 12:28:06.868952  5948 rpcz_store.cc:276] 0709 12:28:03.572413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.415307 (+842894us) service_pool.cc:224] Handling call
0709 12:28:04.443938 (+ 28631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.078612 (+634674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.480196 (+1401584us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54006a3bd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:06.480470 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1412541824}
I20260709 12:28:06.573415 13441 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:06.875082 13441 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:06.352135 13175 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:06.875932 13175 consensus_queue.cc:1059] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:06.876679 13175 consensus_queue.cc:1243] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 25.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:28:06.514252  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2285) took 2504 ms. Trace:
I20260709 12:28:06.877354  5867 rpcz_store.cc:276] 0709 12:28:04.010167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.014042 (+  3875us) service_pool.cc:224] Handling call
0709 12:28:04.014321 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:06.195660 (+2181339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.513853 (+318193us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b5400210a54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:06.514130 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5010811136}
I20260709 12:28:06.403167  6017 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 12:28:06.880373  5949 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:28:06.880705  6611 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:06.881546  6019 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:06.882155  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35204 (request call id 1434) took 1262 ms. Trace:
I20260709 12:28:06.882292 13417 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:06.882565  5949 rpcz_store.cc:276] 0709 12:28:05.619309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.696632 (+ 77323us) service_pool.cc:224] Handling call
0709 12:28:06.881984 (+1185352us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540058f554. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:06.882093 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2709004160}
I20260709 12:28:06.883592 13405 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:06.885507 13461 raft_consensus.cc:3055] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:06.885974 13461 raft_consensus.cc:740] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:06.886772 13461 consensus_queue.cc:260] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:06.887188 13278 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:06.888023  5838 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:06.855718 13147 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:28:06.888867 13147 consensus_queue.cc:1243] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 25.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:28:06.889115 13437 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:06.889775 13304 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:06.890491 13304 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:06.890717 13272 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:06.890573 13437 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:06.891716 13304 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:06.892666 13437 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:06.893903  6016 raft_consensus.cc:1240] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:06.895183 13384 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:28:06.407503  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.896258  6019 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:06.345335 13301 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:06.898844 13197 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.899410 13447 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:06.899739  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:06.899561 13197 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:06.900559  5948 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 25: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:06.901512  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:06.901777  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:06.902221  5867 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:06.902351  5945 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
W20260709 12:28:06.523790  5990 leader_election.cc:336] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:06.903704  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 379922us
W20260709 12:28:06.904400  5990 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:06.840166 13454 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:06.670820  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.371678 13404 consensus_queue.cc:260] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:06.907747 13404 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:06.754626  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.434535 13410 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:06.780726  5865 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.003s	sys 0.012s
I20260709 12:28:06.910176  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:06.910588  5865 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.004s	sys 0.012s
I20260709 12:28:06.741814 13423 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:06.790580  5694 rpcz_store.cc:269] 0709 12:26:47.036049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.036392 (+   343us) service_pool.cc:224] Handling call
0709 12:26:47.275562 (+239170us) spinlock_profiling.cc:243] Waited 41.3 ms on lock 0x7b540077f954. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bed090 00007f39d2c8bf3a 00007f39d8a41e9c 00007f39d8b4004e 00007f39d70012a4 00007f39d7001232 00007f39d7001161 00007f39d70010dc 00007f39d7000392 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:06.656271 (+79380709us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2766835,"reported_tablets":44,"spinlock_wait_cycles":94977920}
W20260709 12:28:06.804095  6015 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.015s	sys 0.016s
W20260709 12:28:06.915997  6015 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.016s	sys 0.016s
I20260709 12:28:06.916112  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:06.787302 13348 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:06.810150  6023 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
W20260709 12:28:06.489481 13419 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 25 current term: 25 is higher.
I20260709 12:28:06.817574  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:06.819278 13407 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:06.822150  5873 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:28:06.398168 13174 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:06.822906  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:06.827725  5940 rpcz_store.cc:276] 0709 12:28:04.419404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.274622 (+855218us) service_pool.cc:224] Handling call
0709 12:28:05.733784 (+459162us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:05.734630 (+   846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.457315 (+722685us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b5400633150. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:06.457520 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1021473024}
I20260709 12:28:06.852329  5941 raft_consensus.cc:1275] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:28:06.922003  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:06.922768  5873 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:28:06.925607  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:06.927081  6021 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:28:06.914470 13374 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.054s	sys 0.080s
W20260709 12:28:06.856704  6020 rpcz_store.cc:269] 0709 12:28:05.173941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.205500 (+ 31559us) service_pool.cc:224] Handling call
0709 12:28:06.438810 (+1233310us) spinlock_profiling.cc:243] Waited 78.8 ms on lock 0x7b5400763754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:06.438992 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181320960}
I20260709 12:28:06.858865  6785 rpcz_store.cc:276] 0709 12:28:03.692038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.969584 (+277546us) service_pool.cc:224] Handling call
0709 12:28:04.084233 (+114649us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:04.089642 (+  5409us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:04.131787 (+ 42145us) log.cc:844] Serialized 14 byte log entry
0709 12:28:04.456295 (+324508us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:04.456952 (+   657us) log.cc:844] Serialized 26 byte log entry
0709 12:28:04.457174 (+   222us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:04.457223 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.457312 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:05.610117 (+1152805us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:28:05.615793 (+  5676us) raft_consensus.cc:1685] finished
0709 12:28:05.619666 (+  3873us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:06.449738 (+830072us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b5400007150. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:06.449977 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":33321,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":63,"raft.queue_time_us":179802,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":7982,"spinlock_wait_cycles":263555840,"thread_start_us":323857,"threads_started":2,"wal-append.queue_time_us":591065,"wal-append.run_cpu_time_us":5550,"wal-append.run_wall_time_us":1882423}
I20260709 12:28:06.861266  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:06.870376 13258 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.827643  5943 rpcz_store.cc:276] 0709 12:28:03.545002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.413213 (+868211us) service_pool.cc:224] Handling call
0709 12:28:04.443809 (+ 30596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.444127 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:04.818524 (+374397us) log.cc:844] Serialized 14 byte log entry
0709 12:28:06.396418 (+1577894us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b18003fcd00. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39d282b4bc 00007f39cfb7948b 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a
0709 12:28:06.396721 (+   303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:06.396786 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:06.396842 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.396944 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:06.397152 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":427089,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":125,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":141,"spinlock_wait_cycles":5330304,"thread_start_us":251049,"threads_started":2,"wal-append.queue_time_us":188871,"wal-append.run_cpu_time_us":3628,"wal-append.run_wall_time_us":1004271}
I20260709 12:28:06.871399 13108 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:06.871879 13284 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.872669 13391 consensus_queue.cc:1059] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:28:06.886878 13382 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:06.898924 13301 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:06.385239  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:06.963734  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.929296 13371 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:06.917703  5942 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 19.
I20260709 12:28:06.516428 13368 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:06.889585 13461 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:06.950167 13258 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:06.975941 13410 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:06.945995 13384 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.978360 13384 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:06.892992 13304 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:06.945454 13284 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:06.930017 13348 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:06.986919 13284 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:06.974242  5942 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:28:06.989374 13258 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:06.915046 13383 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:06.990131 13383 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:06.974208 13368 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:06.990743 13383 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:06.993919  5945 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:28:06.957332 13108 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:06.929699  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.936043  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:07.001221  5949 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 16.
I20260709 12:28:06.937307  5867 raft_consensus.cc:2468] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
W20260709 12:28:06.938978  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 117726us
I20260709 12:28:06.940070  6017 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:06.913436 13428 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:06.946764  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 85551.3us
I20260709 12:28:06.945472  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:06.931556  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:06.951133  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:28:06.931053 13374 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.054s	sys 0.080s
I20260709 12:28:06.952850 13391 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:06.962496  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:06.910200  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:06.934257  5913 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:06.974160  5941 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:28:06.989535  5942 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:28:07.009137  5947 raft_consensus.cc:2410] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 28 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 12:28:06.993245 13368 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:07.010425  5990 leader_election.cc:336] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:06.909348 13454 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:07.040508  5990 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:06.830073 12783 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:07.009102  5868 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:07.042625  5696 catalog_manager.cc:5697] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 18 to 21, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 21 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:28:06.959553 13278 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:06.938344 13412 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:07.048085  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 113952us
I20260709 12:28:07.064214  5913 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:28:07.050060  5990 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 59 similar messages]
I20260709 12:28:06.942092 13197 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:06.967696 13312 raft_consensus.cc:3055] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:06.967913 13373 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:07.091135  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:06.924953 13423 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:07.099138  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:06.947196 13147 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.109s
I20260709 12:28:07.115208  6015 raft_consensus.cc:1275] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:07.061071 13377 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.061537 13286 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:07.123246  5696 catalog_manager.cc:5697] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 19 to 22, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 22 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:07.136130  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:07.138692  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:07.140239  6021 raft_consensus.cc:1240] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:07.071097 13395 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.046587 13136 consensus_queue.cc:1048] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 12:28:07.033576 13174 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 12:28:07.095826 13312 raft_consensus.cc:740] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:07.236176  5696 catalog_manager.cc:5697] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:07.243911 13168 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:07.244591 13168 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:07.250339 13463 log.cc:927] Time spent T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
W20260709 12:28:07.251732 13453 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.392s	user 0.000s	sys 0.000s
I20260709 12:28:07.141719 13384 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:07.254705 13108 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:07.255739 13348 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:07.219107 13391 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:07.258023  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:07.258630 13391 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:07.259140 13391 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:07.240916 13395 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:07.269076  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.274675 13395 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:07.238656 13418 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.280629 13418 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:07.287487 13395 consensus_queue.cc:1059] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:07.289127 13374 raft_consensus.cc:3055] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:07.289544 13374 raft_consensus.cc:740] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:07.290055  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:07.288024 13423 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:07.291463  5867 raft_consensus.cc:2410] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 25 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:28:07.290540 13374 consensus_queue.cc:260] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:07.299612  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.302376 13312 consensus_queue.cc:260] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:07.303975 13312 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:07.305476 13282 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:07.308790 13405 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:07.311779  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:07.227211 13286 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:07.312549 13374 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:07.319362 13168 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:07.323354  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37112 (request call id 1700) took 2155 ms. Trace:
I20260709 12:28:07.340786  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:07.341601  6023 raft_consensus.cc:2468] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:28:07.160058  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:07.346616  6015 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 21: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:07.113355 13278 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:07.327193  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:07.349040  5840 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.024s	sys 0.049s
W20260709 12:28:07.350350  5840 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.024s	sys 0.049s
I20260709 12:28:07.327775 13438 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:07.339109  6022 rpcz_store.cc:276] 0709 12:28:05.167604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.946778 (+779174us) service_pool.cc:224] Handling call
0709 12:28:05.947188 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.947436 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:05.960093 (+ 12657us) log.cc:844] Serialized 14 byte log entry
0709 12:28:06.417186 (+457093us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b18003dfb40. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:28:06.860337 (+443151us) log.cc:844] Serialized 14 byte log entry
0709 12:28:06.973313 (+112976us) spinlock_profiling.cc:243] Waited 4.55 ms on lock 0x7b18003dfb40. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287 00007f39d2c9e208
0709 12:28:07.320972 (+347659us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:07.321056 (+    84us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:07.321125 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:07.323037 (+  1912us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:07.323262 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1885,"prepare.queue_time_us":7483,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":95,"raft.queue_time_us":34183,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":190,"spinlock_wait_cycles":16390784,"thread_start_us":779791,"threads_started":2,"wal-append.queue_time_us":993450}
I20260709 12:28:07.073462 13410 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.110661 13175 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.238s
W20260709 12:28:07.346693  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.142884  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:07.358580  6016 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 22: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:07.106703 13373 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:07.119972 13428 raft_consensus.cc:2804] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:07.341538 13286 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:07.328527  6017 raft_consensus.cc:1240] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:07.090016 13447 consensus_queue.cc:1048] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.254s
W20260709 12:28:07.175683  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.107620 13455 consensus_queue.cc:1048] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.256s
I20260709 12:28:07.108815 13454 consensus_queue.cc:1048] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.239s
W20260709 12:28:07.356205  5987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.023s	sys 0.035s
W20260709 12:28:07.388264  5987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.024s	sys 0.035s
W20260709 12:28:07.388998  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 580964us
I20260709 12:28:07.390677  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:07.356508  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 561027us
I20260709 12:28:07.334384  5696 catalog_manager.cc:5697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 11 to 19. New cstate: current_term: 19 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
W20260709 12:28:07.347628  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 459650us
W20260709 12:28:07.366629  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.332091 13382 consensus_queue.cc:1059] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:28:07.354476 13371 raft_consensus.cc:3055] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:07.412806 13371 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:07.413837 13371 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:07.414163  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:07.415082  6017 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:28:07.415325 13371 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:07.416297 13412 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.416713  5871 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:28:07.417521  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:07.397190 13467 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.021s
W20260709 12:28:07.419180 13403 consensus_meta.cc:287] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.599s	user 0.012s	sys 0.000s
I20260709 12:28:07.419644 13403 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:07.421924  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:07.422953  6023 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:28:07.394677  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:07.398494 12783 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:07.402287 13418 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:07.405447 13108 consensus_queue.cc:1059] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:07.409879 13284 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.411651 13384 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.423815  5696 catalog_manager.cc:5697] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:28:07.416936  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:07.336165 13377 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:07.453938  6022 raft_consensus.cc:1240] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:07.427148  5987 leader_election.cc:400] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22.
I20260709 12:28:07.380198 13428 raft_consensus.cc:697] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:07.457872 13406 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:07.458246 13108 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:07.458521 13428 consensus_queue.cc:237] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:07.459152 13108 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:07.431419 13278 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:07.462517  6016 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:28:07.464098  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:07.465065  6021 raft_consensus.cc:2468] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:28:07.465132  5868 raft_consensus.cc:2468] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:28:07.392656 13454 consensus_queue.cc:1243] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer 6add331326ca47038308a447bf798fa7 log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:28:07.433140  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:07.474406  5913 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:07.476465  6017 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 21: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:07.435022 13373 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:07.436081 13404 consensus_meta.cc:287] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.528s	user 0.009s	sys 0.004s
I20260709 12:28:07.360841 13410 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:07.436538 13391 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:07.450399  5867 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:28:07.444753 13467 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:07.455190  5987 leader_election.cc:403] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:28:07.466439 13464 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:28:07.453943 13284 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:07.440845 13383 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:07.443552 13437 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.496985  5913 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:07.467780 13384 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:07.395845 13441 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:07.467221  5696 catalog_manager.cc:5697] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 19 to 20, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 20 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:28:07.470415 13403 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:07.426299 13432 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.458617 13406 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:07.452543  6611 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:07.444052 12783 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:07.416862 13412 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:07.494180  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 67182us
I20260709 12:28:07.443737 13168 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:28:07.485351 13197 raft_consensus.cc:2804] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:28:07.505179  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:07.445248  5990 consensus_peers.cc:597] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:07.526825  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.530762  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:07.531857  6021 raft_consensus.cc:1275] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:28:07.493408 13455 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.499123 13411 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22.
I20260709 12:28:07.535320  6024 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:28:07.530844  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:07.538234  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:07.535506  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 83082.5us
I20260709 12:28:07.544227  5696 catalog_manager.cc:5697] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 23 to 26, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 26 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:28:07.560473  6017 raft_consensus.cc:2393] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 21: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:07.538861  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:07.561415  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 116359us
I20260709 12:28:07.562354  6020 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
W20260709 12:28:07.542003  5840 consensus_peers.cc:597] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:07.498960 13174 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:28:07.535317  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 140938us
W20260709 12:28:07.556679  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:07.566228  5840 consensus_peers.cc:597] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:07.569453  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:07.526178 13108 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:07.567554  6611 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:28:07.573642  5840 consensus_peers.cc:597] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:07.573735  5947 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:07.533181 13423 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.574499  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:07.578405  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:07.581334  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:07.583158  5948 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:28:07.581693 13406 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:07.591801  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:07.504791 13437 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:07.594921 13437 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:07.536069 13432 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:07.597936 13432 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:07.547941 13348 consensus_queue.cc:1048] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 12:28:07.547807 12783 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:07.602480 12783 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:07.547490 13403 raft_consensus.cc:2804] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:07.602807  5942 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 14. (term mismatch)
I20260709 12:28:07.603201 13403 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.603585  5942 pending_rounds.cc:85] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 13
I20260709 12:28:07.603706 13403 raft_consensus.cc:3060] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:07.571087 13455 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:07.539211 13136 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:07.607293 13136 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.607697 13136 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:07.608217 13384 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:07.608404 13284 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:07.570518 13373 consensus_queue.cc:1059] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:28:07.610849 13373 raft_consensus.cc:3055] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:07.611219 13373 raft_consensus.cc:740] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:07.612083 13373 consensus_queue.cc:260] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:07.613333 13373 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:07.613936 13272 raft_consensus.cc:2804] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:07.614398 13272 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:07.614784 13272 raft_consensus.cc:3060] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:07.614825  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:07.615952 13286 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:07.538491 13304 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:07.604005  5942 pending_rounds.cc:107] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 12:28:07.668217  5942 raft_consensus.cc:2889] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:07.669260 13483 raft_consensus.cc:3055] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:07.622489  5696 catalog_manager.cc:5697] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 16 to 20, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 20 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:28:07.643265  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:07.672019  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 84453.1us
I20260709 12:28:07.672616  5867 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:28:07.582439 13174 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:07.664170 13391 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:07.666548 13377 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:07.666999 13410 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:07.670208 13473 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:28:07.621773 13412 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:07.681972  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 107618us
I20260709 12:28:07.690163  5915 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:07.583043  5840 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.618312 13441 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:07.682875  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.683346  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:07.694437  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:07.681720  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:28:07.697137  5990 leader_election.cc:336] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:07.702047  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 123767us
I20260709 12:28:07.703265  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:07.704716  5990 leader_election.cc:336] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.670365 13197 raft_consensus.cc:697] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:07.577545 13382 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.105s
I20260709 12:28:07.670589 13483 raft_consensus.cc:740] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:28:07.713344  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 130314us
I20260709 12:28:07.719027  5987 leader_election.cc:400] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22.
I20260709 12:28:07.735285  5987 leader_election.cc:403] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:07.680308 13438 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:07.682802 13385 consensus_meta.cc:287] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.859s	user 0.013s	sys 0.001s
I20260709 12:28:07.736747  5987 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:07.738205  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:07.738943  6020 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:28:07.679232 13454 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:07.712311  5696 catalog_manager.cc:5697] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
W20260709 12:28:07.724093  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:07.764146  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.682910 13484 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:28:07.764652  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 91661.5us
I20260709 12:28:07.742041  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:07.649717 13423 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:07.771586  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:07.784426  6022 raft_consensus.cc:2410] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 25 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:28:07.776036  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:07.765107  6611 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:07.740746 13470 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.678177 13304 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:07.787803  5948 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:28:07.790705  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:07.791329  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 443981us
I20260709 12:28:07.786739  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:07.738720 13467 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:28:07.793040  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 102991us
I20260709 12:28:07.735232 13108 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:07.737705 13412 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:07.795301  5696 catalog_manager.cc:5697] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 13 to 20, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 20 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:07.793063  5947 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:28:07.740826 13411 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22.
I20260709 12:28:07.781734 13406 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:07.797588  5915 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:07.794051  6609 leader_election.cc:336] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.702s (SENT)
I20260709 12:28:07.801191  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:07.764353 13258 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.803251  5941 raft_consensus.cc:2393] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 23: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:28:07.767192 13384 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:07.801687 13467 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:07.767230 13474 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:07.798871  5696 catalog_manager.cc:5697] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:07.786433 13417 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:07.795545 13410 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:07.829183  5947 raft_consensus.cc:3055] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:07.796232 13476 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:07.801744 13478 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:07.858940  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:07.859018  5696 catalog_manager.cc:5697] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:07.846105  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.805079 13432 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:07.861866  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:07.861979  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:07.863132  5943 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:28:07.861313  5947 raft_consensus.cc:740] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:07.826188 13147 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:07.839848 13258 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:07.870024 13286 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:07.871241 13174 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:07.870854  5696 catalog_manager.cc:5697] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 17 to 23, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 23 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:07.888965  5696 catalog_manager.cc:5697] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 15 to 19, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 19 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
W20260709 12:28:07.891228 13479 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:28:07.891067  5696 catalog_manager.cc:5697] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 19 to 22, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 22 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:07.892529 13480 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:28:07.794902 13470 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:07.892796  5696 catalog_manager.cc:5697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 20 to 24, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 24 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:28:07.895980 13470 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:07.893595 13456 raft_consensus.cc:3055] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:07.897485 13456 raft_consensus.cc:740] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:07.898264 13438 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.898649 13456 consensus_queue.cc:260] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:07.899078 13438 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:07.900399 13456 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:07.870052 13383 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:07.901995 13438 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:07.902488 13383 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:07.902500  6609 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:07.904645 13383 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:07.841400 13384 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:07.905588  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:07.905694 13420 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:28:07.896965  5696 catalog_manager.cc:5697] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 14 to 22, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 22 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:07.907992 13384 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:07.905836 13383 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 12:28:07.908099 13391 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:28:07.909373 13475 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:28:07.909672 13391 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:07.910254 13391 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:07.910679 13485 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:28:07.862917 13474 raft_consensus.cc:697] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 27 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:07.913964  5696 catalog_manager.cc:5697] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 14 to 24, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 24 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:07.914702 13382 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:07.916471 13382 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:07.917794  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:07.919736 13420 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:07.928397 13423 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:07.915956  5696 catalog_manager.cc:5697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 14 to 19, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 19 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:28:07.791317 13484 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:07.933491  5696 catalog_manager.cc:5697] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 14 to 15, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 15 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:07.935838  5696 catalog_manager.cc:5697] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 10 to 17, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 17 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:07.934643 13391 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:07.936867 13484 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:07.938920 13391 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:07.873883 13483 consensus_queue.cc:260] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:07.943642 13483 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:07.947511  5840 consensus_peers.cc:597] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:07.937656  5696 catalog_manager.cc:5697] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 18 to 20, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 20 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:07.948309  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 212817us
W20260709 12:28:07.949834  5840 consensus_peers.cc:597] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:07.957717  5696 catalog_manager.cc:5697] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af reported cstate change: term changed from 17 to 24, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 24 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:07.960068 13304 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:07.966332  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.912844 13474 consensus_queue.cc:237] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:07.826391 13447 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.842068 13467 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:07.961113 13304 consensus_queue.cc:1048] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:28:07.970114  5947 consensus_queue.cc:260] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:08.016499  5947 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:08.017037  5837 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.976712  5949 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:28:08.007761  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:08.016376  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 60134.3us
I20260709 12:28:08.014652  5871 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:28:07.876264 13410 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.875141 13478 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:08.019145  6609 leader_election.cc:336] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.910s (SENT)
I20260709 12:28:07.859519 13417 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:08.025250  6609 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:08.001825 13312 consensus_meta.cc:287] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.668s	user 0.011s	sys 0.003s
I20260709 12:28:08.034619  5947 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 15. (term mismatch)
I20260709 12:28:08.022143  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:08.020277  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 230815us
I20260709 12:28:08.037302  5947 pending_rounds.cc:85] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 14
I20260709 12:28:08.033730 13272 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:08.019367  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 247882us
I20260709 12:28:08.037773  5873 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:08.038887  5947 pending_rounds.cc:107] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 27.15
I20260709 12:28:08.039811  5913 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:08.047887  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:08.049113 13438 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:08.048432  5947 raft_consensus.cc:2889] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:08.049254  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:08.054447 13438 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:08.054853 13438 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:08.054920  5987 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:08.055265  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:08.056308  5944 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
I20260709 12:28:08.057996 13272 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:08.059058 13412 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:08.010579 13286 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.007745 13472 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:08.027202 13374 consensus_meta.cc:287] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.678s	user 0.011s	sys 0.003s
I20260709 12:28:08.016330 13174 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:08.022941 13410 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:08.045733 13382 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:07.966027 13476 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:08.027674 13478 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:08.055271 13438 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:07.874816 13197 consensus_queue.cc:237] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:08.095537 13147 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:08.018625 13447 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:08.111654  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 252990us
W20260709 12:28:08.117491  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:08.126380  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.087291 13403 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:08.141601 13403 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:08.101473 13455 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:08.105134 13348 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:08.148468  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:08.079834 13286 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:08.102464 12783 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:28:08.177563 13147 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:08.088469 13174 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:08.191707  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:08.192178  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 124415us
I20260709 12:28:08.192790  5947 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 25: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:08.103798 13385 consensus_queue.cc:1048] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 12:28:08.150199 13398 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:08.175060 13488 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.124s	user 0.001s	sys 0.001s
I20260709 12:28:08.205168  6017 pending_rounds.cc:85] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 14
I20260709 12:28:08.211831  6017 pending_rounds.cc:107] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 24.15
I20260709 12:28:08.212342  6017 raft_consensus.cc:2889] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:08.204856  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.213820  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:08.214524  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:08.117602 13491 consensus_queue.cc:1059] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 12:28:08.215245  6024 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:28:08.216565  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:08.216785  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:28:08.195673 13136 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.587s	user 0.013s	sys 0.002s
I20260709 12:28:08.217370  5946 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:08.218163  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:08.195115 13348 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:28:08.220054  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.222283  6024 raft_consensus.cc:1275] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:28:08.174440 13455 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.226397  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:08.248281  6022 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 21: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:08.211431 13304 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.252952  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:08.253218  5915 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:08.217182 13437 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:08.256274  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 211901us
I20260709 12:28:08.260977  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:08.262092  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:08.265749  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:08.266619  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:08.266719  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:08.268253  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:08.268337  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:08.269351  5872 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:08.263038  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:08.270964  5874 raft_consensus.cc:2368] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22.
I20260709 12:28:08.265481  5871 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:28:08.272918  5840 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:08.276150  6024 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 21: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:08.266997 13384 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:08.287357  5867 raft_consensus.cc:1240] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:28:08.291792  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:08.291924  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:08.292172  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:08.290581  5840 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:08.296566  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:08.297286  5866 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
I20260709 12:28:08.308404  5946 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 17.
I20260709 12:28:08.264503 13411 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.214062 13398 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:08.318423  5866 raft_consensus.cc:1275] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:28:08.234771 13136 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:08.356451  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.364526  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:08.365595  5865 raft_consensus.cc:2368] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23.
I20260709 12:28:08.252737 13455 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:08.271711 13484 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 12:28:08.263962 13304 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:08.364008  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:08.287694 13437 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:08.312271 13417 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:08.383728  6609 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:08.317516 13411 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:08.369436 13385 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:28:08.435377  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 51755.6us
W20260709 12:28:08.376582 13486 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.294s	user 0.001s	sys 0.001s
I20260709 12:28:08.444540  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:08.443877  6609 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:28:08.434296 13477 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:28:08.380349 13147 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.463862  6609 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:08.432229 13374 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:08.441974 13507 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:28:08.482461  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.493392  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:08.441507 13385 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.440132 13383 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:08.494927  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:08.373560 13348 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:08.462342 13391 consensus_queue.cc:1059] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 12:28:08.546411 13476 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:08.519613  5865 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:08.627436  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:08.699872  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.576850  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:08.647464  5872 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:28:08.702479  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:08.467330 13403 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:08.701187  5866 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:08.592962  6023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1828) took 1141 ms. Trace:
I20260709 12:28:08.711061  6023 rpcz_store.cc:276] 0709 12:28:07.451331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.452949 (+  1618us) service_pool.cc:224] Handling call
0709 12:28:07.469431 (+ 16482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.470009 (+   578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.470051 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:07.470656 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 12:28:07.805881 (+335225us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:07.805944 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:07.806066 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:08.592665 (+786599us) raft_consensus.cc:1685] finished
0709 12:28:08.592693 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:08.592878 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1144,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"thread_start_us":321349,"threads_started":1,"wal-append.queue_time_us":532261}
I20260709 12:28:08.703395  5872 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
W20260709 12:28:08.571269 13480 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:28:08.615061 13258 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:08.741705  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 451190us
W20260709 12:28:08.578094 13498 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
W20260709 12:28:08.589056 13483 consensus_meta.cc:287] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.645s	user 0.010s	sys 0.005s
W20260709 12:28:08.577311 13494 log.cc:927] Time spent T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
W20260709 12:28:08.576783 13505 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 12:28:08.522708 13385 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:08.495428 13507 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.467084 13282 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.284s
I20260709 12:28:08.514940 13417 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.593339 13174 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:08.574623 13371 consensus_meta.cc:287] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.159s	user 0.017s	sys 0.000s
I20260709 12:28:08.460737 13384 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 22 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:08.617517 13497 raft_consensus.cc:3055] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:28:08.768777  6020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.897s	user 0.006s	sys 0.009s
W20260709 12:28:08.642042 13489 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.390s	user 0.000s	sys 0.001s
I20260709 12:28:08.588696 13286 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:08.563938 13272 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:08.770437  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:28:08.771392  6016 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.914s	user 0.013s	sys 0.024s
I20260709 12:28:08.690121 13411 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:08.563072 13447 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:08.698376 13500 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.648344 13437 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:08.774041  5869 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
I20260709 12:28:08.480993 13374 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:08.579002 13398 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:08.763996  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:08.652522 13455 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:08.914053  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 450444us
I20260709 12:28:08.870368  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:08.773201 13493 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.010s	sys 0.027s
W20260709 12:28:08.878006  6020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.006s	sys 0.009s
I20260709 12:28:08.757495 13403 raft_consensus.cc:697] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:08.862265 13507 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:08.887413  5866 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:08.887434  6016 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.046s	user 0.013s	sys 0.024s
I20260709 12:28:08.930861  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1737) took 1401 ms. Trace:
I20260709 12:28:08.931380  6016 rpcz_store.cc:276] 0709 12:28:07.528999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.538321 (+  9322us) service_pool.cc:224] Handling call
0709 12:28:07.569200 (+ 30879us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.581908 (+ 12708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.581942 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:07.592288 (+ 10346us) log.cc:844] Serialized 26 byte log entry
0709 12:28:07.792239 (+199951us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:08.930034 (+1137795us) log.cc:844] Serialized 14 byte log entry
0709 12:28:08.930383 (+   349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.930512 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:08.930572 (+    60us) raft_consensus.cc:1685] finished
0709 12:28:08.930591 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:08.930784 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":670,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":115,"thread_start_us":1279451,"threads_started":2,"wal-append.queue_time_us":369976}
W20260709 12:28:08.878407 13278 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.045s	sys 0.094s
W20260709 12:28:08.888125  5990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.007s	sys 0.002s
W20260709 12:28:08.900134  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:08.935308  5990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.008s	sys 0.003s
W20260709 12:28:08.893497  5867 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.007s	sys 0.003s
I20260709 12:28:08.756271 13391 consensus_queue.cc:1243] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer 7eabb96dfdd1449c94ad1261ded46c3c log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 12:28:08.938764  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 174822us
W20260709 12:28:08.938176  5913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.013s	sys 0.003s
W20260709 12:28:08.939918  5913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.013s	sys 0.003s
I20260709 12:28:08.941143  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1839) took 1093 ms. Trace:
I20260709 12:28:08.877537 13497 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:08.495805 13147 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:08.918322  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:08.892925 13136 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:08.916931 13374 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:08.941599  6018 rpcz_store.cc:276] 0709 12:28:07.848054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.864475 (+ 16421us) service_pool.cc:224] Handling call
0709 12:28:07.990742 (+126267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.991286 (+   544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.991332 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:07.992034 (+   702us) log.cc:844] Serialized 26 byte log entry
0709 12:28:08.151128 (+159094us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:08.151181 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.151273 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:08.940893 (+789620us) raft_consensus.cc:1685] finished
0709 12:28:08.940915 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:08.941078 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11985,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":8085,"thread_start_us":85486,"threads_started":1,"wal-append.queue_time_us":266305}
W20260709 12:28:08.942974  5947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.018s	sys 0.016s
W20260709 12:28:08.968147  5947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.018s	sys 0.017s
W20260709 12:28:08.916813  5942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.005s	sys 0.009s
W20260709 12:28:08.943567  5915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.000s	sys 0.025s
W20260709 12:28:08.937026  5867 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.008s	sys 0.005s
W20260709 12:28:08.969053  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.000s	sys 0.025s
I20260709 12:28:08.951735  5946 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
W20260709 12:28:08.969954  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 716743us
I20260709 12:28:08.919211  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1742) took 1385 ms. Trace:
I20260709 12:28:08.958451  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:08.971179  5872 raft_consensus.cc:2410] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 27 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
W20260709 12:28:08.940442 13278 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.045s	sys 0.095s
I20260709 12:28:08.959033  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:08.975911  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2327) took 1209 ms. Trace:
I20260709 12:28:08.965133  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:28:08.968909  5942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.005s	sys 0.009s
I20260709 12:28:08.963630  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:08.976352  5868 rpcz_store.cc:276] 0709 12:28:07.767015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.767432 (+   417us) service_pool.cc:224] Handling call
0709 12:28:07.767786 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:07.788386 (+ 20600us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.788438 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:07.800363 (+ 11925us) log.cc:844] Serialized 52 byte log entry
0709 12:28:08.134791 (+334428us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:08.308410 (+173619us) log.cc:844] Serialized 14 byte log entry
0709 12:28:08.449057 (+140647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.449175 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:08.975657 (+526482us) raft_consensus.cc:1685] finished
0709 12:28:08.975681 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:08.975846 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9543,"prepare.queue_time_us":55175,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":5013,"raft.queue_time_us":115692,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":8503,"thread_start_us":539302,"threads_started":3,"wal-append.queue_time_us":393867}
I20260709 12:28:08.979452  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:08.979907  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:08.980132  5867 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:08.948375  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:08.970602  6021 rpcz_store.cc:276] 0709 12:28:07.533950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.664283 (+130333us) service_pool.cc:224] Handling call
0709 12:28:07.664837 (+   554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.700641 (+ 35804us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.700691 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:07.719319 (+ 18628us) log.cc:844] Serialized 26 byte log entry
0709 12:28:08.005310 (+285991us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:08.005348 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.005429 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:08.918924 (+913495us) raft_consensus.cc:1685] finished
0709 12:28:08.918955 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:08.919137 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35262,"raft.queue_time_us":52779,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":4220,"thread_start_us":212942,"threads_started":1,"wal-append.queue_time_us":459907}
I20260709 12:28:08.878176 13417 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:08.976605  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.981040  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:08.971994  5866 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
I20260709 12:28:08.928375 13272 raft_consensus.cc:697] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:08.984601  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:09.001362  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.002440  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:08.994057 13286 consensus_queue.cc:1048] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:08.996219 13383 raft_consensus.cc:3055] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:08.996650  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:08.990125  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:08.942368 13304 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:09.014494  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:09.005002  5866 raft_consensus.cc:1240] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:08.997431  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:09.013903  5873 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 23: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:08.998694 13497 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:09.016860  6018 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
W20260709 12:28:09.014592  5987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.013s	sys 0.027s
W20260709 12:28:09.020267  5943 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.005s	sys 0.039s
I20260709 12:28:09.023852  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.020133  6015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1739) took 1490 ms. Trace:
I20260709 12:28:09.025349  6016 raft_consensus.cc:2393] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 26: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:28:08.997403 13521 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:28:08.975028 13474 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:09.028955 13476 consensus_queue.cc:1048] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:28:09.031033 13474 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:09.033257 13474 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:08.984653 12783 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:08.954300 13517 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:08.926227 13500 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:09.018913  5987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.014s	sys 0.028s
W20260709 12:28:09.025257 13488 log.cc:927] Time spent T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:28:09.059245  5872 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:28:09.024648  6015 rpcz_store.cc:276] 0709 12:28:07.529790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.532430 (+  2640us) service_pool.cc:224] Handling call
0709 12:28:07.532827 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.544606 (+ 11779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.544658 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:07.545346 (+   688us) log.cc:844] Serialized 26 byte log entry
0709 12:28:08.004651 (+459305us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:08.004715 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.004812 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.019884 (+1015072us) raft_consensus.cc:1685] finished
0709 12:28:09.019909 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.020061 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18202,"raft.queue_time_us":14014,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":14860,"thread_start_us":397541,"threads_started":1,"wal-append.queue_time_us":619983}
I20260709 12:28:09.060881  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:09.038407  5867 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:28:09.014209 13383 raft_consensus.cc:740] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:09.019194 13520 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.071094  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:09.064710  5868 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:09.066480  5987 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:09.065600  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:09.087110  5945 raft_consensus.cc:2393] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 24: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:28:09.057003 13412 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.997s	user 0.010s	sys 0.005s
I20260709 12:28:09.097203  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:09.092845  5867 raft_consensus.cc:1275] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 16. (term mismatch)
W20260709 12:28:09.044409 13373 consensus_meta.cc:287] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.376s	user 0.009s	sys 0.007s
I20260709 12:28:09.043726 13517 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.102025  5867 pending_rounds.cc:85] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 15
I20260709 12:28:09.118992  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:28:09.134481  2067 thread.cc:527] Waited for 1000ms trying to join with rpc worker (tid 6688)
I20260709 12:28:09.088359 13483 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:09.028765 13384 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:09.153944  5872 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:09.139164  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:09.139752  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:09.142815  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:09.146852  5867 pending_rounds.cc:107] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 25.16
I20260709 12:28:09.157274  5867 raft_consensus.cc:2889] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:09.153206 13541 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:28:09.158702  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:09.153532 13528 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:28:09.161504  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:09.161999  6020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1836) took 1436 ms. Trace:
I20260709 12:28:09.162220  5872 raft_consensus.cc:2376] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23. Re-sending same reply.
I20260709 12:28:09.145203  5865 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:28:09.101612 13412 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:09.150141 13517 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:09.097565 13537 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:09.162408  6020 rpcz_store.cc:276] 0709 12:28:07.725441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.807835 (+ 82394us) service_pool.cc:224] Handling call
0709 12:28:07.839295 (+ 31460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:07.839559 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.839584 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:07.845713 (+  6129us) log.cc:844] Serialized 26 byte log entry
0709 12:28:08.915176 (+1069463us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:08.915236 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.915329 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.161778 (+246449us) raft_consensus.cc:1685] finished
0709 12:28:09.161801 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.161939 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1043137,"threads_started":1,"wal-append.queue_time_us":1129330}
I20260709 12:28:09.164731  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.167380  5945 raft_consensus.cc:2410] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 26 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:28:09.159373  5867 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 20.
I20260709 12:28:09.179543  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.180397  5867 raft_consensus.cc:2376] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25. Re-sending same reply.
I20260709 12:28:09.078890 13474 consensus_queue.cc:1048] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 12:28:09.172935  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:09.149868 13535 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:28:09.152456  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:09.179544  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:09.151096 13483 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.188661 13483 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:09.194356  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:09.194356  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:09.195254  5945 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:28:09.196364  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:09.196969  5945 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 25.
I20260709 12:28:09.197289  5944 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
W20260709 12:28:09.196563  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 44179.1us
W20260709 12:28:09.170550 13467 consensus_meta.cc:287] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.142s	user 0.011s	sys 0.004s
I20260709 12:28:09.056881 13403 consensus_queue.cc:237] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:09.204028  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:09.206193  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:09.213760  5867 raft_consensus.cc:2368] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23.
I20260709 12:28:09.208230  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:09.215823  5949 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.011s	sys 0.033s
I20260709 12:28:09.215067  5872 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:28:09.087590 13520 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:09.217089  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:09.221177  5872 raft_consensus.cc:2376] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22. Re-sending same reply.
W20260709 12:28:09.217875  5949 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.011s	sys 0.033s
I20260709 12:28:09.174903 13537 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:09.222548  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.227192  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.226348  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:09.225677  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:09.229352  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:09.202932 13508 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:09.203135 13412 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:09.217005 13411 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:09.241581  5866 raft_consensus.cc:1240] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:09.246421  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.314296 13508 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:09.222589 13518 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.230224 13493 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:09.170868 13272 consensus_queue.cc:237] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:09.330324  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.365473  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.208272 13467 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:09.232235 13537 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:09.222519 13423 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:28:09.313555 13547 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.065s	user 0.001s	sys 0.001s
I20260709 12:28:09.356649  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.204885 13383 consensus_queue.cc:260] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:09.382594  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:09.389341  5867 raft_consensus.cc:2376] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25. Re-sending same reply.
I20260709 12:28:09.291515 13412 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:09.357492 13147 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:09.394923  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:09.315434 13411 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:09.392552  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 179750us
I20260709 12:28:09.392151  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:09.385421 13529 log.cc:927] Time spent T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.235s	user 0.003s	sys 0.000s
I20260709 12:28:09.387656 13423 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.405105  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.406015  5872 raft_consensus.cc:2368] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23.
I20260709 12:28:09.406795  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.411415  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.413287  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.387200 13474 consensus_queue.cc:1048] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 12:28:09.416203  5867 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:09.418146  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:09.420253  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:09.423616 13531 log.cc:927] Time spent T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:28:09.426139 13500 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:09.427686  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1746) took 1211 ms. Trace:
I20260709 12:28:09.433409 13417 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:09.406709 13423 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:09.428185  6019 rpcz_store.cc:276] 0709 12:28:08.215974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.216477 (+   503us) service_pool.cc:224] Handling call
0709 12:28:08.250054 (+ 33577us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:08.432281 (+182227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:08.432324 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:08.432839 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 12:28:08.881913 (+449074us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:08.881972 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.882079 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.427382 (+545303us) raft_consensus.cc:1685] finished
0709 12:28:09.427409 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.427608 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14757,"raft.queue_time_us":185523,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":11842,"thread_start_us":582523,"threads_started":2,"wal-append.queue_time_us":766035}
I20260709 12:28:09.407086 13383 raft_consensus.cc:3060] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:09.410382 13490 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.381079 13518 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:09.444202  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.448612  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.449848  6020 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:09.449717  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:09.451432 13467 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:09.454699  6017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1743) took 1250 ms. Trace:
I20260709 12:28:09.461817  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:09.462396  6017 rpcz_store.cc:276] 0709 12:28:08.204226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.204385 (+   159us) service_pool.cc:224] Handling call
0709 12:28:08.204873 (+   488us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:08.213413 (+  8540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:08.213455 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:08.691355 (+477900us) spinlock_profiling.cc:243] Waited 8.22 ms on lock 0x7b2c001ddd28. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d157ca11 00007f39d158c372 00007f39d2cbea01 00007f39d2c8343a 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2
0709 12:28:08.692335 (+   980us) log.cc:844] Serialized 52 byte log entry
0709 12:28:09.180905 (+488570us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:09.180941 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.181003 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.442609 (+261606us) raft_consensus.cc:1685] finished
0709 12:28:09.442637 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.454614 (+ 11977us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":144814,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":128,"spinlock_wait_cycles":18906496,"thread_start_us":488146,"threads_started":1,"wal-append.queue_time_us":555225}
I20260709 12:28:09.464763  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:09.465296  5867 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:09.441326 13538 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:09.472554  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:09.477447  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:09.448834 13417 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:09.481113  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:09.482121  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:09.483788  6020 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:09.483064  6019 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 25.
I20260709 12:28:09.485472  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:09.455950 13472 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:09.456658 13532 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:09.476847 13438 consensus_meta.cc:287] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 1.359s	user 0.016s	sys 0.000s
I20260709 12:28:09.441890 13374 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:09.491852  5872 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 16.
I20260709 12:28:09.456974 13490 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:09.502702  5872 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:28:09.523597  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1459) took 1326 ms. Trace:
I20260709 12:28:09.525946 13493 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:09.526665  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1460) took 1329 ms. Trace:
I20260709 12:28:09.527313 13374 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:09.524133  5947 rpcz_store.cc:276] 0709 12:28:08.197125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.197252 (+   127us) service_pool.cc:224] Handling call
0709 12:28:08.201815 (+  4563us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:08.206325 (+  4510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:08.206376 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:08.214896 (+  8520us) log.cc:844] Serialized 26 byte log entry
0709 12:28:08.968567 (+753671us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:08.968613 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.523020 (+554407us) spinlock_profiling.cc:243] Waited 4.04 ms on lock 0x7b5400459cd4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:09.523116 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.523214 (+    98us) raft_consensus.cc:1685] finished
0709 12:28:09.523231 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.523518 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":40554,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":15097,"spinlock_wait_cycles":9293568,"thread_start_us":676665,"threads_started":1,"wal-append.queue_time_us":784409}
W20260709 12:28:09.514608 13504 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 1.042s	user 0.002s	sys 0.002s
W20260709 12:28:09.529748 13549 log.cc:927] Time spent T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 12:28:09.530148  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:09.515866 13536 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:28:09.532409  5942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1463) took 1333 ms. Trace:
I20260709 12:28:09.532823  5942 rpcz_store.cc:276] 0709 12:28:08.198753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.198944 (+   191us) service_pool.cc:224] Handling call
0709 12:28:08.218164 (+ 19220us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:08.218602 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:08.218644 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:08.227227 (+  8583us) log.cc:844] Serialized 26 byte log entry
0709 12:28:08.977480 (+750253us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:08.977525 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.977603 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.532170 (+554567us) raft_consensus.cc:1685] finished
0709 12:28:09.532193 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.532342 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40272,"raft.queue_time_us":73096,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":13568,"thread_start_us":690170,"threads_started":1,"wal-append.queue_time_us":989267}
I20260709 12:28:09.528007  6020 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:28:09.542325 13520 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:09.487088 13417 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:09.545801 13543 log.cc:927] Time spent T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:28:09.502333 13438 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:09.546711  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:09.547186  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:09.548249 13515 log.cc:927] Time spent T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.605s	user 0.000s	sys 0.005s
I20260709 12:28:09.550297  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1462) took 1352 ms. Trace:
I20260709 12:28:09.550420 13411 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:09.420001 13508 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:09.522114 13546 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 12:28:09.551630  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:09.527140  5949 rpcz_store.cc:276] 0709 12:28:08.197542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.197678 (+   136us) service_pool.cc:224] Handling call
0709 12:28:08.198018 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:08.441028 (+243010us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:08.441076 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:08.452843 (+ 11767us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.223476 (+770633us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:09.223541 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.526187 (+302646us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b5400632254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:09.526247 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.526320 (+    73us) raft_consensus.cc:1685] finished
0709 12:28:09.526336 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.526592 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100826,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":10950,"spinlock_wait_cycles":557680896,"thread_start_us":668742,"threads_started":1,"wal-append.queue_time_us":886337}
W20260709 12:28:09.516765 13545 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:28:09.553598  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54876 (request call id 1536) took 1128 ms. Trace:
I20260709 12:28:09.550801  5943 rpcz_store.cc:276] 0709 12:28:08.198294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.198443 (+   149us) service_pool.cc:224] Handling call
0709 12:28:08.218881 (+ 20438us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:28:08.219250 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:08.219279 (+    29us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:28:08.231060 (+ 11781us) log.cc:844] Serialized 78 byte log entry
0709 12:28:08.436788 (+205728us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:09.028272 (+591484us) log.cc:844] Serialized 14 byte log entry
0709 12:28:09.139570 (+111298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.139656 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:09.436227 (+296571us) spinlock_profiling.cc:243] Waited 8.02 ms on lock 0x7b1800165d60. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb642a6 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db
0709 12:28:09.547526 (+111299us) spinlock_profiling.cc:243] Waited 3.46 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:28:09.549811 (+  2285us) raft_consensus.cc:1685] finished
0709 12:28:09.549837 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:09.550213 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":115817,"prepare.queue_time_us":490641,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":5120,"raft.queue_time_us":202315,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":50242,"spinlock_wait_cycles":26386048,"thread_start_us":687372,"threads_started":2,"wal-append.queue_time_us":604099}
I20260709 12:28:09.606057  5869 rpcz_store.cc:276] 0709 12:28:08.425325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.494647 (+ 69322us) service_pool.cc:224] Handling call
0709 12:28:09.553319 (+1058672us) spinlock_profiling.cc:243] Waited 75.5 ms on lock 0x7b54002f9ed4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:09.553504 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173538688}
I20260709 12:28:09.606241  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:09.495292 13472 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:09.485890 13538 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:09.594493 13478 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 1.476s	user 0.011s	sys 0.005s
I20260709 12:28:09.501556 13532 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:09.550541 13377 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.552536  6019 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 25.
I20260709 12:28:09.610636  6016 raft_consensus.cc:2410] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 26 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 12:28:09.537469 13483 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:09.794865  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:28:09.791599  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 310547us
W20260709 12:28:09.639357  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 199111us
I20260709 12:28:09.692971  6024 raft_consensus.cc:2376] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20. Re-sending same reply.
I20260709 12:28:09.813808  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:09.807071  6019 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 25. Current term is 26. Ops: []
W20260709 12:28:09.809882  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:09.613804 13385 consensus_meta.cc:287] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.744s	user 0.015s	sys 0.000s
I20260709 12:28:09.609764 13374 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.863063  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:09.864732  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:09.868405  6022 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
W20260709 12:28:09.633827 13410 consensus_meta.cc:287] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 1.533s	user 0.008s	sys 0.008s
I20260709 12:28:09.598726 13108 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:09.604185 13438 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:09.614408 13507 consensus_meta.cc:287] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.667s	user 0.014s	sys 0.000s
I20260709 12:28:09.864228 13382 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.745280 13518 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:09.871783  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 266525us
I20260709 12:28:09.873477  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:09.884375  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1539) took 1414 ms. Trace:
W20260709 12:28:09.763073 13517 consensus_meta.cc:287] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.570s	user 0.013s	sys 0.000s
I20260709 12:28:09.638607 13470 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.893081  5868 rpcz_store.cc:276] 0709 12:28:08.470146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.004440 (+534294us) service_pool.cc:224] Handling call
0709 12:28:09.062126 (+ 57686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.072466 (+ 10340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.884062 (+811596us) spinlock_profiling.cc:243] Waited 57.3 ms on lock 0x7b54006db5d0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:09.884291 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131825024}
W20260709 12:28:09.754864 13554 log.cc:927] Time spent T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:28:09.637880 13282 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.690578 13412 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:09.787074 13455 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:09.768199 13490 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:09.896795  5840 consensus_peers.cc:597] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.791450 13552 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:09.898712  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:09.792093 13532 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:09.821847 13411 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:09.862810 13555 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.755070 13538 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:09.899333  6611 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:09.763176 13417 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:09.805979 13474 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.902491  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:09.636435 13373 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.886605 13493 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:09.816349 13472 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:09.634455 13478 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:09.889904 13403 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:09.909315  5840 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.748499 13423 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:09.895911 13384 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.638926 13377 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:09.912400 13518 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:28:09.956292  5840 leader_election.cc:336] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:09.886605 13374 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:09.956764  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 57553.4us
W20260709 12:28:09.954782  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 52411.1us
I20260709 12:28:09.883116 13385 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:09.891441 13108 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:09.895255 13410 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:09.959658  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:09.965123  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:09.980010  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:09.896481 13507 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:09.903196 13382 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:09.901911 13517 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:09.986985  5840 consensus_peers.cc:597] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.989699  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 177264us
I20260709 12:28:09.908077 13470 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:10.013806  5840 consensus_peers.cc:597] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.963277 13518 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:10.015663  5945 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 18: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:09.931506 13411 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:10.009014  5865 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 21: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:09.990723 13490 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:09.931300 13555 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:09.960345 13538 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:09.911999 13282 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:09.973105 13567 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.954109 13474 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:09.962463 13384 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:10.017890 13412 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:09.996613 13493 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:28:10.019042  5840 leader_election.cc:336] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.054870  5873 raft_consensus.cc:1240] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 24. Ops: []
W20260709 12:28:10.020463 13537 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.628s	user 0.013s	sys 0.001s
W20260709 12:28:10.014549 13558 log.cc:927] Time spent T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 12:28:09.960136 13403 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:09.955722 13373 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:10.081532  5990 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:10.110805  6610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.091084  6611 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:10.019479 13417 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:10.016520 13568 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:10.058637 13538 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.054759 13467 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.121165 13410 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:10.100337  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.102931  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:10.020519 13518 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:10.123821  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 96454.4us
W20260709 12:28:10.107684  5838 connection.cc:256] RPC call timeout handler was delayed by -1.2977543179999884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
I20260709 12:28:10.110544  5990 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:10.120903 13467 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:10.098563  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:10.109424 13493 raft_consensus.cc:697] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:10.132762 13493 consensus_queue.cc:237] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:10.135284  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 120806us
W20260709 12:28:10.136503  5840 leader_election.cc:336] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.137979  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34730 (request call id 1725) took 1197 ms. Trace:
I20260709 12:28:10.138427  5867 rpcz_store.cc:276] 0709 12:28:08.940879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.486366 (+545487us) service_pool.cc:224] Handling call
0709 12:28:09.487926 (+  1560us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:09.489859 (+  1933us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.489907 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:09.490719 (+   812us) log.cc:844] Serialized 52 byte log entry
0709 12:28:09.954114 (+463395us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:09.992464 (+ 38350us) log.cc:844] Serialized 14 byte log entry
0709 12:28:10.018023 (+ 25559us) log.cc:844] Serialized 14 byte log entry
0709 12:28:10.018440 (+   417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:10.018564 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:10.137709 (+119145us) raft_consensus.cc:1685] finished
0709 12:28:10.137733 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:10.137910 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":49918,"prepare.run_cpu_time_us":139,"prepare.run_wall_time_us":6053,"raft.queue_time_us":421,"raft.run_cpu_time_us":403,"raft.run_wall_time_us":10118,"thread_start_us":348137,"threads_started":1,"wal-append.queue_time_us":608385}
W20260709 12:28:10.128396  6609 leader_election.cc:336] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.675s (SENT)
W20260709 12:28:10.126356  5838 connection.cc:256] RPC call timeout handler was delayed by -1.2320543179998822s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1874s for a response
W20260709 12:28:10.144043  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 62578.6us
I20260709 12:28:10.144902  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:10.129544  6611 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:28:10.146167  5990 consensus_peers.cc:597] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:10.147248  5838 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:10.119514 13538 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:10.149609  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:10.151399 13561 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:28:10.147815 13490 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:10.152038 13560 log.cc:927] Time spent T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260709 12:28:10.151399 13566 log.cc:927] Time spent T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:28:10.133471 13467 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:10.088653 13567 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:10.154696  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34714 (request call id 2343) took 1010 ms. Trace:
I20260709 12:28:10.175899  5866 rpcz_store.cc:276] 0709 12:28:09.144259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.529946 (+385687us) service_pool.cc:224] Handling call
0709 12:28:09.530276 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:09.530638 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.530668 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:09.531188 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.973226 (+442038us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:09.973272 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.973347 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:10.153863 (+180516us) raft_consensus.cc:1685] finished
0709 12:28:10.153890 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:10.154557 (+   667us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11193,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"thread_start_us":441710,"threads_started":1,"wal-append.queue_time_us":585555}
I20260709 12:28:10.155745 13470 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:10.156595 13423 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:10.158100 13411 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:10.101296 13537 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:10.181910  6022 raft_consensus.cc:1240] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:10.163293 13523 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:10.161494  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:10.123358 13483 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:10.166111  5868 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:28:10.121944 13410 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:10.149226 13383 consensus_meta.cc:287] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.701s	user 0.014s	sys 0.000s
W20260709 12:28:10.145756 13562 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:28:10.139607 13438 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:10.116014 13417 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:10.137490  5840 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:10.235953  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 98467.7us
I20260709 12:28:10.180346 13373 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:10.236765  5840 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.237282  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:10.162380 13552 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:10.160336 13478 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:10.189110  5867 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 23: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:10.178377 13497 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:10.174854 13108 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:10.202169 13518 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:10.195190 13507 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:10.170053 13579 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:10.195358 13385 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:10.235620 13565 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:28:10.230489 13282 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:10.233268  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 86925.2us
I20260709 12:28:10.190719 12783 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:10.234287 13438 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.230592 13517 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:10.247843  5840 leader_election.cc:336] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.258316  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:10.247473 13502 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:10.259035  6019 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:28:10.195200 13382 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:10.230090 13569 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:28:10.245467  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1461) took 2047 ms. Trace:
I20260709 12:28:10.264782  6019 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: []
W20260709 12:28:10.246596  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 118232us
I20260709 12:28:10.254796  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:28:10.265612  6609 leader_election.cc:336] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.687s (SENT)
I20260709 12:28:10.265744  6017 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:10.255532  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:10.266592  6609 leader_election.cc:336] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.693s (SENT)
I20260709 12:28:10.266948  6023 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:28:10.267042  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:10.256168  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:10.190114 13577 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 12:28:10.236184 13377 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:10.248744 13383 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:10.236222 13578 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:10.212126 13410 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:10.247987  5874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1531) took 1980 ms. Trace:
I20260709 12:28:10.267946  6015 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 26.
I20260709 12:28:10.270308  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:10.301343  6023 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:10.256587 13438 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:10.348197  6017 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
I20260709 12:28:10.348316  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:10.348670  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:10.347740  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 81166.8us
W20260709 12:28:10.359982  6024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37112 (request call id 1721) took 1395 ms (client timeout 1503 ms). Trace:
I20260709 12:28:10.254716 12783 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:10.262672  5941 rpcz_store.cc:276] 0709 12:28:08.197861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.197979 (+   118us) service_pool.cc:224] Handling call
0709 12:28:08.219961 (+ 21982us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:09.535295 (+1315334us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b18003ad3e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39d282b4bc 00007f39cfb7948b 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d2c7b060 00007f39d2c7bbfb 00007f39d2c8664b
0709 12:28:09.535437 (+   142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.535461 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:09.535969 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.783109 (+247140us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:09.783172 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.784197 (+  1025us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:10.233456 (+449259us) raft_consensus.cc:1685] finished
0709 12:28:10.233487 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:10.245371 (+ 11884us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1584,"raft.queue_time_us":98513,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":28615,"spinlock_wait_cycles":5078400,"thread_start_us":455542,"threads_started":2,"wal-append.queue_time_us":336658}
W20260709 12:28:10.377033  6024 rpcz_store.cc:269] 0709 12:28:08.964637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.964808 (+   171us) service_pool.cc:224] Handling call
0709 12:28:10.359706 (+1394898us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b5400763754. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:10.359895 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1646057856}
I20260709 12:28:10.316713  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42730 (request call id 1842) took 1230 ms. Trace:
I20260709 12:28:10.378134  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:10.388054  5868 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:10.394846  6609 leader_election.cc:336] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.755s (SENT)
I20260709 12:28:10.282115  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:10.329337 13474 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:10.349587 13578 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.351977  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:10.402248  6018 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
W20260709 12:28:10.361366 13576 log.cc:927] Time spent T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:28:10.360790  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1543) took 1649 ms. Trace:
I20260709 12:28:10.358569  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:28:10.277858 13575 log.cc:927] Time spent T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:28:10.388604  5945 rpcz_store.cc:276] 0709 12:28:09.086606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.202172 (+115566us) service_pool.cc:224] Handling call
0709 12:28:10.316376 (+1114204us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b54003098d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:10.316619 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1800341760}
I20260709 12:28:10.280231  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:10.423434  6020 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:10.282846  6016 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 12:28:10.452147  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.378324 13502 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:10.272101 13537 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:10.359304 13523 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:10.276929 13384 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:10.276538 13403 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:10.417898  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:10.349532 13483 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:10.378011  5874 rpcz_store.cc:276] 0709 12:28:08.268028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.286168 (+ 18140us) service_pool.cc:224] Handling call
0709 12:28:08.312304 (+ 26136us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:08.697777 (+385473us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:08.773090 (+ 75313us) log.cc:844] Serialized 14 byte log entry
0709 12:28:10.088209 (+1315119us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b18001cfb40. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:28:10.140516 (+ 52307us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:10.168586 (+ 28070us) log.cc:844] Serialized 26 byte log entry
0709 12:28:10.174973 (+  6387us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:10.175023 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:10.175106 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:10.233781 (+ 58675us) raft_consensus.cc:1685] finished
0709 12:28:10.247715 (+ 13934us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:10.247915 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":46617,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":8178,"raft.queue_time_us":111052,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":6959,"spinlock_wait_cycles":5968640,"thread_start_us":436938,"threads_started":2,"wal-append.queue_time_us":1360084}
I20260709 12:28:10.418903  5873 rpcz_store.cc:276] 0709 12:28:08.711656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.028166 (+316510us) service_pool.cc:224] Handling call
0709 12:28:09.028564 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:10.098742 (+1070178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:10.360478 (+261736us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b54006f2054. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:10.360713 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2295809280}
W20260709 12:28:10.403470  6609 leader_election.cc:336] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.770s (SENT)
I20260709 12:28:10.489454  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:10.497960  5874 raft_consensus.cc:1275] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:28:10.497766 13483 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:10.498958  5870 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:10.499672  5874 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:10.500063 13483 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:10.501485  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 98080.1us
W20260709 12:28:10.502154  6609 leader_election.cc:336] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.782s (SENT)
I20260709 12:28:10.502638  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:10.497768  5869 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22. Ops: []
W20260709 12:28:10.510608  6609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.421020  5868 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:28:10.399708 13383 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.512955 13383 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:10.349416 13382 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:10.408162 13470 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.522202 13470 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:10.524091 13585 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:10.524504 13470 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:10.411885  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:10.486146 13417 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:10.526846  6609 leader_election.cc:400] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23.
I20260709 12:28:10.527313  6609 leader_election.cc:403] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:28:10.528312 13383 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:10.528541  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:28:10.529121 13383 raft_consensus.cc:3060] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:10.529668 13586 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:10.530205 13586 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:10.530504 13410 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:10.272507 13374 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:10.416795 13578 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:10.513799 13577 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:10.486440 13523 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:10.530652 13586 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:10.534332 13523 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:10.480151 13567 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:10.482035 13502 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:10.535866  5872 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:28:10.537597 13502 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:10.458017  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.500864 13518 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:10.514679 13403 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:10.544981 13417 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:10.557274 13417 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:10.559610 13417 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:10.561326 13578 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:10.549901  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:10.566636  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:10.541138 13538 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:10.573829 13282 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:10.548657 13577 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.578756 13577 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:10.583141  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:10.584378 13567 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:10.585065 13567 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:10.586465  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:10.562274  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 146512us
I20260709 12:28:10.551105  6020 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 25.
I20260709 12:28:10.564129  5870 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 26.
I20260709 12:28:10.591328  6609 leader_election.cc:400] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23.
W20260709 12:28:10.594894  5840 leader_election.cc:336] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835): Timed out: RequestConsensusVote RPC to 127.2.4.196:34835 timed out after 1.503s (SENT)
I20260709 12:28:10.595362  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:10.581328 13577 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:10.625576  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:10.626840  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:28:10.604179  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 743078us
I20260709 12:28:10.627507  6018 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:10.618351  5913 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:10.628543  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:10.629482  5949 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 21: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:10.629539  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:10.630203  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:10.623117  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:10.621022  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:10.633176  6020 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:28:10.635141  6021 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 21: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:10.635035  6016 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 22: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:10.635465  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:10.636368  5943 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:28:10.636610  5947 raft_consensus.cc:2393] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 24: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:10.642657  5943 raft_consensus.cc:1240] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:28:10.652733  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:10.653235  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:10.654120  6016 raft_consensus.cc:2393] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 26: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:10.655900  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:10.662585  5940 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:10.665323  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:10.666074  5946 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:10.665371  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:10.666908  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:10.667184  6020 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:28:10.668371  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:10.665961  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:10.671252  5941 raft_consensus.cc:2368] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24.
I20260709 12:28:10.596966 13282 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:10.615072 13278 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.603266 13578 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:10.635002 13518 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.614833 13423 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:10.681571  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 46630.4us
I20260709 12:28:10.684187 13278 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:10.685326 13588 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:10.805729  5944 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
I20260709 12:28:10.805819 13423 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:10.806479 13588 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:10.807024 13588 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:10.808156 13423 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:10.808519  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:10.809854  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:10.811002  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:10.811239  6022 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:10.808290 13588 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:10.697273 13455 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:10.607035 13438 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:10.673808 13539 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:28:10.693392 13377 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:10.703470 13432 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 12:28:10.719520 13538 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:10.731043 13382 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.658456 13577 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.073s
W20260709 12:28:10.676059  5990 leader_election.cc:336] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.744182  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:10.633971 13174 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:10.870301 13382 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:10.870494 13174 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:10.839092 13555 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:10.872869 13382 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:10.752667  6018 raft_consensus.cc:2468] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
I20260709 12:28:10.768949  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:10.774889  6609 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:10.860752  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42730 (request call id 1845) took 1460 ms. Trace:
W20260709 12:28:10.875705  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 101009us
I20260709 12:28:10.766908  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:10.877327  6016 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:28:10.879163 13532 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:10.880240 13500 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:10.880565  6609 leader_election.cc:400] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24.
I20260709 12:28:10.880719 13500 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:10.881114  6609 leader_election.cc:403] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:28:10.782078  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:10.879205 13539 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:28:10.882320 13588 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:10.804925 13278 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:10.805224 13589 log.cc:927] Time spent T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.283s	user 0.002s	sys 0.000s
I20260709 12:28:10.799458  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:10.883581 13532 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:10.883880  6023 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 26: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:10.654417 13384 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:10.794621 13578 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:10.667490 13478 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:10.822191  6611 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:10.820456 13586 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:10.882539  6024 raft_consensus.cc:2410] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:28:10.785151  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:10.872869 13438 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:10.819900 13455 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:10.881806 13500 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:10.873317 13174 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:10.797590 13518 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:10.822261 13377 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:10.923312 13382 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:10.887989 13403 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:11.022598  5865 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34714 (request call id 2341) took 1924 ms (client timeout 2027 ms). Trace:
I20260709 12:28:10.734884  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:10.742609  5868 raft_consensus.cc:3055] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:28:11.023391  5865 rpcz_store.cc:269] 0709 12:28:09.098356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.208910 (+110554us) service_pool.cc:224] Handling call
0709 12:28:11.021873 (+1812963us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b540069ee54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:11.022287 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1777306368}
I20260709 12:28:11.024528  5868 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:10.690806  5871 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:28:11.026814  5868 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:11.027699  5871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34730 (request call id 1723) took 2087 ms (client timeout 2204 ms). Trace:
W20260709 12:28:10.991991  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.741439  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:10.733680  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:10.998401  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 243058us
I20260709 12:28:10.875793  5948 rpcz_store.cc:276] 0709 12:28:09.399968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.400163 (+   195us) service_pool.cc:224] Handling call
0709 12:28:09.400575 (+   412us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:09.638603 (+238028us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:09.694847 (+ 56244us) log.cc:844] Serialized 14 byte log entry
0709 12:28:10.182978 (+488131us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b1800433900. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:28:10.314172 (+131194us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b1800433900. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb10c0a 00007f39d9250219 00007f39d6b5758f 00007f39cfb96571 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd
0709 12:28:10.618636 (+304464us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:10.619285 (+   649us) log.cc:844] Serialized 52 byte log entry
0709 12:28:10.619475 (+   190us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:10.839927 (+220452us) log.cc:844] Serialized 14 byte log entry
0709 12:28:10.860034 (+ 20107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:10.860155 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:10.860216 (+    61us) raft_consensus.cc:1685] finished
0709 12:28:10.860235 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:10.860663 (+   428us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":645184,"prepare.queue_time_us":55511,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":63,"raft.queue_time_us":611584,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":37209,"spinlock_wait_cycles":79350784,"thread_start_us":281589,"threads_started":2,"wal-append.queue_time_us":557669}
I20260709 12:28:11.030212  5866 raft_consensus.cc:2393] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 24: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:11.000773  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:11.031605  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:11.031929  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:10.743389  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:10.925329 13383 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23.
W20260709 12:28:10.910586  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 88739.5us
I20260709 12:28:10.812130  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:10.823494  5946 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
I20260709 12:28:11.009349 13417 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:11.028640  5868 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:11.031255  5840 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:28:10.859707  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 183671us
I20260709 12:28:11.032681  5872 raft_consensus.cc:2410] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:28:10.740389  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:11.022686 13382 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.030233  5869 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
I20260709 12:28:11.033721  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:10.911463 13478 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:10.898638 13538 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:10.947506 13518 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:10.920288 13578 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:11.192852  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:11.199536  5915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.001s	sys 0.013s
W20260709 12:28:11.200217  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 539058us
I20260709 12:28:10.932307 13586 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:11.028318  5871 rpcz_store.cc:269] 0709 12:28:08.940285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.996341 (+ 56056us) service_pool.cc:224] Handling call
0709 12:28:11.027246 (+2030905us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b540010f9d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c872fa 00007f39d7c48b17 00007f39d11c36d4 00007f39d11c3662 00007f39d11c3591 00007f39d11c350c 00007f39d11c27c2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:11.027506 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3849753472}
I20260709 12:28:11.084560  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:11.203661  5949 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.013s	sys 0.029s
I20260709 12:28:11.020800 13377 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:11.097010  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:11.033728 13384 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 12:28:11.131575  5866 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:28:11.097088 13537 consensus_queue.cc:1048] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 12:28:11.143064 13483 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.210286 13483 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:11.212414  5867 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 27.
W20260709 12:28:11.214570  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 217819us
I20260709 12:28:11.215891  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:11.147562 12783 consensus_meta.cc:287] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.685s	user 0.011s	sys 0.011s
I20260709 12:28:11.160271 13382 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:11.226346  5944 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 25.
W20260709 12:28:11.227375  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 69963.9us
I20260709 12:28:11.228221  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:11.229753  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:11.173256  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 142375us
I20260709 12:28:11.230705  5948 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:28:11.180016 13588 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:11.196483 13417 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:11.234362 13588 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:11.234496  6611 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:11.180917  5870 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
W20260709 12:28:11.235885  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 73826.3us
I20260709 12:28:11.153172 13540 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:11.237149 13540 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:11.238269 13590 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:11.229400  5944 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:28:11.239850 13586 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:11.240374  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:11.205413  5865 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
I20260709 12:28:11.107885 13470 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:11.201073  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:11.244481  5946 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 21: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:11.229844 13478 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:11.244895 13538 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:11.249717  5913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.017s	sys 0.006s
I20260709 12:28:11.250636  5944 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:11.250123 12783 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:11.188421  5874 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 18: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:11.237192 13417 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:11.253750  5947 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.008s	sys 0.005s
I20260709 12:28:11.240414 13586 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.256417 13586 raft_consensus.cc:3060] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:11.257987  5872 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:11.258765 13592 consensus_queue.cc:1059] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:11.260087 13502 raft_consensus.cc:3055] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:11.260637 13502 raft_consensus.cc:740] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:11.240653 13540 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:11.262223 13502 consensus_queue.cc:260] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:11.263513  5838 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:11.263896 13502 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:11.267032 13588 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:11.269953 13577 consensus_queue.cc:1059] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:11.249588 13377 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:11.238435  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:11.250415  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 590295us
W20260709 12:28:11.193038  5990 leader_election.cc:336] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:11.232144  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:11.250168  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:11.253472 12783 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:11.284412  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 91684.4us
I20260709 12:28:11.271490 13383 raft_consensus.cc:3055] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:11.295010  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:11.295485 13383 raft_consensus.cc:740] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:11.297034  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:11.302496 13578 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:11.303386 13578 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.304244 13578 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:11.283928 13384 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:11.310261  5948 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:11.310724  5990 leader_election.cc:336] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:11.311749  5990 leader_election.cc:336] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:11.312635  5990 leader_election.cc:336] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:11.314008  6611 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:11.315783  6611 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:11.317269 13539 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:11.318008 13539 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 25 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:11.319730 13539 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:11.323150 13500 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:11.332872  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 69649.7us
W20260709 12:28:11.334775  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.317487 13272 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:11.338698 13272 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.339074 13591 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:11.338759 13470 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:11.339983 13591 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:11.289547  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 158094us
I20260709 12:28:11.343179  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:11.343864  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:11.345412 13417 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:11.345889 13537 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:11.346169 13417 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:11.348014  5946 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:28:11.353499  5946 pending_rounds.cc:85] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 10
I20260709 12:28:11.248673 13538 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:11.347944 13537 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:11.348666 13417 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:11.351856  5940 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:28:11.352139  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:28:11.313529  5990 leader_election.cc:336] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:11.339257 13272 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:11.361528  5867 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:11.374459  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 122117us
I20260709 12:28:11.354031  5946 pending_rounds.cc:107] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:28:11.377194  5946 raft_consensus.cc:2889] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:11.382357  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 68943.3us
I20260709 12:28:11.386360  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:11.391040  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:11.400094  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:11.410162  6611 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:11.411734 13577 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:11.412298 13577 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.412720 13577 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:11.413828 13470 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:11.424985 13483 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:11.425882 13537 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:28:11.426353  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 73416.2us
I20260709 12:28:11.427240 13537 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:11.427759  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:11.393008  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:11.431108  5865 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 21: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:11.432621  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:28:11.434159  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.437913  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:11.396837  5872 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
I20260709 12:28:11.417212 13470 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:11.437295  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:11.438915  5870 raft_consensus.cc:3060] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:28:11.452651  5944 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 25.
W20260709 12:28:11.459396  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 57803us
I20260709 12:28:11.297127 13383 consensus_queue.cc:260] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:11.347826 13518 consensus_queue.cc:1059] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.097s
I20260709 12:28:11.478276  5948 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 26.
I20260709 12:28:11.479138 13432 raft_consensus.cc:2764] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:28:11.505786  5867 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
W20260709 12:28:11.500240  5990 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:11.513911  5990 leader_election.cc:336] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:11.515040  5990 leader_election.cc:336] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:11.519562 13590 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:11.521680 13603 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.522464 13603 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:11.532310 13455 consensus_meta.cc:287] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.603s	user 0.016s	sys 0.000s
I20260709 12:28:11.533171 13455 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:11.534960  5942 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24. Ops: []
W20260709 12:28:11.538522  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.540942  5872 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:11.542605  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:11.475497 13538 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:11.551182  5870 raft_consensus.cc:2468] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:28:11.461334  5874 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
W20260709 12:28:11.514273 13532 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.630s	user 0.011s	sys 0.012s
W20260709 12:28:11.601531  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 256412us
I20260709 12:28:11.525553 13603 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:11.633090  5915 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:11.645697  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.683076  5874 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:11.683147  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:11.686630  5868 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:28:11.687611  5870 raft_consensus.cc:1275] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:28:11.645401 13532 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329) with higher term. Message: Invalid argument: T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24.
I20260709 12:28:11.679894 13455 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:11.694227 13168 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.710103 13377 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:11.711127 13383 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:11.712003  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 130795us
I20260709 12:28:11.601367 13382 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:11.720959  5867 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:28:11.661250 13577 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:11.716967 13578 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:11.719592 13377 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.713491 13591 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:11.722908 13538 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.734398 13518 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:28:11.748728  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 336347us
I20260709 12:28:11.744035  5868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1579) took 1009 ms. Trace:
I20260709 12:28:11.754366  5868 rpcz_store.cc:276] 0709 12:28:10.734116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.742030 (+  7914us) service_pool.cc:224] Handling call
0709 12:28:10.742400 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.719516 (+977116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.743942 (+ 24426us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.745656 13377 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:11.763115  5947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2222) took 1129 ms. Trace:
W20260709 12:28:11.769608  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 325255us
W20260709 12:28:11.771298  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 171197us
W20260709 12:28:11.778208  5943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.009s	sys 0.023s
I20260709 12:28:11.787761 13577 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:11.743139 13168 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:11.791039  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 107994us
W20260709 12:28:11.778708  5943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.009s	sys 0.023s
W20260709 12:28:11.794843  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 278340us
I20260709 12:28:11.766561 13518 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 26 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:11.757474 13586 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:11.758917 13538 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:11.789546 13438 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:11.838908  5949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2219) took 1209 ms. Trace:
I20260709 12:28:11.842269  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:11.846850  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:11.853820  5945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2229) took 1038 ms. Trace:
I20260709 12:28:11.857009 13438 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:11.860008  6611 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:11.860868 13438 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:11.862342  5869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1588) took 1115 ms. Trace:
I20260709 12:28:11.842759  5949 rpcz_store.cc:276] 0709 12:28:10.629422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.637546 (+  8124us) service_pool.cc:224] Handling call
0709 12:28:10.637943 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:10.638181 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:11.332727 (+694546us) log.cc:844] Serialized 14 byte log entry
0709 12:28:11.762600 (+429873us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:11.777694 (+ 15094us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:11.809380 (+ 31686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.809497 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:11.838819 (+ 29322us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2151,"prepare.queue_time_us":588879,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":89,"raft.queue_time_us":2506,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":2205,"thread_start_us":934767,"threads_started":2,"wal-append.queue_time_us":452974}
I20260709 12:28:11.862929  5869 rpcz_store.cc:276] 0709 12:28:10.746742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.222975 (+476233us) service_pool.cc:224] Handling call
0709 12:28:11.259993 (+ 37018us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:11.260684 (+   691us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:11.260713 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:11.261240 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 12:28:11.631466 (+370226us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:11.631527 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.631631 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:11.862032 (+230401us) raft_consensus.cc:1685] finished
0709 12:28:11.862056 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:11.862254 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1888,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":124,"thread_start_us":369645,"threads_started":1,"wal-append.queue_time_us":498104}
I20260709 12:28:11.865563 13586 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:11.870505 13613 log.cc:927] Time spent T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
W20260709 12:28:11.871769 13502 consensus_meta.cc:287] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.607s	user 0.012s	sys 0.006s
W20260709 12:28:11.873351  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 55786.8us
I20260709 12:28:11.875240 13540 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:11.875765 13540 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:11.876184 13540 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:11.878211  6609 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:28:11.879184 13612 log.cc:927] Time spent T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:28:11.881117  5873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1590) took 1130 ms. Trace:
I20260709 12:28:11.882201  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2224) took 1231 ms. Trace:
I20260709 12:28:11.883019  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:11.883791  5946 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:28:11.885036  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:11.886545 13615 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:28:11.886807 13586 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:11.887235 13626 consensus_queue.cc:1059] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:11.886336 13577 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:11.882725  5943 rpcz_store.cc:276] 0709 12:28:10.650592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.650813 (+   221us) service_pool.cc:224] Handling call
0709 12:28:10.651369 (+   556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:11.270419 (+619050us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:11.270480 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:11.271166 (+   686us) log.cc:844] Serialized 26 byte log entry
0709 12:28:11.802406 (+531240us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:11.802469 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.808714 (+  6245us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:11.881926 (+ 73212us) raft_consensus.cc:1685] finished
0709 12:28:11.881949 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:11.882130 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8714,"raft.queue_time_us":51239,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"thread_start_us":1036522,"threads_started":2,"wal-append.queue_time_us":585937}
I20260709 12:28:11.894215 13518 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:11.895713 13626 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:11.896250 13626 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.896598 13626 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:11.900676 13272 consensus_meta.cc:287] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.522s	user 0.006s	sys 0.006s
I20260709 12:28:11.887444 13586 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 26 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:11.881546  5873 rpcz_store.cc:276] 0709 12:28:10.750170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.190584 (+440414us) service_pool.cc:224] Handling call
0709 12:28:11.191409 (+   825us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:11.192142 (+   733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:11.192168 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:11.193024 (+   856us) log.cc:844] Serialized 26 byte log entry
0709 12:28:11.361419 (+168395us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:11.760164 (+398745us) log.cc:844] Serialized 14 byte log entry
0709 12:28:11.762889 (+  2725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.762980 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:11.880764 (+117784us) raft_consensus.cc:1685] finished
0709 12:28:11.880787 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:11.880950 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2099,"prepare.queue_time_us":500293,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":2503,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":2300,"thread_start_us":529228,"threads_started":2,"wal-append.queue_time_us":452577}
I20260709 12:28:11.908444  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1481) took 1130 ms. Trace:
I20260709 12:28:11.767957  5947 rpcz_store.cc:276] 0709 12:28:10.630495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.659447 (+ 28952us) service_pool.cc:224] Handling call
0709 12:28:10.666615 (+  7168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:10.666857 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:11.264519 (+597662us) log.cc:844] Serialized 14 byte log entry
0709 12:28:11.728152 (+463633us) log.cc:844] Serialized 14 byte log entry
0709 12:28:11.734791 (+  6639us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:11.740937 (+  6146us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:11.750881 (+  9944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.755632 (+  4751us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:11.759799 (+  4167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60445,"prepare.queue_time_us":614424,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":114,"raft.queue_time_us":35607,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"spinlock_wait_cycles":11051904,"thread_start_us":963508,"threads_started":2,"wal-append.queue_time_us":476304}
W20260709 12:28:11.901938 13478 consensus_meta.cc:287] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.655s	user 0.011s	sys 0.007s
I20260709 12:28:11.912596  5865 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:28:11.901065 13377 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:11.878265 13382 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:11.904687 13599 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:11.902755 13438 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:28:11.931289  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 187615us
I20260709 12:28:11.854347  5945 rpcz_store.cc:276] 0709 12:28:10.698945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.331084 (+632139us) service_pool.cc:224] Handling call
0709 12:28:11.331667 (+   583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.332210 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:28:11.332774 (+   564us) log.cc:844] Serialized 14 byte log entry
0709 12:28:11.715660 (+382886us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:11.715720 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:11.731958 (+ 16238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.733700 (+  1742us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:11.736795 (+  3095us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":895,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":6815,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":11576,"thread_start_us":379635,"threads_started":1,"wal-append.queue_time_us":446158}
I20260709 12:28:11.901379 13272 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:11.915174 13478 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:11.957944  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:11.994771 13538 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:11.995754 13616 log.cc:927] Time spent T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:28:11.944628 13599 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:11.947264 13438 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:12.017027  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 175643us
I20260709 12:28:12.028079  5941 raft_consensus.cc:1240] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:12.030452  5941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1483) took 1023 ms. Trace:
I20260709 12:28:12.033700 13478 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:12.035993 13639 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.038300  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.040227  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:12.042023  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.043411  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.044485  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.044917 13438 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:12.046139  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.047539  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:12.048409  5944 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 26: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:28:12.050055  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.051528  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:12.052263  5869 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 17.
I20260709 12:28:12.051623 13599 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:12.054921 13635 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:28:12.056582  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1486) took 1047 ms. Trace:
I20260709 12:28:12.057168  5948 rpcz_store.cc:276] 0709 12:28:11.008981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.479333 (+470352us) service_pool.cc:224] Handling call
0709 12:28:11.479922 (+   589us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:11.509134 (+ 29212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:11.509166 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:11.586816 (+ 77650us) log.cc:844] Serialized 26 byte log entry
0709 12:28:11.922491 (+335675us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:11.922541 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.922619 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:12.055652 (+133033us) raft_consensus.cc:1685] finished
0709 12:28:12.055673 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:12.055824 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9486,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":152,"thread_start_us":244847,"threads_started":1,"wal-append.queue_time_us":368233}
I20260709 12:28:12.053318  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:12.059813  5868 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:12.062692  5947 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:12.067979 13578 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:11.954227 13586 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:12.041455  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:12.045612  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.071802  5943 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:28:12.049381  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:12.073510  5840 leader_election.cc:336] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.594s (SENT)
I20260709 12:28:12.078429  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:12.078872 13417 raft_consensus.cc:3055] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:12.079640 13417 raft_consensus.cc:740] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:28:12.080215 13623 log.cc:927] Time spent T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:28:12.080567  5913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:12.081562  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.081840 13470 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:12.080910 13417 consensus_queue.cc:260] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:12.084071  5840 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.2.4.196:34835 timed out after 1.597s (SENT) [suppressed 91 similar messages]
W20260709 12:28:12.084790  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.086902 13438 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.087510 13438 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:12.091476  5868 raft_consensus.cc:2468] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:28:12.030987  5941 rpcz_store.cc:276] 0709 12:28:11.007318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.436295 (+428977us) service_pool.cc:224] Handling call
0709 12:28:11.436781 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:12.029005 (+592224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:12.030034 (+  1029us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b5400632254. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:12.030350 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1265437056}
I20260709 12:28:12.018882 13538 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:12.099311  5913 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:12.036871 13639 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:12.101740  5942 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
W20260709 12:28:12.102720  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 75643.7us
I20260709 12:28:12.105052 13438 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:12.105497 13630 consensus_queue.cc:1059] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:12.106616 13630 raft_consensus.cc:3055] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:12.106974 13630 raft_consensus.cc:740] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:12.107688 13639 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:12.109066  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.109102  5838 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:12.111545 13502 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:12.112155 13502 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.112668 13502 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:12.098004 13538 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:12.117959 13538 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:12.113243 13591 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:28:12.107899 13630 consensus_queue.cc:260] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:12.121299  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 267363us
I20260709 12:28:12.128520  5946 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
W20260709 12:28:12.159365  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 68607.9us
W20260709 12:28:12.152486  5840 leader_election.cc:336] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Timed out: RequestConsensusVote RPC to 127.2.4.193:38701 timed out after 1.653s (SENT)
I20260709 12:28:12.165365  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:28:12.162315  5915 connection.cc:256] RPC call timeout handler was delayed by -1.5433305850001489s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
I20260709 12:28:12.179914  6609 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:28:12.186362  5915 connection.cc:256] RPC call timeout handler was delayed by -1.4731305850000354s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
W20260709 12:28:12.190353  5915 connection.cc:256] RPC call timeout handler was delayed by -1.3975305849999131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1675s for a response
W20260709 12:28:12.190843  5915 connection.cc:256] RPC call timeout handler was delayed by -1.3885305849998986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1685s for a response
W20260709 12:28:12.191258  5915 connection.cc:256] RPC call timeout handler was delayed by -1.3273305850002544s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1753s for a response
W20260709 12:28:12.191640  5915 connection.cc:256] RPC call timeout handler was delayed by -1.3084305850002238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
W20260709 12:28:12.192010  5915 connection.cc:256] RPC call timeout handler was delayed by -1.0987305849998847s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20070000000000002s for a response
W20260709 12:28:12.192407  5915 connection.cc:256] RPC call timeout handler was delayed by -1.0492305850002595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2062s for a response
W20260709 12:28:12.195171  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 348468us
I20260709 12:28:12.197000 13500 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.197275 13382 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:12.189110 13455 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:28:12.198120 13382 raft_consensus.cc:697] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:12.198809 13630 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:12.197821 13500 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:12.199271 13382 consensus_queue.cc:237] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:12.201476 13500 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:12.161442 13501 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:12.208676 13501 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:12.211930 13501 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:12.166513 13168 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:12.178191 13478 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:12.221489  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:12.229116  5941 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:28:12.231848 13637 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:12.232878 13637 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:12.235818 13637 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:12.236399 13417 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:12.240191  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.239631 13478 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:12.243279 13478 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:12.248237 13567 raft_consensus.cc:3055] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:12.248750 13567 raft_consensus.cc:740] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:12.250800  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:12.251754  6019 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:12.226528  5913 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:12.198925 13403 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.218242 13377 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:12.268266  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 51612.8us
I20260709 12:28:12.237479  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:12.220362 13272 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:12.282505  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 64759us
I20260709 12:28:12.283226  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:12.284134  6021 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:12.284627  6611 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:12.288440 13501 consensus_queue.cc:1048] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:12.290349 13501 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:12.290896 13641 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:12.291599 13641 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:12.292816  6018 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:28:12.294134  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:12.297571  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:12.294420  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:12.303995  6015 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 25: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:12.304700  6020 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 18: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:12.282624  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:12.311996  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 193235us
I20260709 12:28:12.294917 13603 raft_consensus.cc:2804] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:12.306387  6018 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:28:12.304342 13377 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:12.354625  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:12.358733  6015 raft_consensus.cc:2410] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 29 because replica has last-logged OpId of term: 28 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
W20260709 12:28:12.352928  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 107284us
I20260709 12:28:12.363457  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:12.297668 13403 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:12.375314 13588 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:12.375890  6018 raft_consensus.cc:2393] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 20: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:12.351017 13603 raft_consensus.cc:697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:28:12.383306  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 157420us
I20260709 12:28:12.294914 13567 consensus_queue.cc:260] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:12.394841  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:12.370523 13641 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:12.393141  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 105238us
I20260709 12:28:12.405776  5913 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:28:12.409184  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 256736us
I20260709 12:28:12.386937 13377 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:12.413498  6609 leader_election.cc:336] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.570s (SENT)
I20260709 12:28:12.422847  5941 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:28:12.439399 13649 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:12.439723 13626 consensus_meta.cc:287] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.543s	user 0.008s	sys 0.006s
I20260709 12:28:12.440325 13649 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:12.441087 13603 consensus_queue.cc:237] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:12.442819 13649 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:12.440315 13626 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:12.444808 13641 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:12.445297 13641 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.445911 13626 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:12.447701  5840 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:12.448288 13617 log.cc:927] Time spent T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.616s	user 0.002s	sys 0.002s
I20260709 12:28:12.450721 13655 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:12.454536  6019 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:28:12.457906  5913 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:12.462414  6021 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:28:12.421975 13599 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:12.463588 13599 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:12.463864 13501 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:12.464457 13599 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:12.464438  5913 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:12.465956 13501 raft_consensus.cc:697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:28:12.490805  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 103964us
I20260709 12:28:12.498821  6015 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 12:28:12.493391  5915 consensus_peers.cc:597] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:12.461608 13641 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:12.427616 13377 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:12.428555 13502 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:12.528162  6022 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:12.528901  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:12.469455 13501 consensus_queue.cc:237] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:12.539661  5865 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:28:12.540130  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:12.544624  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:12.545871  5868 raft_consensus.cc:2410] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
W20260709 12:28:12.545337  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 92399.3us
W20260709 12:28:12.550458  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:12.523070 13383 consensus_meta.cc:287] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.795s	user 0.017s	sys 0.000s
W20260709 12:28:12.575752  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 178818us
I20260709 12:28:12.571679  5987 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:12.574440  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:12.580171  5867 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 22: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:28:12.570290  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 51346.8us
I20260709 12:28:12.538877 13168 raft_consensus.cc:2804] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:28:12.586848  6609 leader_election.cc:336] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.627s (SENT)
W20260709 12:28:12.589448  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 141876us
W20260709 12:28:12.592964  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 50400us
I20260709 12:28:12.597060 13417 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:12.597769 13417 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:12.587685  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:12.579022  5990 leader_election.cc:336] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:12.601306  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 53733.6us
W20260709 12:28:12.602015  5915 leader_election.cc:336] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.603853  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.611055  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:12.610405  5913 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:12.640758  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.641191  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:12.643713  5942 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:28:12.647384  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.648779  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.649989  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:12.653020 13652 log.cc:927] Time spent T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:28:12.601457 13168 raft_consensus.cc:697] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:12.647315 13272 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:12.668036  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.665413  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:12.666744  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.670960  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.658661 13478 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:12.677022  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:12.686028  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 108149us
W20260709 12:28:12.683068  5987 connection.cc:256] RPC call timeout handler was delayed by -1.0683601449995876s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
I20260709 12:28:12.683241 13518 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.683852 13438 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:12.684577  5866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1592) took 1689 ms. Trace:
W20260709 12:28:12.685964  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.686214 13403 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:12.677870  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:12.668478  5990 consensus_peers.cc:597] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:12.694231  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.695681  5941 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 25.
I20260709 12:28:12.697199  5946 raft_consensus.cc:2393] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
W20260709 12:28:12.698945  5990 leader_election.cc:336] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:12.704062 13647 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.268s	user 0.002s	sys 0.000s
I20260709 12:28:12.705868 13678 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:12.706652 13678 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:12.711174 13668 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:12.690169 13478 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.723140  5873 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:12.693365  5866 rpcz_store.cc:276] 0709 12:28:10.995199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.210319 (+215120us) service_pool.cc:224] Handling call
0709 12:28:11.254025 (+ 43706us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:11.260356 (+  6331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:11.260391 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:11.261188 (+   797us) log.cc:844] Serialized 26 byte log entry
0709 12:28:11.601298 (+340110us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:11.601349 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.601429 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:11.998606 (+397177us) raft_consensus.cc:1685] finished
0709 12:28:11.998644 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:12.684215 (+685571us) spinlock_profiling.cc:243] Waited 43.1 ms on lock 0x7b5400186d50. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:12.684482 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5653,"raft.queue_time_us":7623,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":178,"spinlock_wait_cycles":99212544,"thread_start_us":339322,"threads_started":1,"wal-append.queue_time_us":496506}
W20260709 12:28:12.728060  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 141332us
I20260709 12:28:12.728597 13168 consensus_queue.cc:237] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:12.729261  5915 leader_election.cc:336] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.730057  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:28:12.731606  5915 consensus_peers.cc:597] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:12.732365  5915 consensus_peers.cc:597] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:12.733579  5915 leader_election.cc:336] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.735140  5987 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:12.735157  5915 leader_election.cc:336] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:12.737360  5915 leader_election.cc:336] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.700241 13518 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:12.737442  5990 leader_election.cc:336] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.737987  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:12.668124 13417 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:12.743057  5987 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:12.744038 13518 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:12.746667  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 85797.5us
I20260709 12:28:12.745208 13537 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:12.724622 13540 consensus_meta.cc:287] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.848s	user 0.008s	sys 0.009s
W20260709 12:28:12.750411  5990 leader_election.cc:336] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.729172 13658 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:12.753369  5990 leader_election.cc:336] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.727342 13478 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:12.757995  5990 leader_election.cc:336] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.708963 13438 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:12.767752  5990 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:12.779587  5990 leader_election.cc:336] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.725076 13599 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:12.746985 13641 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:12.782341  6609 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.728s (SENT)
I20260709 12:28:12.757596 13540 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:12.786876 13540 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:12.787721  5942 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 26: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:12.788157  5840 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:12.771338 13518 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:12.792186  6609 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:12.775317 13403 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.778102 13377 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:12.778671 13502 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:12.798437 13668 raft_consensus.cc:2804] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:28:12.808137  5990 leader_election.cc:336] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:12.803867 13630 consensus_meta.cc:287] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.604s	user 0.013s	sys 0.002s
I20260709 12:28:12.798157 13540 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:12.818470  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 81172.9us
I20260709 12:28:12.804854 13626 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.802739 13588 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:12.817334 13377 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:12.833544 13438 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:11.912708  5940 rpcz_store.cc:276] 0709 12:28:10.778425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.393782 (+615357us) service_pool.cc:224] Handling call
0709 12:28:11.488236 (+ 94454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.545283 (+ 57047us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:11.607991 (+ 62708us) log.cc:844] Serialized 14 byte log entry
0709 12:28:11.906054 (+298063us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:11.906115 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:11.906162 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.906231 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:11.908365 (+  2134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45653,"prepare.queue_time_us":53764,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":16484,"raft.queue_time_us":3465,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":4872,"thread_start_us":180839,"threads_started":1}
W20260709 12:28:12.838958 13671 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:28:12.826248 13626 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:12.847106 13626 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:12.848704  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:12.850042  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:12.851090  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:12.851437  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:12.854012  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:12.855279  6020 raft_consensus.cc:2410] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:28:12.855281  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:12.878260  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 70327.4us
W20260709 12:28:12.880327  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.881479  6024 raft_consensus.cc:2393] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 24: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:12.882829  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:12.883527  6019 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:12.883854  5990 leader_election.cc:336] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.809020 13403 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:12.824672 13383 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.902993  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:28:12.902904  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 151161us
I20260709 12:28:12.887104  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:12.905304  6020 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:12.822906 13668 raft_consensus.cc:697] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 24 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:28:12.891587  5990 leader_election.cc:336] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.898806  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:12.899542 13403 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:12.902375  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 129252us
I20260709 12:28:12.914494  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:12.909935  5915 leader_election.cc:336] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.915812  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:12.901582  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:12.898087  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:12.912956  5941 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:28:12.913707  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:12.914060  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:12.892498  5840 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:28:12.917599  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 174657us
I20260709 12:28:12.922098  5873 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:12.922448  5868 raft_consensus.cc:2368] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24.
I20260709 12:28:12.922794  5987 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:12.889528 13538 consensus_meta.cc:287] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: Time spent flushing consensus metadata: real 0.769s	user 0.013s	sys 0.003s
W20260709 12:28:12.920411  5990 leader_election.cc:336] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.924831  6019 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 27.
I20260709 12:28:12.926574  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:12.927466  5868 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:12.928980  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:12.929768  5869 raft_consensus.cc:2468] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 18.
I20260709 12:28:12.931799  6016 raft_consensus.cc:2393] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 25: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:28:12.932371  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.826342 13588 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:12.932953  6611 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.942478  5940 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:28:12.944190  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:12.944998  6024 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:12.948472 13688 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:12.949891 13668 consensus_queue.cc:237] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:12.949376 13688 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:12.954217 13599 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:12.955154 13599 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:12.920156 13383 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:12.955319 13688 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:12.957798  6609 leader_election.cc:336] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Timed out: RequestConsensusVote RPC to 127.2.4.194:38005 timed out after 1.733s (SENT)
I20260709 12:28:12.958134 13383 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:12.958389  6609 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:12.966900 13403 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:12.959431  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:12.970018  5869 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:28:12.969301  5868 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25.
W20260709 12:28:12.962239  5990 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.975713  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:12.980645  5865 raft_consensus.cc:1240] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:28:12.982950  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:12.983971  6019 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:28:12.984905  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:12.976222  5990 leader_election.cc:336] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.934265  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:13.012635  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:13.013386  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:13.020251  5868 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:28:13.020510  5866 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:13.014189  6016 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:13.023166  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:12.931043 13538 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:13.026805  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:13.039242  5867 raft_consensus.cc:2410] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:28:13.028452  6611 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.030586 13588 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:13.031801  6016 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:28:12.990923 13383 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.009212 13403 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.036559  5873 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
I20260709 12:28:13.046340  6020 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:28:13.055091  6611 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.061659 13438 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:13.062006  6609 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:28:13.061786  6015 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:28:13.064483 13538 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:13.065922  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 143253us
I20260709 12:28:13.066854  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.068243  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.069574 13648 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:13.069656 13403 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:13.070550 13648 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:13.072090 13403 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:13.064617  6017 raft_consensus.cc:1240] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:13.078629  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:13.081036  6016 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 20.
W20260709 12:28:13.084510  5990 leader_election.cc:336] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:13.069574  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.091840 13500 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:13.092413 13500 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:13.093925  5940 raft_consensus.cc:1240] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:13.093328 13500 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:13.049921 13626 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:13.102617  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:13.105531  5942 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:13.106281  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 51318.4us
I20260709 12:28:13.113229  6611 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:13.114820  6611 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:28:13.125444  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 233008us
W20260709 12:28:13.125913  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 64028.8us
I20260709 12:28:13.117444  6024 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
I20260709 12:28:13.094377 13383 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:13.103641  5869 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 27.
I20260709 12:28:13.106324 13630 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:13.131660  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 62204us
I20260709 12:28:13.108624 13403 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:28:13.136902  6608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.117112 13538 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:13.126119 13377 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:13.127308 13626 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:13.155092  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.168481  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:13.138015 13403 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.128836 13645 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:28:13.157477 13626 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.133298 13591 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:28:13.160871 13692 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:13.106130 13689 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
W20260709 12:28:13.180084  5840 connection.cc:256] RPC call timeout handler was delayed by -1.0402805710000393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
I20260709 12:28:13.184357  5942 raft_consensus.cc:2468] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 26.
W20260709 12:28:13.191476  5840 connection.cc:256] RPC call timeout handler was delayed by -1.0293116100001498s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16820000000000002s for a response
I20260709 12:28:13.187119  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:13.192504  6020 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
W20260709 12:28:13.175606  5990 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:13.154840 13478 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:13.169479 13377 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 27 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:13.173990 13403 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:13.198695  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.199929  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:13.201356  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 316163us
I20260709 12:28:13.204901  5942 raft_consensus.cc:2393] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 23: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:13.165663 13577 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:28:13.181131 13626 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:13.212852  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:13.214252  6020 raft_consensus.cc:2393] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 24: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:28:13.227504  5949 raft_consensus.cc:2393] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 22: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:13.229341 13377 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:13.229909 13538 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.230973  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:13.232031 13537 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:13.232393  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.232703 13537 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:13.233314 13537 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:13.234419 13478 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:13.235342 13417 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:13.235836 13645 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:13.237861  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.238752 13417 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:13.245364 13383 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:13.245697 13648 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:13.246496 13567 raft_consensus.cc:3060] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:13.248804  5987 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.230727 13538 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:13.251585 13678 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:13.251812 13648 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:13.252672  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.254899  5987 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.256642  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.257678  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:13.266732  5915 leader_election.cc:336] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:13.268615  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.270491  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.251610 13538 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:13.259500 13678 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:13.258153 13648 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:13.287940  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:13.298691  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:13.299769 13648 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:13.300279  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:13.300510 13648 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.301038 13648 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:13.261252 13692 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:13.285385 13518 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:13.314724 13417 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:13.282858 13641 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:13.284968 13639 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:13.330066  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 74374.7us
W20260709 12:28:13.335093  5990 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:13.343935  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 209097us
I20260709 12:28:13.344889  6611 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:13.345981 13630 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:13.330948 13537 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:13.346433 13630 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.346870 13630 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:13.329576 13403 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:13.348580 13537 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:13.341385  5866 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
W20260709 12:28:13.353029  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 118171us
I20260709 12:28:13.350608  6611 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:13.363221 13539 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:13.390791  5942 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:28:13.394845  5942 raft_consensus.cc:1240] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 23. Ops: []
I20260709 12:28:13.400568  5942 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:13.403909  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.404538 13668 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.405150 13668 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:13.407706 13668 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:13.409845  5915 leader_election.cc:336] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:13.411360 13539 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 24 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:13.412451 13539 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:13.415448  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.419834 13703 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:13.422864 13403 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:13.428727  5915 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:13.429445  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:13.429816 13689 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:13.430832  6609 leader_election.cc:400] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for earlier term 23. Current term is 24.
I20260709 12:28:13.432226  6609 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:28:13.439981  6609 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:28:13.443758  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 69256.5us
W20260709 12:28:13.444772  5915 leader_election.cc:336] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:13.425426  5987 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:13.449672  5990 consensus_peers.cc:597] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:13.453513  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:13.454739  5869 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:13.452401  5838 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:13.467979  5872 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:28:13.468614  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:13.469800  5866 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:28:13.476058  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 219547us
I20260709 12:28:13.476694  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:13.482376  5915 leader_election.cc:336] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:13.484026  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:13.478834  5942 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:28:13.450956 13272 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:13.485347  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.461643 13645 raft_consensus.cc:2804] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:13.461071 13692 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:13.474773 13657 raft_consensus.cc:2804] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:28:13.496861  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.496937  5873 raft_consensus.cc:1275] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:28:13.498147 13455 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:13.496304  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:13.519407  5867 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:13.505342  5869 raft_consensus.cc:2468] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 18.
I20260709 12:28:13.505998 13383 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:13.499603 13501 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
W20260709 12:28:13.506184 13687 log.cc:927] Time spent T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.426s	user 0.000s	sys 0.007s
I20260709 12:28:13.506469 13272 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 25 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:13.486865 13690 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:13.523509 13690 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:13.523878 13272 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:13.507397 13709 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.525952 13657 raft_consensus.cc:697] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 26 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:13.526177 13709 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:13.527244 13657 consensus_queue.cc:237] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:13.530489 13626 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:13.488005 13577 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:13.517066  6609 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:28:13.487018 13698 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.489781 13403 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:13.461335 13588 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:28:13.492242 13645 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:13.516140 13693 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:13.529441 13709 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:13.476965 13537 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:28:13.543826  5913 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:13.546103  5943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35204 (request call id 1495) took 1471 ms. Trace:
I20260709 12:28:13.492826 13641 consensus_queue.cc:1059] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
W20260709 12:28:13.562857  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 45821.2us
W20260709 12:28:13.552424  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 79941.8us
W20260709 12:28:13.545179  5840 leader_election.cc:336] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:13.547729 13501 consensus_queue.cc:1048] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 12:28:13.548763 13645 raft_consensus.cc:3060] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:13.542660 13698 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:13.564124  5943 rpcz_store.cc:276] 0709 12:28:12.074831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.079435 (+  4604us) service_pool.cc:224] Handling call
0709 12:28:12.079949 (+   514us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:12.085681 (+  5732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:12.085711 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:12.086470 (+   759us) log.cc:844] Serialized 52 byte log entry
0709 12:28:12.316891 (+230421us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:12.360484 (+ 43593us) log.cc:844] Serialized 14 byte log entry
0709 12:28:13.032420 (+671936us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b180040d020. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb96522 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:28:13.444235 (+411815us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b180040d020. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d2c7b060 00007f39d2c8d716 00007f39d2c9e287 00007f39d2c9e208
0709 12:28:13.544865 (+100630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.545710 (+   845us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:13.545782 (+    72us) raft_consensus.cc:1685] finished
0709 12:28:13.545804 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:13.546009 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5888,"prepare.queue_time_us":41726,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":38655,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":1097,"spinlock_wait_cycles":12002816,"thread_start_us":286419,"threads_started":2,"wal-append.queue_time_us":309561}
I20260709 12:28:13.575392  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:13.577461  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:13.578526  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:13.588750  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:13.614096  5873 raft_consensus.cc:2468] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 27.
I20260709 12:28:13.610926  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:13.614949  5942 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:28:13.622936  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:13.614462  5868 raft_consensus.cc:2376] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 25. Re-sending same reply.
I20260709 12:28:13.622612  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:13.627074  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:13.572247 13695 raft_consensus.cc:2804] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:13.627110  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:13.629112  5945 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 19.
I20260709 12:28:13.631930  5870 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:28:13.612671  5872 raft_consensus.cc:2376] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 27. Re-sending same reply.
I20260709 12:28:13.578676 13681 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:13.641119 13712 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:28:13.642889 13696 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:13.647462  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:13.649928  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3145d5d96e41518c7bf41999289ac3" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:13.650894  5943 raft_consensus.cc:2410] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:28:13.652345  5943 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:28:13.653748 13695 raft_consensus.cc:697] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:13.654870 13695 consensus_queue.cc:237] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:13.650833  5941 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:28:13.581928 13383 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:13.645326  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:13.648927  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:13.662495  5838 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:13.636906  6609 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:28:13.674993  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:13.676895  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:13.676173  5867 raft_consensus.cc:2468] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:13.634980 13690 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:13.649278 13630 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:13.686141  5913 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:13.581400 13648 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:13.695945  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:13.669749 13403 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.701035  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:13.671978 13383 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:28:13.704381  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 159199us
I20260709 12:28:13.711913  5866 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:13.695056 13694 raft_consensus.cc:2804] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:13.714264 13694 raft_consensus.cc:697] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:13.714721 13630 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:13.685839 13567 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:13.715364 13694 consensus_queue.cc:237] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:13.737936  5840 leader_election.cc:336] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:13.738576  5840 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:13.740056  5840 leader_election.cc:336] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:13.740545  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:13.707825 13690 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:13.736126 13698 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:13.761734  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 99363.4us
I20260709 12:28:13.708761 13403 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:13.750531 13591 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:13.770368 13539 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:13.754562 13672 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:13.795701  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.836035  5866 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:28:13.735919 13383 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:13.863394  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 245238us
I20260709 12:28:13.789830 13567 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:13.886621 13588 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:13.864977 13688 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:13.904418  5915 leader_election.cc:336] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:13.838347 13672 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:13.908797  5915 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:13.911741 13715 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 12:28:13.919651 13417 raft_consensus.cc:3055] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:13.925364 13648 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:13.932888  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:13.941920  5840 leader_election.cc:336] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:13.944655  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:13.945703  5947 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:13.946166  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 202573us
I20260709 12:28:13.946786  5840 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:13.934890 13694 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:13.948259  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:13.936548 13417 raft_consensus.cc:740] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:13.903720 13688 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:13.953935 13723 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:13.958067 13630 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:13.970112 13688 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:13.970316 13645 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:13.972332  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 80246us
I20260709 12:28:13.950915 13567 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:13.971997 13630 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.990559 13630 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:13.971376 13403 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:13.997944  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 60303.6us
W20260709 12:28:14.001415  5915 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:14.012005 13672 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:14.014626  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 455183us
W20260709 12:28:14.015172  5840 leader_election.cc:336] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.015862  5840 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:14.021080  5947 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 26.
I20260709 12:28:13.967569 13729 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
W20260709 12:28:14.023564  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 63855.8us
W20260709 12:28:14.031833 13719 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.051s	user 0.000s	sys 0.002s
I20260709 12:28:14.035323  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:14.036048  5867 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:14.024657 13672 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:14.021549 13417 consensus_queue.cc:260] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:14.042279  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:14.053298  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:14.054577  5947 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:28:14.062546  6609 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:28:14.026217 13500 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:14.064841  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:14.064842  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 48618.5us
W20260709 12:28:14.070042  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.070307  5867 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:28:14.074083  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:14.081724  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:14.084261  5946 raft_consensus.cc:2393] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 26: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:28:14.084503  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 69396.8us
I20260709 12:28:14.077248 13709 raft_consensus.cc:2804] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:14.102319 13709 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:14.102761 13709 raft_consensus.cc:3060] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:14.103046  6019 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:28:14.088984  5940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2242) took 1230 ms. Trace:
I20260709 12:28:14.105799  5940 rpcz_store.cc:276] 0709 12:28:12.858995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.943476 (+ 84481us) service_pool.cc:224] Handling call
0709 12:28:13.088461 (+144985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:13.099941 (+ 11480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.088629 (+988688us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400467dd0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c828ad 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6 00007f39d0fbd26f
0709 12:28:14.088884 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332553472}
I20260709 12:28:14.108863 13577 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.108685 13688 consensus_queue.cc:1059] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:14.109510 13577 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:14.110023 13688 raft_consensus.cc:3055] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:14.110550 13688 raft_consensus.cc:740] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:14.095536 13645 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:14.085700 13417 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:28:14.113308  5866 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:28:14.111719 13688 consensus_queue.cc:260] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:14.118716  6016 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:28:14.119089  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:14.118716  6022 raft_consensus.cc:1275] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:28:14.123728 13688 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:14.124541 13577 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:14.130604  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:14.142316  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:14.147035  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:14.154212  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:14.101478 13630 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:14.160287  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:14.167320  5873 raft_consensus.cc:2368] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22.
I20260709 12:28:14.168314  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:14.153342 13540 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:28:14.170655  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:14.171840  5869 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:28:14.170061  6609 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:14.169588  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:14.174479  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:14.174551  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:14.174913  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:14.175339  5874 raft_consensus.cc:2393] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 18: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:14.177615 13539 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:14.177886 13403 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:14.182691 13577 consensus_queue.cc:1059] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:14.183943 13403 raft_consensus.cc:3055] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:14.184348 13403 raft_consensus.cc:740] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:14.172739  5945 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 25. Current term is 26. Ops: [23.14-25.15]
I20260709 12:28:14.185248 13403 consensus_queue.cc:260] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:14.186657  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:14.188158  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:14.189673  5945 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:28:14.190224 13577 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:14.172762  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:14.191080  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:14.190037  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:14.191905  5945 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:28:14.192193  6023 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:14.188198  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:14.193020  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:14.172663 13630 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:14.163659  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:14.194968  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:14.197010  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:14.196947  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:14.199968  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:14.204756  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.208978  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:14.193260  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:14.211292  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:14.223964  5867 raft_consensus.cc:2393] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 25: Already voted for candidate 1fc57407d56241b38a94d543e073f8af in this term.
I20260709 12:28:14.162369 13709 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:14.213694  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:28:14.206131  5990 consensus_peers.cc:597] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:14.215467  5947 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:28:14.221077  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:14.228216  5838 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:14.217600 13641 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 12:28:14.230274 13168 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:14.233418 13502 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:14.234067 13502 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:14.234537 13502 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:14.221441 13726 log.cc:927] Time spent T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.299s	user 0.002s	sys 0.002s
I20260709 12:28:14.237272  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:14.238140  6019 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:14.232096  5867 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
W20260709 12:28:14.221778 13721 log.cc:927] Time spent T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.299s	user 0.003s	sys 0.001s
I20260709 12:28:14.252831  6023 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
W20260709 12:28:14.222924 13702 log.cc:927] Time spent T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.896s	user 0.004s	sys 0.008s
W20260709 12:28:14.253554  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 106643us
I20260709 12:28:14.254379  6019 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 21.
I20260709 12:28:14.255277  5869 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:28:14.259939  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:14.255540  5865 raft_consensus.cc:1240] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 23. Ops: []
W20260709 12:28:14.222584 13727 log.cc:927] Time spent T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.299s	user 0.004s	sys 0.000s
I20260709 12:28:14.262019  6017 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 12:28:14.257256  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 64342.2us
I20260709 12:28:14.267288  5873 raft_consensus.cc:1240] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:14.236629 13537 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:14.273201  6023 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:28:14.277508  5869 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:28:14.254767 13672 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:14.230289 13668 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:28:14.257664 13733 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.283053  6609 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:14.283758  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:14.260757 13377 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.257743 13738 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.270219 13729 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:14.291898  5868 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:28:14.292865  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:28:14.295476  5838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.297919  5874 raft_consensus.cc:2410] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:28:14.284637 13672 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:14.284804 13537 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 26 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:14.287631 13630 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.312467  6017 raft_consensus.cc:1240] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:14.299721 13577 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:14.301506 13685 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:14.322615  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:14.323616  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:14.307034 13672 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:14.290869 13733 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:14.294309 13377 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:14.311089 13729 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.345088 13729 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:14.345664  6024 raft_consensus.cc:1275] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:28:14.337817  5948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42740 (request call id 2243) took 1452 ms. Trace:
I20260709 12:28:14.339881  6017 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:28:14.343508  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:14.349197  6019 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
I20260709 12:28:14.336602 13577 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:14.357453  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:14.346760  5948 rpcz_store.cc:276] 0709 12:28:12.885309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.972230 (+ 86921us) service_pool.cc:224] Handling call
0709 12:28:12.972713 (+   483us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:12.973228 (+   515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:12.973261 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:12.990113 (+ 16852us) log.cc:844] Serialized 52 byte log entry
0709 12:28:13.208378 (+218265us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:13.399833 (+191455us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b1800380ce0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39d282b4bc 00007f39cfb7948b 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d667b3c7 00007f39d2c8d9a7 00007f39d2c9d0bf
0709 12:28:13.400029 (+   196us) log.cc:844] Serialized 14 byte log entry
0709 12:28:13.400303 (+   274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.400398 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:14.176358 (+775960us) spinlock_profiling.cc:243] Waited 7.15 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:28:14.337536 (+161178us) raft_consensus.cc:1685] finished
0709 12:28:14.337558 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.337753 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38518,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":4366,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":7962,"spinlock_wait_cycles":19235968,"thread_start_us":300375,"threads_started":2,"wal-append.queue_time_us":299491}
W20260709 12:28:14.361722  5840 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.325958 13685 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:14.362669  5987 leader_election.cc:400] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22.
I20260709 12:28:14.299603 13738 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:14.363822  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:14.324456 13630 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:14.374393  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:14.373440 13540 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:28:14.395852 13733 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:14.396723 13709 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 28 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:14.365870 13685 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:14.381886  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:14.403581  5915 leader_election.cc:336] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.406105  5915 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:14.406781 13729 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:14.403489 13537 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:14.411990 13689 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:14.413733  5915 leader_election.cc:336] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.414253  5915 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:14.377986 13747 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:14.398557 13739 log.cc:927] Time spent T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:28:14.415019 13689 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:14.420174 13655 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:14.420753 13655 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:14.421384  5942 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 25. Current term is 26. Ops: [23.14-25.15]
I20260709 12:28:14.415083 13630 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:14.411113 13738 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:14.416555  5915 leader_election.cc:336] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.382282 13695 consensus_queue.cc:1048] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:28:14.382042 13383 raft_consensus.cc:493] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.378043 13694 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:28:14.433501 13383 raft_consensus.cc:515] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:14.445129 13383 leader_election.cc:290] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:14.436596 13502 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:14.439448  5946 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:28:14.440169  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 90305.4us
I20260709 12:28:14.456766  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:14.431016  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:14.383790 13168 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:14.470635  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:14.487471  5868 raft_consensus.cc:1240] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23. Ops: []
W20260709 12:28:14.496511  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 208772us
I20260709 12:28:14.398651 13577 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:14.504374  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 87834us
I20260709 12:28:14.488507 13500 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:14.514865  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 44357.1us
I20260709 12:28:14.430541 13747 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:14.520073 13747 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:14.499979 13626 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:14.512663  5915 leader_election.cc:336] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.497278 13641 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:14.516098  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:14.523603  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:28:14.528311  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 152965us
I20260709 12:28:14.514351 13668 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:14.535369  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:14.539168 13696 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:14.561286  6609 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
W20260709 12:28:14.566310  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 53703us
W20260709 12:28:14.567940  5915 leader_election.cc:336] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.477012 13655 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:14.572392  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:14.538146 13500 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:14.563843 13709 raft_consensus.cc:3060] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:14.584064 13567 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:14.609841 13567 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:14.610276  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:14.611042 13567 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:14.611776 13755 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:14.612247 13755 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:14.612614 13755 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:14.586998 13698 consensus_queue.cc:1048] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.012s
I20260709 12:28:14.568797 13591 consensus_queue.cc:1059] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:28:14.597234 13502 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:14.601666 13655 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:14.604070 13626 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:14.604672 13685 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:14.583608 13733 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
W20260709 12:28:14.572105 13751 log.cc:927] Time spent T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.064s	user 0.002s	sys 0.001s
I20260709 12:28:14.564086 13696 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:14.617393 13591 raft_consensus.cc:3055] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:28:14.618606  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 50686.7us
W20260709 12:28:14.641899  5915 leader_election.cc:336] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:14.643162  5915 leader_election.cc:336] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.643576  5915 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:14.644464 13681 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:14.645869  5915 leader_election.cc:336] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:14.646865  5915 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:14.647398  5915 leader_election.cc:336] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.647595 13591 raft_consensus.cc:740] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:14.627867 13738 consensus_queue.cc:1059] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:28:14.649174 13591 consensus_queue.cc:260] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:14.649932 13272 raft_consensus.cc:3055] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:14.650347 13272 raft_consensus.cc:740] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:14.650585 13591 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:14.651517 13272 consensus_queue.cc:260] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:14.632258  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 58179.4us
I20260709 12:28:14.637120  6023 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 26 LEADER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:28:14.659973  6611 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:14.661428  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 204800us
I20260709 12:28:14.661933  5946 raft_consensus.cc:1275] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:28:14.662935  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:14.653366 13272 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:14.666752  6611 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
W20260709 12:28:14.669731  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 185546us
I20260709 12:28:14.647802  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:28:14.672640  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 102498us
W20260709 12:28:14.674799  5915 leader_election.cc:336] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.675562  6022 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:28:14.678148  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:14.658723 13696 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:14.684254 13694 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.684355 13630 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:14.685055 13694 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:14.685649 13645 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:14.689374  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:28:14.690830  5915 leader_election.cc:336] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.691361  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:14.673772 13738 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:14.697858 13738 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:14.699056 13738 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:14.676090 13716 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:14.679347 13695 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:14.673030 13383 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:14.710704  5915 leader_election.cc:336] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.712064 13500 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:14.713804 13694 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:14.715528  5840 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.716259  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:28:14.717183  5915 leader_election.cc:336] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.687446 13672 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:14.726790 13685 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:14.728204 13672 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:14.720331 13655 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:14.742095  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:14.743063  5948 raft_consensus.cc:3060] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:14.744493 13709 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:14.748974  5840 leader_election.cc:336] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.749565  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:14.684280 13765 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.037s
W20260709 12:28:14.773166  5840 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.775126  5868 raft_consensus.cc:1275] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:28:14.778416  5840 leader_election.cc:336] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.778721  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:14.779059  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:14.770726 13655 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:14.787221  5840 leader_election.cc:336] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.772511 13698 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.792232  5948 raft_consensus.cc:2468] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:28:14.775923 13668 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
W20260709 12:28:14.797755  5840 leader_election.cc:336] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.798465  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:28:14.801364  5840 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.802569  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:28:14.815006  5840 leader_election.cc:336] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.815732  5840 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:14.777088 13588 consensus_queue.cc:1048] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:28:14.800779 13672 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:14.790534 13502 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:14.784736 13755 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:14.831144  5838 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:14.821557 13591 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:14.820572 13668 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:14.859578  5839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.813520 13698 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:14.876529  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 102457us
W20260709 12:28:14.879390  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 64433.1us
W20260709 12:28:14.877300  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 199221us
W20260709 12:28:14.882226  5840 leader_election.cc:336] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.882795  6609 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:14.865134 13657 raft_consensus.cc:2804] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:28:14.883610  5840 leader_election.cc:336] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.890470  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:14.891076  5840 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:14.893406 13755 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:14.894745 13696 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:14.906236  5874 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:28:14.900943  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:14.906369  5873 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:28:14.902268  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:14.905709  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:14.898777  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:14.908495  5871 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:28:14.909893  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:14.911931  5944 raft_consensus.cc:2393] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fc57407d56241b38a94d543e073f8af in current term 20: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:14.912350  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:14.912457  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:14.913556  5866 raft_consensus.cc:2468] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
I20260709 12:28:14.913578  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:14.913058  5865 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 26.
W20260709 12:28:14.914706  5840 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.916100  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:28:14.918406  5840 leader_election.cc:336] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.919077  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:14.922288  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:14.925522  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:14.901335 13779 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:14.926416  5941 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:28:14.915052 13657 raft_consensus.cc:697] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
W20260709 12:28:14.926800  5840 leader_election.cc:336] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:14.921543 13777 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:14.932749  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:14.935947  5943 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:28:14.942849  5945 raft_consensus.cc:1240] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:14.943317 13773 consensus_queue.cc:1048] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:14.945446  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:14.924173 13591 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:14.933524  5840 consensus_peers.cc:597] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:14.925814 13672 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:14.952953  5949 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: []
W20260709 12:28:14.957988  5840 consensus_peers.cc:597] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:14.960266  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:14.960453  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:28:14.961508  5840 leader_election.cc:336] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:14.962973  2067 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 6831)
I20260709 12:28:14.975497  5874 raft_consensus.cc:1275] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:28:14.977331  5869 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:14.965158 13779 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:15.034404 13779 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.034855 13779 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:15.026378 13769 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.204s	user 0.000s	sys 0.003s
I20260709 12:28:15.028034 13696 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:15.031833 13698 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:15.042457  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 685200us
W20260709 12:28:15.033692 13774 log.cc:927] Time spent T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.075s	user 0.002s	sys 0.000s
I20260709 12:28:14.990067 13657 consensus_queue.cc:237] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:15.032966  5870 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:28:15.047968 13377 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:15.063840 13698 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:28:15.067200  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 404374us
I20260709 12:28:15.068226  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:15.072168  5840 leader_election.cc:336] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:15.073261  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 101371us
I20260709 12:28:15.077917  5872 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 12:28:15.086714  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:15.117507  5915 leader_election.cc:336] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.118163  5915 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:15.098353 13641 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:15.098397 13417 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:15.097966 13648 raft_consensus.cc:493] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:15.080901 13765 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
W20260709 12:28:15.143025  5915 leader_election.cc:336] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.144523  5915 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:15.103276 13500 consensus_queue.cc:1048] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:28:15.118896 13756 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.151188  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:15.121680 13777 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:15.166982  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:15.162120 13766 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.125195 13417 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.174362  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:15.174589  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:15.177495 13455 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:15.180796  6609 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:15.151988 13168 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:15.129150 13689 consensus_queue.cc:1059] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:28:15.147301 13645 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:28:15.173869 13377 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:15.197773  5838 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:15.194109 13730 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 12:28:15.183187 13417 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:15.142699 13648 raft_consensus.cc:515] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:15.201007 13790 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:15.185583 13733 raft_consensus.cc:2804] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:15.172402 13540 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:28:15.157552 13703 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.038s
W20260709 12:28:15.220111  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 151971us
I20260709 12:28:15.241796  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:15.172789 13756 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:15.245695  5870 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:15.218834 13709 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:15.219619 13657 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:15.254096 13785 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.107s
W20260709 12:28:15.256660 13782 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:28:15.262696  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:15.264153  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:15.266356 13788 log.cc:927] Time spent T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:28:15.267867 13645 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:28:15.271852  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 100414us
I20260709 12:28:15.272244 13772 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:15.273058 13772 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:15.274753 13763 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:15.275800 13772 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:15.278669 13648 leader_election.cc:290] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:15.279352 13648 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:15.265417  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:15.239672 13733 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.282351  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:15.238327 13790 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:15.282680  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 61090.9us
I20260709 12:28:15.285013  5865 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:28:15.238487 13641 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:15.275827 13763 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:15.291471  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:15.188719 13766 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:15.273074  5915 leader_election.cc:336] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.249577 13709 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.305895  5869 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:15.324585  5871 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:28:15.289579 13648 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:15.332437  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:15.333210  6015 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:15.290246 13733 raft_consensus.cc:3060] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:15.331698  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:15.293273 13537 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:15.337782  6022 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:28:15.340862  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:15.338929  5915 consensus_peers.cc:597] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:15.309696 13709 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:15.342973  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:15.352331  5871 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:15.298123 13641 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:15.368279  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d95abc0e3a9452eb13b2625af34008e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:15.363547  5915 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:15.374910  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:15.373906  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:15.354200 13763 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:15.342365 13793 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:28:15.392287  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 90399.6us
I20260709 12:28:15.386137  6016 raft_consensus.cc:3055] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:15.394889  6016 raft_consensus.cc:740] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:15.410995  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:15.392063  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:15.417613  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:28:15.419836 13787 log.cc:927] Time spent T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:28:15.423442  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:15.362974 13785 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:28:15.365442 13648 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:28:15.403450 13779 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:15.418004 13537 consensus_queue.cc:1059] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:28:15.410161 13793 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:15.446906  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 75706.4us
I20260709 12:28:15.474792  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:15.495512  2067 raft_consensus.cc:2243] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.496502  2067 raft_consensus.cc:2272] T 402bd52b71ce4044b0ec86bf6f8d73fc P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.479912 13756 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:15.487387 13790 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:15.511360  6021 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:28:15.516615 13779 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:15.529541 13786 raft_consensus.cc:3055] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:15.530369 13786 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:15.531620 13786 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:15.533193 13786 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:15.547188  5915 leader_election.cc:336] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:15.554714  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 143898us
W20260709 12:28:15.564342  5915 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:15.569142  5870 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:15.440217  6016 consensus_queue.cc:260] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:15.573416  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:15.576052  5840 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:15.576987 13796 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:15.585564 13793 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:15.586601 13766 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:15.588243 13641 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:15.588505  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:15.591785  5840 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:15.554083 13763 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.594868  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:15.597810  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:15.601147  5870 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:15.560662 13403 raft_consensus.cc:3060] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:15.590181 13688 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:15.608796  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:15.632611  5870 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
I20260709 12:28:15.635062  6609 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:15.639667  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:15.599861 13755 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:15.606034 13645 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:15.606484 13777 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:15.644173 13539 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:15.655507 13641 consensus_queue.cc:1059] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 12:28:15.661743 13539 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 24 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:15.606521 13763 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:15.662966 13539 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:15.664155 13763 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:15.683514 13804 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.120s	user 0.002s	sys 0.000s
I20260709 12:28:15.604415 13779 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:28:15.695876 13417 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:15.697430  6019 raft_consensus.cc:1240] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:28:15.708653  5915 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:15.707577  6015 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
W20260709 12:28:15.738015  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 169115us
I20260709 12:28:15.748337  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:15.758858  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:15.759337  5990 leader_election.cc:336] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.732357 13537 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:15.745421 13785 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:15.767515 13785 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:15.773460  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:15.749243 13800 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:28:15.777921  5940 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:28:15.754213 13763 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.019s
W20260709 12:28:15.782823  5912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.756527 13793 consensus_queue.cc:1059] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:28:15.786319 13500 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:15.788450  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:15.776084 13785 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:15.793748  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:15.796428 13709 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:15.797339  2067 raft_consensus.cc:2243] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.798058  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e8568a9892438c91e30ddc62fef67c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:15.801743  5942 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:15.791606  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:15.810300  6611 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:15.810567  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:15.786355 13630 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:15.813575  6021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:15.820755  5949 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:28:15.822039  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:15.829790  5946 raft_consensus.cc:2393] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:15.784505 13648 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:15.767757 13537 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:15.831476  6611 leader_election.cc:304] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:15.826352 13630 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:15.784894 13780 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:15.804893 13403 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.835562  6611 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:15.806454 13500 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:15.826155 13766 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.104s
W20260709 12:28:15.914489  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 79060.4us
I20260709 12:28:15.949122  6017 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:15.809243 13645 raft_consensus.cc:493] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.835984 13756 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.861517 13709 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:15.885788 13799 raft_consensus.cc:3055] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:15.836567 13738 raft_consensus.cc:2749] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:15.935020 13733 consensus_meta.cc:287] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af: Time spent flushing consensus metadata: real 0.584s	user 0.016s	sys 0.005s
W20260709 12:28:15.967612  5990 leader_election.cc:336] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.919997 13758 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:15.997588 13500 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:16.020740  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:16.001178 13779 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.002847 13537 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:15.933871 13383 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:28:16.005724 13763 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.996114 13799 raft_consensus.cc:740] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:28:16.007304 13816 log.cc:927] Time spent T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:28:15.988843 13825 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:15.934556 13823 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.975694 13735 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
W20260709 12:28:16.008260 13812 log.cc:927] Time spent T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:28:15.919572 13403 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:15.913760 13780 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:15.950510 13790 raft_consensus.cc:3055] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:16.013970  5868 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:16.015092  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 51780.8us
I20260709 12:28:15.959760 13814 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:15.835821 13747 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:16.021593  5872 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:16.000140  6018 raft_consensus.cc:3055] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:16.036279 13825 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:16.018316 13657 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.994799 13645 raft_consensus.cc:515] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:16.040567  5943 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:16.041095  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 130795us
I20260709 12:28:16.027740 13723 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:15.995450 13756 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:16.058933  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:28:16.068671  5990 leader_election.cc:336] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.060788  6018 raft_consensus.cc:740] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:16.040992 13698 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.019073 13733 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:16.028672 13779 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:16.032823 13763 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:16.089849  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:16.092689  6611 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:16.043314 13500 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:16.096712  5943 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 24.
I20260709 12:28:16.046547 13815 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:16.085980  5990 leader_election.cc:336] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.057262 13790 raft_consensus.cc:740] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:16.070637  6018 consensus_queue.cc:260] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:16.119596  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:16.101209  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:16.123193  6019 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 23: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:16.125429  6018 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:16.124461  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:16.126330  5990 leader_election.cc:336] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.131394  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:16.134099  5913 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:28:16.070277 13688 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.136312  6023 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 12:28:16.139420  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 45564.5us
I20260709 12:28:16.089035 13825 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:16.088999 13780 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:16.140864  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 48282.7us
I20260709 12:28:16.148105  6611 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:16.143814  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 130706us
I20260709 12:28:16.090543 13403 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:16.121495 13455 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:16.118640 13738 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.145290  6019 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:28:16.084218 13828 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:16.156984  6611 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:16.085023 13766 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 12:28:16.070039 13657 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:16.047197 13823 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:16.165676  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.173681  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:16.148427 13168 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:16.067699 13814 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:16.117913 13698 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:16.185458  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:16.186959  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:16.188354  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:16.151382 13383 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:16.126837 13478 consensus_queue.cc:1059] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:28:16.197101  5869 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 15. (term mismatch)
I20260709 12:28:16.148185 13709 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:28:16.197563  5869 pending_rounds.cc:85] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Aborting all ops after (but not including) 14
I20260709 12:28:16.197989  5869 pending_rounds.cc:107] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: Aborting uncommitted NO_OP operation due to leader change: 25.15
I20260709 12:28:16.198354  5869 raft_consensus.cc:2889] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:16.092803 13723 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:16.161199 13758 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:16.160686 13826 raft_consensus.cc:670] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 12:28:16.179250 13825 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:16.201943  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 55293.6us
I20260709 12:28:16.126473 13815 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:16.149878 13688 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:16.178958 13539 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:16.178681 13168 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:16.216111  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee70b2d75314f788ad873b1950fb890" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:16.163730 13738 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:16.117801 13799 consensus_queue.cc:260] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:16.223271  6609 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:16.239364  5873 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:28:16.241245  5871 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:16.241288  5868 raft_consensus.cc:1240] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:28:16.245664  5872 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 27.
I20260709 12:28:16.251102  5870 raft_consensus.cc:1240] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 26. Current term is 27. Ops: [26.14-26.14]
I20260709 12:28:16.254348  6016 raft_consensus.cc:3060] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:16.254664  5946 raft_consensus.cc:1275] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:28:16.208763 13747 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:28:16.166056 13828 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:16.221779 13168 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:16.229470 13455 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:16.228643 13827 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:16.221037 13383 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:28:16.260442 13648 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:16.284905  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 61874us
I20260709 12:28:16.196597 13790 consensus_queue.cc:260] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:16.286855  5867 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54876 (request call id 1634) took 1139 ms. Trace:
I20260709 12:28:16.297832  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:16.294234  5867 rpcz_store.cc:276] 0709 12:28:15.147712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.147939 (+   227us) service_pool.cc:224] Handling call
0709 12:28:15.148253 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:15.161201 (+ 12948us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:15.479537 (+318336us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b18001b50e0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb93455 00007f39d667b3c7 00007f39d2c8d9a7 00007f39d2c9d0bf 00007f39d2c9d038
0709 12:28:15.589875 (+110338us) log.cc:844] Serialized 14 byte log entry
0709 12:28:16.125137 (+535262us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:16.152295 (+ 27158us) log.cc:844] Serialized 26 byte log entry
0709 12:28:16.152508 (+   213us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:16.152533 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.152636 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:16.281270 (+128634us) raft_consensus.cc:1685] finished
0709 12:28:16.286525 (+  5255us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.286764 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":289465,"prepare.queue_time_us":280958,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":1516,"raft.queue_time_us":288657,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":19094,"spinlock_wait_cycles":6194048,"thread_start_us":317645,"threads_started":3,"wal-append.queue_time_us":232488}
I20260709 12:28:16.302688  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:16.287086 13539 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:16.344413 13748 raft_consensus.cc:493] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.345135 13748 raft_consensus.cc:515] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:16.347723 13829 log.cc:927] Time spent T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:28:16.352813  6024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1791) took 1053 ms. Trace:
I20260709 12:28:16.284015 13668 consensus_queue.cc:1048] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
W20260709 12:28:16.353967  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.360628 13738 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:16.364791 13455 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:16.340480 13821 consensus_queue.cc:1175] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" responder_term: 22 status { last_received { term: 22 index: 13 } last_committed_idx: 12 last_received_current_leader { term: 22 index: 13 } }
I20260709 12:28:16.341652 13790 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:16.369835  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.369658  2067 raft_consensus.cc:2272] T f3c3647dbbb74fbd86870b12d1409976 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:16.386140  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.353438  6024 rpcz_store.cc:276] 0709 12:28:15.298964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.299094 (+   130us) service_pool.cc:224] Handling call
0709 12:28:16.089854 (+790760us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b5800480098. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfb5ee9c 00007f39cfb5ea5b 00007f39d8a778d9 00007f39d7bd1ee0 00007f39d7c9b0ab 00007f39d7c9b131 00007f39d7c5cfc0 00007f39d7c47929 00007f39d7c473d2 00007f39d11bebd4
0709 12:28:16.090044 (+   190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.201582 (+111538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.201657 (+    75us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:16.201726 (+    69us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:16.201788 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.352414 (+150626us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400160b54. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:16.352489 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.352691 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258826496}
I20260709 12:28:16.407240 13814 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:16.403777 13828 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:16.411492 13540 raft_consensus.cc:3055] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:16.413993  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:16.423066  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:16.434165  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 256191us
I20260709 12:28:16.438010  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:16.412065 13540 raft_consensus.cc:740] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:16.293874 13827 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:16.444540 13688 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:16.433168 13763 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:16.589243  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 275153us
I20260709 12:28:16.558660 13748 leader_election.cc:290] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:16.592800  6018 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:28:16.595943  6018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1905) took 1259 ms. Trace:
I20260709 12:28:16.568022 13723 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:16.596338  6018 rpcz_store.cc:276] 0709 12:28:15.336429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.436312 (+ 99883us) service_pool.cc:224] Handling call
0709 12:28:15.502082 (+ 65770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.000014 (+497932us) spinlock_profiling.cc:243] Waited 72.9 ms on lock 0x7b540018eaa0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bfadc0 00007f39d2c851c0 00007f39d2c86231 00007f39d2c82d8f 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c
0709 12:28:16.595455 (+595441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.595881 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167748480}
I20260709 12:28:16.574335 13823 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:16.592617 13815 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:16.571379 13838 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:28:16.593657 13747 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:16.608572  5988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:16.611125 13818 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:28:16.590931 13540 consensus_queue.cc:260] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:16.607452 13786 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:28:16.614785  5986 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.619340 13786 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:16.631866  6016 raft_consensus.cc:1275] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:28:16.632478  6016 pending_rounds.cc:85] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 11
I20260709 12:28:16.632700  6016 pending_rounds.cc:107] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:28:16.632929  6016 raft_consensus.cc:2889] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:16.633904  6016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37100 (request call id 1799) took 1326 ms. Trace:
W20260709 12:28:16.635421 13817 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:28:16.636148 13539 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:16.613852 13747 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:16.639034 13747 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:16.640291 13853 log.cc:927] Time spent T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
W20260709 12:28:16.644425 13794 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 12:28:16.646975 13827 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:16.647796 13168 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:16.648448 13500 consensus_meta.cc:287] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.003s
I20260709 12:28:16.634259  6016 rpcz_store.cc:276] 0709 12:28:15.307588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.384881 (+ 77293us) service_pool.cc:224] Handling call
0709 12:28:15.385246 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.253760 (+868514us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b58003e4ed0. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2bdf0aa 00007f39d2c61447 00007f39d2c79fe7 00007f39d2c7b7de 00007f39d2c85b49 00007f39d2c86416 00007f39d2c82d8f 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4
0709 12:28:16.633447 (+379687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.633811 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":368402304}
I20260709 12:28:16.652652 13168 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:16.661419 13785 consensus_meta.cc:287] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Time spent flushing consensus metadata: real 0.829s	user 0.013s	sys 0.003s
I20260709 12:28:16.661988 13785 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:16.663203  5691 catalog_manager.cc:5697] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 12 to 15, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 15 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:28:16.664551 13785 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:16.666635 13855 log.cc:927] Time spent T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:28:16.646976 13842 log.cc:927] Time spent T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.308s	user 0.002s	sys 0.000s
I20260709 12:28:16.668053 13786 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:16.668118  5949 raft_consensus.cc:1240] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Rejecting Update request from peer 7a0204b931394fc48ffc503716e0d111 for earlier term 21. Current term is 22. Ops: []
W20260709 12:28:16.659720  5990 leader_election.cc:336] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.672968  5990 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:16.665252  5691 catalog_manager.cc:5697] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 15 to 18, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 18 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:16.677417  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 81004.4us
I20260709 12:28:16.678808  6609 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7
I20260709 12:28:16.686177 13851 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:28:16.684476  5990 leader_election.cc:336] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.678507 13815 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:16.692766 13859 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.697430  6019 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:28:16.697991  6019 pending_rounds.cc:85] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 12
I20260709 12:28:16.698246  6019 pending_rounds.cc:107] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:28:16.698500  6019 raft_consensus.cc:2889] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:16.699106  5990 leader_election.cc:336] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:16.699910  5990 leader_election.cc:336] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:16.700554  5990 leader_election.cc:336] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.700614 13733 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:16.690004 13851 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 27 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
I20260709 12:28:16.702656 13851 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:16.719332 13630 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:16.724220 13645 leader_election.cc:290] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:16.728147 13630 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:16.693490 13859 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:16.741525 13799 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:16.711863  5990 leader_election.cc:336] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:16.750597  5990 leader_election.cc:336] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:16.751706 13846 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:16.752410 13846 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:16.754869 13846 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:16.756450 13417 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:16.761096 13723 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:16.761668 13672 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:16.761864 13723 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:16.762905 13672 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:16.763916 13655 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:16.764540 13723 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:16.764489 13655 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:16.767051 13655 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:16.767282  5869 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:16.774101  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:16.777576  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:16.778739  5866 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 23: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:16.782442  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:16.784089 13779 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:16.785377  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:16.786163  5869 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:28:16.791965 13755 raft_consensus.cc:493] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:28:16.792635  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 663609us
I20260709 12:28:16.792948 13755 raft_consensus.cc:515] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:16.795996 13755 leader_election.cc:290] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:16.800441 13698 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:16.808909 13698 consensus_queue.cc:1059] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:16.810606 13698 raft_consensus.cc:3055] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:16.811055 13698 raft_consensus.cc:740] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:16.811926 13698 consensus_queue.cc:260] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:16.813242 13698 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:16.816833  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:16.822167  6016 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 26.
I20260709 12:28:16.819748  6021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1909) took 1396 ms. Trace:
I20260709 12:28:16.809055 13756 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:16.829023  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:16.829422  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:16.840899  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:16.842208  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:16.842533  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:16.843924  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:16.844416  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:16.843963  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:16.845602  6015 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:28:16.846987  6024 raft_consensus.cc:2393] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 22: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:16.847098  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:16.849234  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:16.847937  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:16.852123  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:16.852967  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:16.853997  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbedb48e03420dbf7f8646e65b1886" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:16.826735 13730 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.052s	sys 0.038s
I20260709 12:28:16.862308  5913 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:16.856817  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:16.865370  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:16.741518 13859 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:16.872925 13591 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:16.873668 13591 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:16.874131  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:16.874596 13773 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:28:16.873068 13730 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.053s	sys 0.038s
I20260709 12:28:16.838459 13478 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.873687  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:16.821414 13763 raft_consensus.cc:493] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:16.821663 13779 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:16.880524  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d45107f59eee48f79ae2b6006319cd62" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:16.882261  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:16.882978  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:16.851678 13668 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:16.886523  5942 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:16.887838  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:16.863325 13696 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:16.891369  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:16.875063 13773 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:16.881290 13539 raft_consensus.cc:493] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.887267 13779 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:16.901755 13847 log.cc:927] Time spent T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.562s	user 0.003s	sys 0.002s
I20260709 12:28:16.902860 13779 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:16.907079  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:16.908769  5943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:16.909786 13672 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:16.910475 13773 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:16.894986 13839 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 12:28:16.884928 13763 raft_consensus.cc:515] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:16.887809 13668 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.918664  6022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1911) took 1328 ms. Trace:
I20260709 12:28:16.919154  6022 rpcz_store.cc:276] 0709 12:28:15.590097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.948063 (+357966us) service_pool.cc:224] Handling call
0709 12:28:15.948450 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:16.194088 (+245638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.194125 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:16.207393 (+ 13268us) log.cc:844] Serialized 26 byte log entry
0709 12:28:16.291702 (+ 84309us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:16.291770 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.796827 (+505057us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400464454. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:16.796905 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:16.900949 (+104044us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b4400022b4c. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39cfa1edb1 00007f39cfabcc85 00007f39cfabd6a2 00007f39d282f1db 00007f39d282aaaf 00007f39d281e86e 00007f39d281dbb6 00007f39d281d7d8 00007f39d2827221 00007f39d28271d9 00007f39d2827169
0709 12:28:16.918420 (+ 17471us) raft_consensus.cc:1685] finished
0709 12:28:16.918445 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.918590 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2391,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":7490,"spinlock_wait_cycles":637755648,"thread_start_us":53445,"threads_started":1,"wal-append.queue_time_us":118473}
I20260709 12:28:16.881918 13478 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:16.921401 13539 raft_consensus.cc:515] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:16.894042 13696 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.922763  6019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34888 (request call id 1921) took 1141 ms. Trace:
I20260709 12:28:16.927632 13696 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:16.923378  6019 rpcz_store.cc:276] 0709 12:28:15.781078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.158501 (+377423us) service_pool.cc:224] Handling call
0709 12:28:16.158834 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.698949 (+540115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.922415 (+223466us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b540065b3d4. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d2bec571 00007f39d2c83e01 00007f39d2c8283e 00007f39d7c47419 00007f39d11bebd4 00007f39d11beb62 00007f39d11bea91 00007f39d11bea0c 00007f39d11bdcc2 00007f39d0fbe48c 00007f39d0fbd8c6
0709 12:28:16.922684 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235418368}
W20260709 12:28:16.931737  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 43707.4us
I20260709 12:28:16.934199  6611 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:16.919126 13668 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:16.950969  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.943575 13417 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.955137 13417 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:16.958580  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 65537.8us
I20260709 12:28:16.960264  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:16.963766 13540 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:16.968168 13748 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:16.968257 13845 log.cc:927] Time spent T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.629s	user 0.004s	sys 0.000s
I20260709 12:28:16.974466  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:16.976212  2067 raft_consensus.cc:2243] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:16.948462 13796 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:28:16.957470 13417 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:16.979564  6024 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:28:16.946954 13790 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.981323 13790 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:16.982280 13763 leader_election.cc:290] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:16.958298 13772 consensus_queue.cc:1048] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:28:17.001848 13763 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.002718 13763 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:17.003360 13773 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.004428 13478 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:17.005142 13763 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:17.005270 13815 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:28:17.007021  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 48559.5us
I20260709 12:28:17.008472  5915 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:17.010586 13776 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:17.010758  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:16.953890 13827 consensus_queue.cc:1059] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 12:28:17.012887 13763 raft_consensus.cc:3055] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:17.013319 13763 raft_consensus.cc:740] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:17.013954 13785 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:17.026885  5946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:17.005546 13773 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:17.025389 13672 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:17.038079 13672 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.039644 13730 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:17.039894  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:17.040974  6016 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 26.
I20260709 12:28:17.042016  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:17.042768  6016 raft_consensus.cc:2393] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 23: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:17.045964  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.046548  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:17.046762  5942 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:28:17.047456  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:17.040961  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 80824.2us
I20260709 12:28:17.050701  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.051851  5949 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 26.
I20260709 12:28:16.823148  6021 rpcz_store.cc:276] 0709 12:28:15.423771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.833322 (+409551us) service_pool.cc:224] Handling call
0709 12:28:15.864203 (+ 30881us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.864453 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:15.881005 (+ 16552us) log.cc:844] Serialized 14 byte log entry
0709 12:28:16.798305 (+917300us) spinlock_profiling.cc:243] Waited 4.63 ms on lock 0x7b1800258580. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb78e07 00007f39d89d39a9 00007f39cfb9569b 00007f39cfb9393f 00007f39cfb95c02 00007f39d281d20b 00007f39d2822d23 00007f39d282323a 00007f39d2c8da2a
0709 12:28:16.819296 (+ 20991us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:16.819366 (+    70us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:16.819437 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.819525 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.819676 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16250,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"raft.queue_time_us":1747,"spinlock_wait_cycles":10638592,"thread_start_us":20715,"threads_started":1,"wal-append.queue_time_us":918122}
I20260709 12:28:17.054826  6020 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:17.054836  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.055918  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.056104  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e0b097c72c4c748e146f8303c90f0c" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.058305  2067 pending_rounds.cc:70] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Trying to abort 1 pending ops.
I20260709 12:28:17.057179  6609 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.058863  2067 pending_rounds.cc:77] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af: Aborting op as it isn't in flight: id { term: 19 index: 11 } timestamp: 7305625910288617472 op_type: NO_OP noop_request { }
W20260709 12:28:17.097357  5840 consensus_peers.cc:597] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:17.074679  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4765e1fc00c485086dc883bea4b87f7" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.097301  2067 raft_consensus.cc:2889] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:17.103107  2067 raft_consensus.cc:2272] T 7a3145d5d96e41518c7bf41999289ac3 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:17.064711  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.022966 13846 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:17.108532  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.110071  5949 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 26.
W20260709 12:28:17.112382  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 47750.9us
I20260709 12:28:17.026155 13827 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:17.076792  5838 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:17.113526  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.105849 13383 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:17.059600 13709 raft_consensus.cc:493] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.118541 13839 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:17.115253 13846 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:17.116986 13655 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:17.143342 13655 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.143754 13655 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:17.014228 13763 consensus_queue.cc:260] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:17.031308 13790 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:17.108881  5840 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 136 similar messages]
I20260709 12:28:17.119827  6609 leader_election.cc:304] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.130697 13709 raft_consensus.cc:515] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:17.063433 13831 raft_consensus.cc:493] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:17.132894 13859 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:17.117357 13755 raft_consensus.cc:493] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:28:17.188560  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 229687us
I20260709 12:28:17.144695 13756 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.191371  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:17.191679  5987 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.195700  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ce59a08f3427581e5e6b5d87ad757" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 22 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.195740  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:17.198450  5987 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.114157 13645 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:28:17.167286 13748 raft_consensus.cc:2749] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:17.204962  5987 leader_election.cc:304] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.135973 13827 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.222378  5946 raft_consensus.cc:2410] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:28:17.197830 13755 raft_consensus.cc:515] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.227497 13846 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:17.195811 13763 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:17.235190  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 219603us
I20260709 12:28:17.238633  5987 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.240499  5987 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:17.242220  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 46636us
I20260709 12:28:17.244628 13796 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:17.245010 13383 raft_consensus.cc:2749] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:17.248051 13672 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:17.235419 13539 leader_election.cc:290] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:17.250574 13657 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:17.206661 13417 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:17.201277 13799 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:17.174666 13831 raft_consensus.cc:515] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:17.201898 13859 raft_consensus.cc:493] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.254101  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:17.267613  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.199570 13756 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:17.258967  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.255661  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:17.256819  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.269804  5987 leader_election.cc:304] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.237170 13794 raft_consensus.cc:2749] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:17.273156  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:17.254915 13648 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:17.281040  5987 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.279382 13681 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:17.270061 13859 raft_consensus.cc:515] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:17.281571 13799 raft_consensus.cc:2749] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:17.281695 13417 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:17.327466  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 46540.8us
I20260709 12:28:17.285143 13540 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:17.333277  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:17.336277 13859 leader_election.cc:290] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:17.339462 13831 leader_election.cc:290] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:17.341341 13756 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:17.342118 13827 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:17.343364 13591 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:17.344095  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:17.345885  6018 raft_consensus.cc:2393] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 22: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:17.345896  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.284650 13838 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:17.327925 13681 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.331609 13794 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:17.366591  5987 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.379900 13591 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.380553 13591 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.383113 13591 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:17.395208  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:17.347839 13540 raft_consensus.cc:493] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
W20260709 12:28:17.373018  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 81687us
I20260709 12:28:17.397233  5987 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.398434  6609 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.401335  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:17.401748  2067 raft_consensus.cc:2243] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:17.363171 13709 leader_election.cc:290] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:17.367595 13827 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:17.367866 13755 leader_election.cc:290] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:17.397580 13540 raft_consensus.cc:3060] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:17.405030  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.405952  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6803fb2b32049f3935eeb6331dcde39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.406953  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:17.376554 13681 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:17.409309  6020 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 20.
I20260709 12:28:17.352156 13417 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:17.398559 13799 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:17.415815  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:17.416081  5949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.416612  6018 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 22.
I20260709 12:28:17.417774  6020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e657475654794984a13c42a668f3158b" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:17.418576  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.420535  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:17.420818  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:17.421504  5869 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:28:17.422554  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:17.405300 13794 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:17.422631  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.426409  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:17.427227  5869 raft_consensus.cc:2393] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:17.419474 13648 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:17.428683 13648 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:17.405349 13668 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.432502 13668 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:17.432246 13655 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.433718  5867 raft_consensus.cc:1240] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 12:28:17.436054  5947 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:28:17.437691 13417 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:17.439062 13696 consensus_meta.cc:287] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.511s	user 0.008s	sys 0.006s
I20260709 12:28:17.440397 13875 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:17.440994  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:17.450325  6015 raft_consensus.cc:2410] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:28:17.445652  5840 leader_election.cc:304] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.425916  5940 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:28:17.446838  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:17.457952  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:17.463512  5872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:17.464427  5872 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 27: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:17.423732 13884 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:17.466307  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:17.469007  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:28:17.457222 13698 consensus_meta.cc:287] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Time spent flushing consensus metadata: real 0.617s	user 0.016s	sys 0.012s
I20260709 12:28:17.473377  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.473672  6015 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:17.479595  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 70455.4us
I20260709 12:28:17.472150  5840 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.482626  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:17.484167  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:17.484535  2067 raft_consensus.cc:2272] T d6803fb2b32049f3935eeb6331dcde39 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:17.461789 13417 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:17.482793 13777 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.462956 13846 raft_consensus.cc:2749] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:17.466265 13834 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:17.500613  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:17.481616 13827 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:17.502555 13777 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.510780 13777 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:17.510995 13648 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:17.457237 13696 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.517306 13834 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:17.495038 13417 raft_consensus.cc:493] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.517719  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:17.527385  2067 raft_consensus.cc:2243] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:17.528097  2067 pending_rounds.cc:70] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Trying to abort 1 pending ops.
I20260709 12:28:17.528195  6023 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:28:17.499301 13698 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:17.530872 13834 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:17.531536 13834 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:17.531894 13834 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:17.532248 13834 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:17.532395  5873 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:28:17.519745 13696 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:17.520367 13417 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:17.533943  5866 raft_consensus.cc:2368] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 23. Current term is 24.
W20260709 12:28:17.537947  5840 leader_election.cc:336] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.538956  5840 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:17.540270 13655 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:17.540635 13834 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:17.541157 13655 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:17.543274 13834 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:17.544849  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 44393.7us
I20260709 12:28:17.528420  2067 pending_rounds.cc:77] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af: Aborting op as it isn't in flight: id { term: 21 index: 12 } timestamp: 7305625901762867200 op_type: NO_OP noop_request { }
I20260709 12:28:17.551105  2067 raft_consensus.cc:2889] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:17.551587  2067 raft_consensus.cc:2272] T fee70b2d75314f788ad873b1950fb890 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:17.553257  6015 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 27.
I20260709 12:28:17.557323  5840 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:17.563822  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.568665  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:17.567783 13668 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:17.573550  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:17.574400  6022 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:17.558302 13540 raft_consensus.cc:515] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.575114  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:17.576427  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:17.577268 13540 leader_election.cc:290] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:17.580602  5913 leader_election.cc:400] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 23. Current term is 24.
I20260709 12:28:17.557925 13796 consensus_queue.cc:1059] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:28:17.582221 13796 raft_consensus.cc:3055] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:17.582584 13796 raft_consensus.cc:740] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:17.583544 13796 consensus_queue.cc:260] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.591954  6019 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 14. (term mismatch)
I20260709 12:28:17.592559  6019 pending_rounds.cc:85] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 13
I20260709 12:28:17.592800  6019 pending_rounds.cc:107] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 26.14
I20260709 12:28:17.593050  6019 raft_consensus.cc:2889] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:17.597659 13796 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:17.581478  5913 leader_election.cc:403] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:28:17.605859 13668 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 27 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:17.607080 13668 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:17.609874  5840 leader_election.cc:336] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.577543  6024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:17.610565  5840 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.611431  6024 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
W20260709 12:28:17.612797  5840 leader_election.cc:336] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.613303  5840 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:28:17.614629  5840 leader_election.cc:336] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.615101  5840 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.616024  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:17.617162  6611 leader_election.cc:304] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:17.618047 13383 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:17.618386 13766 raft_consensus.cc:2749] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:17.581744 13826 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:17.620625  5840 leader_election.cc:336] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:17.625384  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 128245us
W20260709 12:28:17.630196  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 49817.9us
I20260709 12:28:17.638840  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:17.636618  5915 consensus_peers.cc:597] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:17.643076  5987 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.649858  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d001526e46b24f429f57c91e4a270036" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:17.641840  5915 consensus_peers.cc:597] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:17.655615  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.657785  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:17.655005  5840 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:17.666195  5870 raft_consensus.cc:2393] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 23: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:17.618494 13383 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.668497  5838 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.642629 13455 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:17.643946 13834 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:17.671614  5838 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.641528 13832 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:17.645241 13688 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:17.679914  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1efdcc52fa4f98acdbe3c7f5c38570" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.642396 13826 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:17.656673 13648 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:17.656598 13827 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:17.719880 13827 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:17.720383 13827 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:17.721078  6611 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 77318.6us
I20260709 12:28:17.721412 13681 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.724241 13681 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:17.667003 13846 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:17.740548 13846 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.740998 13846 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:17.741070  5942 raft_consensus.cc:1240] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:17.739645  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:17.691128  5696 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50588 (request call id 60) took 17848 ms (client timeout 15000 ms). Trace:
I20260709 12:28:17.688387 13417 raft_consensus.cc:515] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:17.681511 13799 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:17.688783 13672 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:17.670658 13383 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:17.674881 13777 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:17.681988 13756 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 12:28:17.671725 13689 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.706089  2067 tablet_replica.cc:333] stopping tablet replica
W20260709 12:28:17.709601  5990 consensus_peers.cc:597] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:17.712366 13730 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:28:17.714942  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 52048.1us
I20260709 12:28:17.716218 13826 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:17.739037  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:17.686270  5691 catalog_manager.cc:5697] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 11 to 14, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 14 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:17.754904  2067 raft_consensus.cc:2243] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:28:17.747104  6022 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
W20260709 12:28:17.753943  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 44423.3us
W20260709 12:28:17.756822  5990 leader_election.cc:336] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.757727  5990 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:17.761926  5990 leader_election.cc:336] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.762383  5990 leader_election.cc:304] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:17.765578  5990 leader_election.cc:336] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:17.766875  5990 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.767311  5990 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:17.768469  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:17.769745  6023 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 27: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:17.772578 13875 raft_consensus.cc:2749] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:17.771840  5840 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:17.774318  2067 raft_consensus.cc:2272] T 1ba7af1a4a744df6bc8c20ec72e21519 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:17.775576  5990 leader_election.cc:336] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.775513  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0006f949032c4034b4f4872ee332386e" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.776312  5990 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.777078  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:17.769042 13794 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:17.786670 13793 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:17.787096 13790 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:17.788466 13417 leader_election.cc:290] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:17.792428  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45edb6642b84468aa6fc19a7fd2ddf6c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.800048  6022 raft_consensus.cc:2468] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:28:17.797480  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bd52b71ce4044b0ec86bf6f8d73fc" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:17.803025  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3c3647dbbb74fbd86870b12d1409976" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.796312  5691 catalog_manager.cc:5697] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:28:17.805313  5691 catalog_manager.cc:5697] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 15 to 18, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 18 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:28:17.808677 13885 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:17.750833 13799 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.753605 13648 consensus_queue.cc:1059] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 12:28:17.795639 13668 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:17.817703  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a7ca03d5d3d4b13bee5481a90dc7bdc" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.819056  5940 raft_consensus.cc:2410] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:28:17.818549 13455 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 23. Current term is 24.
I20260709 12:28:17.763368 13689 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:17.837050  5840 leader_election.cc:336] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.841807 13168 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:17.843031  5945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.844120  5945 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:28:17.845175  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:17.846298  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:17.846921  5942 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:17.837926  5840 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.852576  6609 leader_election.cc:304] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.825666 13668 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 24 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:17.854925  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:17.857784  5940 raft_consensus.cc:2393] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 27: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:17.865697  5691 catalog_manager.cc:5697] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
W20260709 12:28:17.743698  5696 rpcz_store.cc:269] 0709 12:27:59.840296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.840488 (+   192us) service_pool.cc:224] Handling call
0709 12:28:17.688540 (+17848052us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:08.607445 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:08.634557 (+ 27112us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:08.634633 (+    76us) write_op.cc:435] Acquired schema lock
0709 12:28:08.634657 (+    24us) tablet.cc:662] Decoding operations
0709 12:28:08.636942 (+  2285us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:08.637077 (+   135us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:08.637119 (+    42us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:28:09.234160 (+597041us) tablet.cc:701] Row locks acquired
0709 12:28:09.234221 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:28:09.234422 (+   201us) write_op.cc:270] Start()
0709 12:28:09.234600 (+   178us) write_op.cc:276] Timestamp: P: 1783600089234342 usec, L: 0
0709 12:28:09.234644 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 12:28:09.235514 (+   870us) log.cc:844] Serialized 8931 byte log entry
0709 12:28:10.096494 (+860980us) op_driver.cc:464] REPLICATION: finished
0709 12:28:10.189469 (+ 92975us) write_op.cc:301] APPLY: starting
0709 12:28:10.199615 (+ 10146us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:10.907619 (+708004us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:10.907768 (+   149us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:11.328631 (+420863us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:11.329151 (+   520us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=0,mrs_lookups=0
0709 12:28:11.329251 (+   100us) write_op.cc:312] APPLY: finished
0709 12:28:11.393141 (+ 63890us) log.cc:844] Serialized 203 byte log entry
0709 12:28:11.755321 (+362180us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:11.856963 (+101642us) write_op.cc:454] Released schema lock
0709 12:28:11.857084 (+   121us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:11.865483 (+  8399us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:11.865531 (+    48us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":81294,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":64480,"apply.run_cpu_time_us":65571,"apply.run_wall_time_us":1686429,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100359,"num_ops":23,"prepare.queue_time_us":321089,"prepare.run_cpu_time_us":29385,"prepare.run_wall_time_us":876557,"raft.queue_time_us":460128,"raft.run_cpu_time_us":4190,"raft.run_wall_time_us":57447,"replication_time_us":819819,"thread_start_us":962363,"threads_started":5,"wal-append.queue_time_us":683280,"wal-append.run_cpu_time_us":31591,"wal-append.run_wall_time_us":2455784}]]}
I20260709 12:28:17.859270  5838 leader_election.cc:304] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.875803 13672 raft_consensus.cc:2749] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:17.881173  5838 leader_election.cc:304] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.885463  5942 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 27.
I20260709 12:28:17.894382  6609 leader_election.cc:304] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:17.833909 13777 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.889029  5838 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:17.893527  5840 consensus_peers.cc:597] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:17.900972  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 80487.2us
I20260709 12:28:17.906268  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:17.916947  5915 leader_election.cc:304] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:17.918708  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 143133us
I20260709 12:28:17.895370 13688 raft_consensus.cc:2749] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:17.909412 13655 raft_consensus.cc:2749] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:17.893160 13672 raft_consensus.cc:2749] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:17.920754 13886 raft_consensus.cc:2749] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:17.921685 13799 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:17.921962 13681 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:17.922712  5691 catalog_manager.cc:5697] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 6 to 13, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 13 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:28:17.902073 13838 raft_consensus.cc:493] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.906642 13738 raft_consensus.cc:2749] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:17.935200  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:17.930542 13648 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:17.935884 13827 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.903795 13777 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.948199  2067 raft_consensus.cc:2243] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:17.940474  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 125727us
I20260709 12:28:17.933171  5838 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:28:17.942507  5990 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:17.943455  5691 catalog_manager.cc:5697] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 16 to 17, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 17 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:17.961215  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 60458.6us
W20260709 12:28:17.962066  5840 leader_election.cc:336] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.962486  5840 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:17.963497  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:17.921830 13668 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.964255  5942 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:28:17.965169  5691 catalog_manager.cc:5697] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 13 to 16, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 16 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:17.967025  5840 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:17.969965  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:17.975868 13867 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:17.976224 13689 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:17.976536 13867 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 27 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:17.976954 13689 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.977509 13689 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.977589 13867 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.980003 13689 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:17.984643  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:17.986055 13796 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:17.986392 13688 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:17.986482 13796 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:17.988875 13796 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:17.994881 13827 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:17.947007 13648 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:17.997859 13648 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:17.998721  5874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:18.000568 13846 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:18.003796  5874 raft_consensus.cc:2368] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23.
I20260709 12:28:18.006124 13846 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:18.009336  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:18.010989  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:18.022452  5948 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:18.011478  5944 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
I20260709 12:28:17.997649  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:18.027426  5942 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:28:18.027760  5870 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:18.014149 13796 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:18.015938  5840 consensus_peers.cc:597] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:18.041989  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:18.042778  5867 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:18.043335 13796 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:18.044421  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:18.045296  5942 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:18.047303  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:18.048090  5944 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:28:18.013252 13834 raft_consensus.cc:2749] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:17.959748 13776 raft_consensus.cc:2804] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:18.029863  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:18.049530  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:18.050928 13755 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:18.051380 13755 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.051687  5866 raft_consensus.cc:3060] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:18.051757 13755 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:18.058805 13796 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:18.059479 13796 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:17.968174 13832 raft_consensus.cc:2749] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:17.998828 13892 log.cc:927] Time spent T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.144s	user 0.001s	sys 0.001s
I20260709 12:28:18.060818 13417 raft_consensus.cc:3055] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:28:17.999332 13894 log.cc:927] Time spent T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:28:18.001904  5691 catalog_manager.cc:5697] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 13 to 15, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 15 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:18.063187  5691 catalog_manager.cc:5697] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 12 to 15, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 15 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:28:17.947074 13838 raft_consensus.cc:515] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:18.039656  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:18.067569  5987 leader_election.cc:400] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Vote denied by peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005) with higher term. Message: Invalid argument: T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23.
I20260709 12:28:18.071241  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:18.072264  5867 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:28:18.061342 13417 raft_consensus.cc:740] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:18.073966 13417 consensus_queue.cc:260] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:18.075531 13417 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:18.076447  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:18.076802  5942 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:28:18.077730 13648 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:18.078215 13648 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:18.078348  5838 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:18.049335 13776 raft_consensus.cc:697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:18.060449 13796 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:18.079958 13776 consensus_queue.cc:237] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:18.080996  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:18.078594 13648 raft_consensus.cc:3060] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:18.084559  5840 consensus_peers.cc:597] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:18.086724 13832 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:18.087427 13832 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:18.032910 13777 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:18.088495 13832 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:18.067512 13838 leader_election.cc:290] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:18.094266  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:18.096966 13383 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:18.038431  2067 raft_consensus.cc:2272] T 7d95abc0e3a9452eb13b2625af34008e P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:18.106244  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:18.126247  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:18.126748  2067 raft_consensus.cc:2243] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:18.128106  2067 raft_consensus.cc:2272] T 48c0a9da01454a2794160d9c0ff08ea9 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:18.107880 13648 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:18.101984  5866 raft_consensus.cc:2468] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 23.
W20260709 12:28:18.098861  5990 leader_election.cc:336] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:18.135330  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 106553us
I20260709 12:28:18.119187 13383 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:18.139254  5942 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:28:18.150008  5987 leader_election.cc:304] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:18.155441  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:18.159644  6018 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:28:18.156580  6023 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:28:18.160112  6017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:18.160754  6017 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:18.161774  5913 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:18.164480 13648 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:18.160712 13815 raft_consensus.cc:2804] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:18.167405  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
W20260709 12:28:18.169325  5990 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:18.174240  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:28:18.183682  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 259693us
W20260709 12:28:18.184458  5990 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:18.165984 13834 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.007s
I20260709 12:28:18.196513  6609 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:18.165998 13815 raft_consensus.cc:697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 23 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:18.171659 13455 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:18.199008  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:18.204499  5944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:18.205904  5944 raft_consensus.cc:2393] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:18.201953 13455 raft_consensus.cc:493] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.207376 13859 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:18.212924 13417 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:18.246454  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2de53e733442a69e97d39affab60ba" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:18.248068  5942 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:28:18.252283  5866 raft_consensus.cc:2468] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:28:18.234771 13455 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:18.255162  5987 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:18.260516  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:28:18.266181  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 290263us
I20260709 12:28:18.266957  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:18.267995  2067 raft_consensus.cc:2243] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:28:18.250133 13417 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:18.274291  2067 raft_consensus.cc:2272] T 41bc2065e43e40c98ee821925758a3f3 P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:18.277130  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:18.278398  6017 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc57407d56241b38a94d543e073f8af in term 23.
I20260709 12:28:18.279495  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:18.281399  6018 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:18.285388  5871 raft_consensus.cc:1240] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Rejecting Update request from peer 1fc57407d56241b38a94d543e073f8af for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:18.288937  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:18.291783  2067 raft_consensus.cc:2243] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:18.293336  2067 raft_consensus.cc:2272] T 681f64c726104caf875ae7d4329d0676 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:18.296837  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:18.297912  2067 raft_consensus.cc:2243] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:18.298604  2067 raft_consensus.cc:2272] T d001526e46b24f429f57c91e4a270036 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:18.302995  6609 leader_election.cc:304] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:18.303982 13884 raft_consensus.cc:2749] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:18.304711 13383 consensus_queue.cc:1059] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:18.265193 13875 raft_consensus.cc:2804] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:18.282675  6611 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7; no voters: 
I20260709 12:28:18.305908 13756 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.315012  6021 raft_consensus.cc:1240] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Rejecting Update request from peer 7eabb96dfdd1449c94ad1261ded46c3c for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:28:18.323208 13884 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:18.323874 13884 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Becoming Leader. State: Replica: 1fc57407d56241b38a94d543e073f8af, State: Running, Role: LEADER
W20260709 12:28:18.324376 13901 log.cc:927] Time spent T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:28:18.324913 13884 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:18.256287 13755 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:18.349210 13875 raft_consensus.cc:697] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:18.350723 13875 consensus_queue.cc:237] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:18.269423 13815 consensus_queue.cc:237] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:18.359614 13755 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:18.366883 13417 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:18.369160  5869 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 23: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:18.321058 13832 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 12:28:18.345072  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:18.408203  2067 raft_consensus.cc:2243] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:18.406890  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:18.422363  2067 raft_consensus.cc:2272] T 99b9212afb8b459bb96908f8259bb8c0 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:18.428191  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 122057us
W20260709 12:28:18.452883  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 104132us
I20260709 12:28:18.473058  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:18.473927  5942 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:18.533785  6019 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 15. (term mismatch)
I20260709 12:28:18.534478  6019 pending_rounds.cc:85] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 14
I20260709 12:28:18.534776  6019 pending_rounds.cc:107] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 26.15
I20260709 12:28:18.535063  6019 raft_consensus.cc:2889] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:18.343760 13756 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:18.548254  5948 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:28:18.553325  5913 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:18.511925  5840 consensus_peers.cc:597] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:18.572314  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 60457us
W20260709 12:28:18.573292  5840 consensus_peers.cc:597] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:18.586550 13834 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:28:18.597272 13897 consensus_queue.cc:1059] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:28:18.598112 13906 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:28:18.603404 13681 raft_consensus.cc:3055] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:18.603986 13681 raft_consensus.cc:740] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:18.604784  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:18.605973  2067 raft_consensus.cc:2243] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:18.604905 13681 consensus_queue.cc:260] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:18.617170  5942 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:28:18.615000  2067 pending_rounds.cc:70] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Trying to abort 1 pending ops.
I20260709 12:28:18.623600  2067 pending_rounds.cc:77] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af: Aborting op as it isn't in flight: id { term: 26 index: 15 } timestamp: 7305625949441257472 op_type: NO_OP noop_request { }
I20260709 12:28:18.624251  2067 raft_consensus.cc:2889] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:18.624974  2067 raft_consensus.cc:2272] T bc2de53e733442a69e97d39affab60ba P 1fc57407d56241b38a94d543e073f8af [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:18.619626 13903 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:18.662372  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 93027.1us
I20260709 12:28:18.662410  5838 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7
W20260709 12:28:18.662685  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 109488us
I20260709 12:28:18.674248  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:18.675361  2067 raft_consensus.cc:2243] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:28:18.677274 13756 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:18.682070 13455 raft_consensus.cc:515] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:18.695441  5948 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:28:18.712208  6021 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:28:18.720296  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:18.690028 13779 raft_consensus.cc:932] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 20 LEADER]: Unable to handle notification of new term (21): Illegal state: RaftConsensus is not running: State = Stopping
I20260709 12:28:18.673574 13672 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:18.743546  2067 raft_consensus.cc:2272] T 685933c4b2364e1c90d92edb4c091799 P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:18.684774 13455 leader_election.cc:290] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:18.745914 13672 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:18.799713 13852 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.766578 13756 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.808701  6021 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 12:28:18.825182  5914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:18.824105  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 152296us
W20260709 12:28:18.834538  5915 consensus_peers.cc:597] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:18.858646  5871 raft_consensus.cc:1275] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:28:18.819308 13852 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:18.865172  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "926e6a2e59d94f64ba48bae2ca5f3f55" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:18.862250  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:18.870230  5869 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:18.874485  5915 consensus_peers.cc:597] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:18.821277 13756 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:18.885624 13672 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:18.895479 13756 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:18.895988 13681 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:18.897514 13852 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:18.899801  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:18.899957  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:18.879558 13668 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:28:18.882023 13928 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.910638 13928 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:18.914629  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfa3fcaa83734d509f6b31c3874873be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:18.866593 13884 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:28:18.919883  5871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52af756fdd9b411c8c4b494e2552a3fb" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:18.925803 13921 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:18.926537  6022 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:28:18.928141  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed8b895df664a3b8f7934c37fd8f181" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:18.928798  6022 raft_consensus.cc:3060] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:18.934051  6609 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:18.936529 13884 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:18.936769  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:18.938046  2067 raft_consensus.cc:2243] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:28:18.938694 13928 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:18.943718 13539 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.944521 13539 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:18.945871  5874 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:28:18.945384  5867 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:18.949816 13539 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:18.950349  5865 raft_consensus.cc:1275] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:28:18.958835  6023 raft_consensus.cc:3055] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:18.959499  6023 raft_consensus.cc:740] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:18.959254 13799 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:18.960638  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:18.967077  5840 consensus_peers.cc:597] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:18.973248  2067 raft_consensus.cc:2272] T 4fa58bdc1b834feda10d16a00cd81355 P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:18.981369  5990 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:19.003661  5941 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:28:19.007946  6609 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:19.010434 13698 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:19.010943 13794 consensus_queue.cc:1048] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:18.960649  6023 consensus_queue.cc:260] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:19.013372  6023 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:19.015473  5942 raft_consensus.cc:1275] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:28:19.018406 13799 consensus_queue.cc:1175] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7a0204b931394fc48ffc503716e0d111" responder_term: 23 status { last_received { term: 23 index: 11 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:28:19.036357  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c69fa4d49547f8a9e76c055c3ae7be" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:19.040927  6023 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 11. (term mismatch)
I20260709 12:28:19.041515  6023 pending_rounds.cc:85] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 10
I20260709 12:28:19.048456  6023 pending_rounds.cc:107] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 22.11
I20260709 12:28:19.048781  6023 raft_consensus.cc:2889] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:19.055814 13885 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:19.096192  5869 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
W20260709 12:28:19.097918  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 63095.5us
W20260709 12:28:19.097955  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 90240.3us
I20260709 12:28:19.099366  6609 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:19.100353  6022 raft_consensus.cc:2468] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:28:19.118146  5948 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Refusing update from remote peer 1fc57407d56241b38a94d543e073f8af: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 12:28:19.118857  5948 pending_rounds.cc:85] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting all ops after (but not including) 11
I20260709 12:28:19.119297  5948 pending_rounds.cc:107] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 12:28:19.119585  5948 raft_consensus.cc:2889] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:19.125387  6609 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:19.111223 13929 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:19.111384 13921 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:28:19.136603 13383 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:19.137779  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:19.131307 13417 raft_consensus.cc:493] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:19.139604 13794 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:28:19.154778  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 45316.8us
I20260709 12:28:19.150018  2067 raft_consensus.cc:2243] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:19.108321 13672 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:28:19.159704  5869 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:19.161130 13794 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:19.167833  5840 consensus_peers.cc:597] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:19.172073  5840 consensus_peers.cc:597] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:19.172278  5913 leader_election.cc:304] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:19.174675  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:19.155756 13885 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:28:19.163939 13776 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:28:19.184562 13500 raft_consensus.cc:2804] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:19.158596 13417 raft_consensus.cc:515] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:19.189898  5946 raft_consensus.cc:1240] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:19.190625  2067 raft_consensus.cc:2272] T 85d9b48bdd4046d788b310a73871c31c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:19.171574 13929 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.194715 13799 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:19.197721 13500 raft_consensus.cc:697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:19.198987 13540 consensus_queue.cc:1048] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:19.191847 13903 raft_consensus.cc:493] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.223022  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:19.223898  6018 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:28:19.191753 13875 consensus_queue.cc:1048] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.042s
W20260709 12:28:19.267988  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 57242.7us
I20260709 12:28:19.265393  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:19.251471 13885 raft_consensus.cc:493] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:19.288976  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 57286.2us
I20260709 12:28:19.250965 13921 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:19.209441 13929 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:19.315896 13500 consensus_queue.cc:237] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:19.320173 13938 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:28:19.321890 13933 log.cc:927] Time spent T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
W20260709 12:28:19.322413 13934 log.cc:927] Time spent T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:28:19.324272  5941 raft_consensus.cc:1240] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Rejecting Update request from peer 6add331326ca47038308a447bf798fa7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:19.326557 13794 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:19.285287 13885 raft_consensus.cc:515] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:19.311338 13921 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:19.328939 13885 leader_election.cc:290] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:19.329316 13921 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:19.333616  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:19.339531  5940 raft_consensus.cc:2393] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 23: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:19.336639  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:19.340368  6016 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:28:19.342175  5840 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:19.351390  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:19.301252 13417 leader_election.cc:290] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:19.243583 13903 raft_consensus.cc:515] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:19.355644  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 79437us
I20260709 12:28:19.317760 13846 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:19.364898  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef7a99e80494a989eda0b1a37482656" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:19.371337 13799 consensus_queue.cc:1243] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer 7a0204b931394fc48ffc503716e0d111 log is divergent from this leader: its last log entry 23.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:28:19.373957  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:19.353754 13834 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:19.379230  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e301d9d5b914f3b9ec07dea273d5126" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:19.381928  5987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.387276  6609 leader_election.cc:304] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:19.389555  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:19.390376  5869 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 27.
I20260709 12:28:19.357471 13772 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:19.368104 13776 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:19.382735 13834 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:19.399605 13772 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:19.401160 13903 leader_election.cc:290] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:19.402824  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:19.403240  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:19.404150  5948 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:28:19.398161  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:19.392066 13709 raft_consensus.cc:493] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.428143 13709 raft_consensus.cc:515] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:19.416370  5990 leader_election.cc:336] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.430660 13709 leader_election.cc:290] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:19.379917 13815 consensus_queue.cc:1059] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:28:19.402673 13689 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:19.431963 13689 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:19.421144 13954 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:19.410523 13834 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:19.439710 13799 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:19.440343 13799 raft_consensus.cc:493] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:19.440722 13799 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:19.425213  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:19.411592 13776 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:19.445747 13776 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:19.446239  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 48158.4us
I20260709 12:28:19.403430 13826 raft_consensus.cc:2749] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:19.452035  2067 raft_consensus.cc:2243] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:28:19.432385 13689 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:19.453225  2067 raft_consensus.cc:2272] T 595c6563cdd74f47bda68ead57106a8b P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:19.454413 13941 raft_consensus.cc:3055] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:19.455132 13941 raft_consensus.cc:740] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:19.448385 13772 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:19.424299 13929 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:19.380875 13846 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:19.464870  5873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e55fd31b2e414885a9bd5c9b40e04e19" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:19.467626  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:19.468618  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 51912.8us
I20260709 12:28:19.486909  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59cab2d7e5234efd9a97b8377f1919f7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:19.481493  5987 leader_election.cc:304] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:19.492349  6609 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:19.495268  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 82652.4us
W20260709 12:28:19.523481  5915 consensus_peers.cc:597] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:19.525182  6609 leader_election.cc:304] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:19.525375  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30da8f1ded30432a87f01684d88a9969" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:19.481030 13730 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.467252 13945 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:19.536921 13730 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:19.537038 13945 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
W20260709 12:28:19.538245  5915 consensus_peers.cc:597] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:19.539646 13945 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:19.533783  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:19.540580  6018 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 26.
I20260709 12:28:19.539693 13730 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:19.542003 13730 raft_consensus.cc:2749] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:19.548483  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:19.517403 13417 raft_consensus.cc:2749] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:19.561949  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 153927us
I20260709 12:28:19.520390 13383 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:19.456208 13941 consensus_queue.cc:260] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:19.552958  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:19.571522  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 48914us
W20260709 12:28:19.586426  5990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.587036  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:19.587867  5942 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 26.
I20260709 12:28:19.588798 13941 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:19.591216  5915 leader_election.cc:304] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:19.587070 13799 raft_consensus.cc:515] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:19.592947 13958 log.cc:927] Time spent T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:28:19.574476 13815 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:19.594115 13815 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:19.606550  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:19.600191  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:19.613080  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:28:19.634878 13960 log.cc:927] Time spent T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:28:19.630805 13417 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:19.637595 13417 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:19.642692 13417 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:19.629946 13926 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:19.643630  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:19.643893 13926 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:19.644452 13926 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:19.649204 13902 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.649729 13846 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:19.649933 13940 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:28:19.596313 13815 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:19.594772 13799 leader_election.cc:290] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:19.600401 13681 raft_consensus.cc:2749] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:19.596221 13886 raft_consensus.cc:493] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.656982 13417 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:19.657191 13886 raft_consensus.cc:515] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:19.640342  6016 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 12:28:19.672329  6016 pending_rounds.cc:85] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Aborting all ops after (but not including) 11
I20260709 12:28:19.672700  6016 pending_rounds.cc:107] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 12:28:19.673036  6016 raft_consensus.cc:2889] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:19.679080  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:28:19.667900  5990 leader_election.cc:336] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:19.681339  5990 leader_election.cc:336] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.683147  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:19.625206 13939 log.cc:927] Time spent T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:28:19.687788  2067 raft_consensus.cc:2243] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:19.685370  5990 leader_election.cc:336] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.688817  5990 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:19.638978 13961 log.cc:927] Time spent T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:28:19.690825 13834 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:19.695829 13834 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:19.704383 13967 raft_consensus.cc:2749] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:19.660064 13886 leader_election.cc:290] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:19.666098 13417 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:19.720929  2067 pending_rounds.cc:70] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Trying to abort 2 pending ops.
I20260709 12:28:19.721517  2067 pending_rounds.cc:77] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Aborting op as it isn't in flight: id { term: 21 index: 13 } timestamp: 7305625942907125760 op_type: NO_OP noop_request { }
I20260709 12:28:19.722005  2067 raft_consensus.cc:2889] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:19.722455  2067 pending_rounds.cc:77] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af: Aborting op as it isn't in flight: id { term: 20 index: 12 } timestamp: 7305625917128278016 op_type: NO_OP noop_request { }
I20260709 12:28:19.722841  2067 raft_consensus.cc:2889] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:19.723311  2067 raft_consensus.cc:2272] T d45107f59eee48f79ae2b6006319cd62 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:19.661938 13926 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:19.731033  5948 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 27: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:28:19.733086 13956 log.cc:927] Time spent T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:28:19.734129 13689 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:19.736922 13689 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:19.649953 13902 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:19.740796 13902 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:19.742089  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:19.743966  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:19.747195  6019 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:19.748832  5869 raft_consensus.cc:3060] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:19.751737 13799 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:19.763777  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:19.764688  2067 raft_consensus.cc:2243] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 29 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:19.769351  5990 leader_election.cc:336] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:19.772449  5915 consensus_peers.cc:597] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:19.781996 13885 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.782735 13885 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:19.785147 13885 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:19.786041  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:19.795346  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:19.798138  5915 leader_election.cc:336] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:19.799306  5915 leader_election.cc:336] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.799706  5915 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
W20260709 12:28:19.801087  5915 consensus_peers.cc:597] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:19.801910  5915 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:19.769929 13799 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:19.803279 13919 raft_consensus.cc:2749] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:19.802376 13949 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:19.810375 13926 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:19.810932 13949 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:19.811614 13875 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:19.813302 13949 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:19.802897  2067 raft_consensus.cc:2272] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 1fc57407d56241b38a94d543e073f8af [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:19.843489  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:19.844506  2067 raft_consensus.cc:2243] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:19.845116  2067 raft_consensus.cc:2272] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:19.827173  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:19.822209  5915 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:19.824373  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:19.850596  5942 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
W20260709 12:28:19.852113  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 184100us
I20260709 12:28:19.833964  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ab14137002240c2909503f8b4e55c79" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:19.853865  6019 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:28:19.854660  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:19.855050  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07d59a4ebda543dfbfa20847a8e9e4a7" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 28 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:19.855618  6022 raft_consensus.cc:2393] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 27: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:19.861805  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:19.862607  6022 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
I20260709 12:28:19.864503  5915 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:19.852401  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:19.864646  6611 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:19.867040 13875 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:19.867564 13875 raft_consensus.cc:493] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:19.867933 13875 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:19.869419 13838 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:19.858232 13776 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:19.871017 13776 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:19.876938 13954 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:19.833451 13799 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:19.880172 13776 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:19.882788  5870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9bb01883059454e80b19d5333aad71e" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 27 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:19.883386  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "860d4f6b513041829ed9cd83880edc5c" candidate_uuid: "1fc57407d56241b38a94d543e073f8af" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:19.894842  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:19.885342  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:19.897836  6016 raft_consensus.cc:2368] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24.
I20260709 12:28:19.895669  6015 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:28:19.903314  5869 raft_consensus.cc:2468] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
W20260709 12:28:19.906590  5990 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:19.908469  5990 leader_election.cc:336] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.901988 13926 raft_consensus.cc:2804] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:19.920049  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:19.921082  2067 raft_consensus.cc:2243] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:19.922389  2067 pending_rounds.cc:70] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Trying to abort 1 pending ops.
I20260709 12:28:19.922657  2067 pending_rounds.cc:77] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af: Aborting op as it isn't in flight: id { term: 22 index: 9 } timestamp: 7305625927095619584 op_type: NO_OP noop_request { }
I20260709 12:28:19.923007  2067 raft_consensus.cc:2889] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:19.923538  2067 raft_consensus.cc:2272] T d04ce59a08f3427581e5e6b5d87ad757 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:19.925441  5869 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:28:19.930778  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:19.930850  6609 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:19.937125  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:19.937160  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ba7af1a4a744df6bc8c20ec72e21519" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:19.938024  5867 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 22.
I20260709 12:28:19.938519  5869 raft_consensus.cc:3060] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:19.942945  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "256f8c690fb2456f92b4035fc3bf5e23" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:19.944782  6609 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:19.945186  5869 raft_consensus.cc:2468] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 28.
I20260709 12:28:19.946718  6019 raft_consensus.cc:1275] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:28:19.926213 13926 raft_consensus.cc:697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:28:19.952023  5915 leader_election.cc:336] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.946135 13776 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:19.954293 13645 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:19.954754 13776 raft_consensus.cc:493] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:19.955195 13776 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:19.935487 13929 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:19.942265 13900 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 12:28:19.957829 13168 consensus_queue.cc:1048] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:19.961350 13926 consensus_queue.cc:237] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:19.953006  5915 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:19.966434 13900 raft_consensus.cc:2749] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:19.969928 13776 raft_consensus.cc:515] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:19.981341 13776 leader_election.cc:290] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:19.977671  5915 leader_election.cc:336] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:19.994555  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99b9212afb8b459bb96908f8259bb8c0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:19.995386  6016 raft_consensus.cc:3060] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:19.997288  6609 leader_election.cc:304] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fc57407d56241b38a94d543e073f8af; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:20.006134  6016 raft_consensus.cc:2468] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:28:19.995949 13875 raft_consensus.cc:515] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:20.012779  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681f64c726104caf875ae7d4329d0676" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:20.013670  5941 raft_consensus.cc:3060] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:20.019016 13875 leader_election.cc:290] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:20.019706  5990 leader_election.cc:336] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.021323  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:20.023434  5915 leader_election.cc:304] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:20.023902 13949 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:20.024335 13949 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:20.024731 13949 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:20.030520  5987 leader_election.cc:304] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:28:20.042092  6609 raft_consensus.cc:2686] T e9bb01883059454e80b19d5333aad71e P 1fc57407d56241b38a94d543e073f8af: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:28:20.034868  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:20.051553  5840 leader_election.cc:400] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24.
I20260709 12:28:20.052007  5840 leader_election.cc:403] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
W20260709 12:28:20.049525  6609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 52366.5us
W20260709 12:28:20.054268  5915 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:20.059022  5941 raft_consensus.cc:2468] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 23.
I20260709 12:28:20.072047  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.073169  2067 raft_consensus.cc:2243] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.073719  5987 leader_election.cc:304] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:20.048115 13900 raft_consensus.cc:2804] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:20.058251 13921 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:20.058287 13794 raft_consensus.cc:2804] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:20.057987 13966 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835) with higher term. Message: Invalid argument: T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 for earlier term 23. Current term is 24.
I20260709 12:28:20.086155 13921 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:20.089174 13957 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.086400 13900 raft_consensus.cc:697] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:28:20.093683  5691 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40830 (request call id 152) took 60758 ms (client timeout 15000 ms). Trace:
I20260709 12:28:20.089502 13921 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:20.086297 13799 raft_consensus.cc:2804] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:20.095779 13794 raft_consensus.cc:697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 28 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:28:20.124356  5840 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:20.125660  5840 consensus_peers.cc:597] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:20.095330  5691 rpcz_store.cc:269] 0709 12:27:19.333062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.362445 (+ 29383us) service_pool.cc:224] Handling call
0709 12:28:20.090944 (+60728499us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:18.599655 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:18.600465 (+   810us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:18.600536 (+    71us) write_op.cc:435] Acquired schema lock
0709 12:28:18.600557 (+    21us) tablet.cc:662] Decoding operations
0709 12:28:18.620409 (+ 19852us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:18.638047 (+ 17638us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:18.641937 (+  3890us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:28:18.800644 (+158707us) tablet.cc:701] Row locks acquired
0709 12:28:18.800707 (+    63us) write_op.cc:260] PREPARE: finished
0709 12:28:18.800930 (+   223us) write_op.cc:270] Start()
0709 12:28:18.801119 (+   189us) write_op.cc:276] Timestamp: P: 1783600098800846 usec, L: 0
0709 12:28:18.801144 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:28:18.802327 (+  1183us) log.cc:844] Serialized 4355 byte log entry
0709 12:28:19.086115 (+283788us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b1800334000. stack: 00007f39cfb64343 00007f39cfb64240 00007f39cf4f31d4 00007f39d9231304 00007f39d9231299 00007f39d8a5eab1 00007f39d8a6b281 00007f39cfb963f3 00007f39cfb987c1 00007f39cfb98779 00007f39cfb98709 00007f39cfb986d1 00007f39cfb979cd 0000000000388634 0000000000388479 00007f39cfb7983a
0709 12:28:19.299693 (+213578us) op_driver.cc:464] REPLICATION: finished
0709 12:28:19.581087 (+281394us) write_op.cc:301] APPLY: starting
0709 12:28:19.625050 (+ 43963us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:19.702410 (+ 77360us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:19.702464 (+    54us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:19.788815 (+ 86351us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:19.789155 (+   340us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 12:28:19.789207 (+    52us) write_op.cc:312] APPLY: finished
0709 12:28:19.789614 (+   407us) log.cc:844] Serialized 104 byte log entry
0709 12:28:19.789846 (+   232us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:19.790024 (+   178us) write_op.cc:454] Released schema lock
0709 12:28:19.790079 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:19.790705 (+   626us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:19.790743 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1958878,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":251461,"apply.run_cpu_time_us":18937,"apply.run_wall_time_us":214905,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50151,"num_ops":11,"prepare.queue_time_us":495799,"prepare.run_cpu_time_us":74905,"prepare.run_wall_time_us":405647,"raft.queue_time_us":210844,"raft.run_cpu_time_us":51130,"raft.run_wall_time_us":172766,"replication_time_us":498366,"spinlock_wait_cycles":4995456,"thread_start_us":621406,"threads_started":4,"wal-append.queue_time_us":252616}]]}
W20260709 12:28:20.128159  5915 leader_election.cc:336] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.126482 13900 consensus_queue.cc:237] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:20.127077 13799 raft_consensus.cc:697] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 23 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:20.093225 13957 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:20.156869  2067 raft_consensus.cc:2272] T 02e8ec653e484ba58498cc78f412de75 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:20.162256  5990 consensus_peers.cc:597] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:20.161988 13794 consensus_queue.cc:237] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:20.199582 13957 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:20.184707 13799 consensus_queue.cc:237] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:20.201974  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.202980  2067 raft_consensus.cc:2243] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.203557  2067 raft_consensus.cc:2272] T e55fd31b2e414885a9bd5c9b40e04e19 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.211369  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:20.221434  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.222430  2067 raft_consensus.cc:2243] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 25 LEADER]: Raft consensus shutting down.
W20260709 12:28:20.219357  5915 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:20.228760 13921 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:20.230304 13875 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:20.249533 13875 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:20.252626 13875 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:20.253546  5948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:20.258051  2067 raft_consensus.cc:2272] T 238b79ac4a244a85a83ab0d01b84e117 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:20.239138  5840 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.242133 13168 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:20.247200 13949 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:20.259433  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:20.262504 13168 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:20.264894 13949 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:20.264993  5948 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 21.
I20260709 12:28:20.272403  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:20.290794  6015 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 21.
I20260709 12:28:20.291736 13921 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:20.290477  5690 catalog_manager.cc:5697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 10 to 16, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 16 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:20.286567  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 47458.2us
I20260709 12:28:20.268137  5987 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:20.297199 13875 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:20.297647 13875 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:20.298036 13875 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:20.301009  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:28:20.301734  5990 leader_election.cc:336] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.301978  5867 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 23: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:20.301211  5690 catalog_manager.cc:5697] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 12 to 17, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 17 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:20.283535  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.310878  2067 raft_consensus.cc:2243] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.311812  2067 raft_consensus.cc:2272] T c3c69fa4d49547f8a9e76c055c3ae7be P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:20.285413  5915 leader_election.cc:336] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.286386 13972 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:20.275507 13949 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:20.280464 13799 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:28:20.306529  5840 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.265522 13168 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:20.314615  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
W20260709 12:28:20.309165  5990 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.318091 13900 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:20.318648 13900 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:20.319010 13900 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:20.315984 13949 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:20.324867 13949 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:20.332024  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.333069  2067 raft_consensus.cc:2243] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.310918  5690 catalog_manager.cc:5697] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 10 to 19. New cstate: current_term: 19 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:20.308359  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:20.338243  6015 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 23: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:20.340071  5690 catalog_manager.cc:5697] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:28:20.340270 13875 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:20.337782 13799 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:20.341739  5690 catalog_manager.cc:5697] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:28:20.343240 13799 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:20.345089 13875 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:20.348866  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:20.359993  5869 raft_consensus.cc:2393] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 21: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:20.338316  2067 raft_consensus.cc:2272] T aa54a35cf4c54bb1a1c28658c11d174c P 1fc57407d56241b38a94d543e073f8af [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.362243  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:20.362782 13900 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:20.364298  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.364260  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:20.354168  5990 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:20.365429  5941 raft_consensus.cc:2393] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 22: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
W20260709 12:28:20.366351  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 57222.6us
I20260709 12:28:20.368911 13875 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:20.369923  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:20.370864 13921 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:20.374346  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:20.375205  6015 raft_consensus.cc:2393] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 22: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:20.379000  5690 catalog_manager.cc:5697] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:20.390877  5915 leader_election.cc:336] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.394042  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:20.395102 13928 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:20.387377 13900 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 22 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:20.396595  5690 catalog_manager.cc:5697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 14 to 21, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 21 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:20.397439  6015 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:28:20.365414  2067 raft_consensus.cc:2243] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.403668  2067 raft_consensus.cc:2272] T cfa3fcaa83734d509f6b31c3874873be P 1fc57407d56241b38a94d543e073f8af [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.404342 13900 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:20.407259  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.408337  2067 raft_consensus.cc:2243] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.409022  2067 pending_rounds.cc:70] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Trying to abort 1 pending ops.
I20260709 12:28:20.415050  5869 raft_consensus.cc:1275] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:28:20.420272  5690 catalog_manager.cc:5697] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 20 to 21, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 21 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:20.421582 13168 consensus_queue.cc:1048] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:20.436947  5690 catalog_manager.cc:5697] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 20 to 21, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 21 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:20.439254  5690 catalog_manager.cc:5697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:20.446615  5690 catalog_manager.cc:5697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 11 to 20, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 20 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:20.455338  5990 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.462039  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:20.456022  2067 pending_rounds.cc:77] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af: Aborting op as it isn't in flight: id { term: 20 index: 14 } timestamp: 7305625933503422464 op_type: NO_OP noop_request { }
I20260709 12:28:20.462992  2067 raft_consensus.cc:2889] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:20.463533 13875 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:20.464687  5990 leader_election.cc:336] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.488448  5990 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:20.489542 13799 raft_consensus.cc:2749] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:20.491020  5990 leader_election.cc:336] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.491556  5990 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:20.492415 13799 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:20.507390 13875 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:20.541324  2067 raft_consensus.cc:2272] T 45edb6642b84468aa6fc19a7fd2ddf6c P 1fc57407d56241b38a94d543e073f8af [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.545351  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.553985  2067 raft_consensus.cc:2243] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.554684  2067 pending_rounds.cc:70] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Trying to abort 1 pending ops.
I20260709 12:28:20.554983  2067 pending_rounds.cc:77] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af: Aborting op as it isn't in flight: id { term: 23 index: 12 } timestamp: 7305625823928561664 op_type: NO_OP noop_request { }
I20260709 12:28:20.555359  2067 raft_consensus.cc:2889] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:20.556411  2067 raft_consensus.cc:2272] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.559633  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.560503  2067 raft_consensus.cc:2243] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.561589  2067 raft_consensus.cc:2272] T 428a50885681465a98089cb853b233b9 P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.565268  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.566198  2067 raft_consensus.cc:2243] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.566915  2067 raft_consensus.cc:2272] T 59cab2d7e5234efd9a97b8377f1919f7 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.570768  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.571705  2067 raft_consensus.cc:2243] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.572911  2067 raft_consensus.cc:2272] T ba269f0998844ab5b461587eaa90df32 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.588276  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.589313  2067 raft_consensus.cc:2243] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.591667  2067 raft_consensus.cc:2272] T edbd0e138da244d48f133d23b4aa1eda P 1fc57407d56241b38a94d543e073f8af [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.604071  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.604908  2067 raft_consensus.cc:2243] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.605527  2067 raft_consensus.cc:2272] T 1ab14137002240c2909503f8b4e55c79 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.608764  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.609557  2067 raft_consensus.cc:2243] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.610267  2067 raft_consensus.cc:2272] T 2dcbedb48e03420dbf7f8646e65b1886 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.613478  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.614463  2067 raft_consensus.cc:2243] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.615140  2067 raft_consensus.cc:2272] T 75e8568a9892438c91e30ddc62fef67c P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.618296  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.626748 13900 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:20.627887 13900 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:20.619177  2067 raft_consensus.cc:2243] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.629894  2067 raft_consensus.cc:2272] T 52af756fdd9b411c8c4b494e2552a3fb P 1fc57407d56241b38a94d543e073f8af [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.633661  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.634490  2067 raft_consensus.cc:2243] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.635068  2067 pending_rounds.cc:70] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Trying to abort 1 pending ops.
I20260709 12:28:20.635358  2067 pending_rounds.cc:77] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af: Aborting op as it isn't in flight: id { term: 21 index: 14 } timestamp: 7305625925322354688 op_type: NO_OP noop_request { }
I20260709 12:28:20.635728  2067 raft_consensus.cc:2889] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:20.636124  2067 raft_consensus.cc:2272] T 4ef7a99e80494a989eda0b1a37482656 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.638613 13900 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:20.639464  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.640386  2067 raft_consensus.cc:2243] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.641606  2067 raft_consensus.cc:2272] T 048b40ab425f451c983713b7df96c732 P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.644915  5690 catalog_manager.cc:5697] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:20.648334  5690 catalog_manager.cc:5697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 18 to 25, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 25 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:28:20.652078  5690 catalog_manager.cc:5697] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 15 to 20, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 20 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:28:20.653982  5690 catalog_manager.cc:5697] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 10 to 18, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 18 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:20.655565  5690 catalog_manager.cc:5697] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 15 to 22, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 22 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:20.660125  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.661123  2067 raft_consensus.cc:2243] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.664149  2067 raft_consensus.cc:2272] T e657475654794984a13c42a668f3158b P 1fc57407d56241b38a94d543e073f8af [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.692588  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.693337  2067 raft_consensus.cc:2243] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.694144  2067 raft_consensus.cc:2272] T 860d4f6b513041829ed9cd83880edc5c P 1fc57407d56241b38a94d543e073f8af [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.869241  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:20.869984  6019 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:20.882761 13955 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:20.883440 13955 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:20.888420  5869 raft_consensus.cc:1275] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 28 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:28:20.892414  5941 raft_consensus.cc:1275] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:28:20.896840 13799 consensus_queue.cc:1048] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:20.885562 13955 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:20.902760  5990 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:20.903512  5990 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:20.904058  5990 consensus_peers.cc:597] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:20.907747  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:20.912140 13799 consensus_queue.cc:1048] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:20.928303  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.929303  2067 raft_consensus.cc:2243] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.934469  6019 raft_consensus.cc:1275] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 12:28:20.935305  5990 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:20.943195  5990 consensus_peers.cc:597] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:20.944291  2067 raft_consensus.cc:2272] T 9bede7b5a3c84fe28a073542219f3fe4 P 1fc57407d56241b38a94d543e073f8af [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.947249  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.947958  2067 raft_consensus.cc:2243] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.948910  2067 raft_consensus.cc:2272] T 18a324a652924631995e78ee3fda8422 P 1fc57407d56241b38a94d543e073f8af [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.957252  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:20.963550 13799 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:20.965850 13799 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:20.968530 13799 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:20.974066  5990 leader_election.cc:336] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:20.974737 13986 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:20.975291 13986 raft_consensus.cc:493] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:20.975672 13986 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:20.978536  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.979421  2067 raft_consensus.cc:2243] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.980715  2067 raft_consensus.cc:2272] T e42697dffe5a495ea1ba4a111df5ba39 P 1fc57407d56241b38a94d543e073f8af [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.983779  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:20.984743  2067 raft_consensus.cc:2243] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.985376 13986 raft_consensus.cc:515] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:20.985405  2067 pending_rounds.cc:70] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Trying to abort 1 pending ops.
I20260709 12:28:20.986627  2067 pending_rounds.cc:77] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af: Aborting op as it isn't in flight: id { term: 21 index: 11 } timestamp: 7305625934090797056 op_type: NO_OP noop_request { }
I20260709 12:28:20.986966  2067 raft_consensus.cc:2889] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:20.988073 13986 leader_election.cc:290] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:20.989545  2067 raft_consensus.cc:2272] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 1fc57407d56241b38a94d543e073f8af [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.990633 13168 consensus_queue.cc:1048] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:28:20.998498  5915 consensus_peers.cc:597] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:21.010123  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d9b48bdd4046d788b310a73871c31c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:21.011018  6019 raft_consensus.cc:3060] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:21.020787  6019 raft_consensus.cc:2468] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
W20260709 12:28:21.044363  5915 leader_election.cc:336] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:21.046043  5915 consensus_peers.cc:597] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:21.048350  5915 leader_election.cc:304] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:21.049352 13168 raft_consensus.cc:2804] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:21.049929 13168 raft_consensus.cc:697] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:21.050973 13168 consensus_queue.cc:237] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:21.058518  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:21.059371  5869 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 25.
W20260709 12:28:21.061403  5840 consensus_peers.cc:597] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:21.062803  5840 consensus_peers.cc:597] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:21.063563  5840 consensus_peers.cc:597] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:21.064201  5840 leader_election.cc:336] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.064724  5840 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:21.066917 13955 raft_consensus.cc:2749] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:21.069202  5915 leader_election.cc:336] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.071852  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:21.074011 13417 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:21.074862 13417 raft_consensus.cc:493] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:21.075274 13417 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:21.087661 13417 raft_consensus.cc:515] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:21.093662  5990 leader_election.cc:336] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:21.094508 13417 leader_election.cc:290] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:21.094743  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595c6563cdd74f47bda68ead57106a8b" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:21.095520  5865 raft_consensus.cc:3060] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:21.109472  5865 raft_consensus.cc:2468] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 26.
I20260709 12:28:21.110795  5987 leader_election.cc:304] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:21.112265 13417 raft_consensus.cc:2804] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:21.113090 13417 raft_consensus.cc:697] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 26 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:21.114200 13417 consensus_queue.cc:237] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:21.145766  5840 consensus_peers.cc:597] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:21.304385  5691 catalog_manager.cc:5697] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:28:21.329252 13903 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:21.330654 13903 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:21.333858 13903 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:21.336815  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:21.338690  6015 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 25.
I20260709 12:28:21.338496  5691 catalog_manager.cc:5697] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:21.361929  5691 catalog_manager.cc:5697] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 17 to 18, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 18 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:28:21.367188  5691 catalog_manager.cc:5697] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 19 to 23, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 23 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:28:21.371493  5691 catalog_manager.cc:5697] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 16 to 22, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 22 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:21.365268  5690 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40818 (request call id 152) took 18602 ms (client timeout 15000 ms). Trace:
W20260709 12:28:21.375530  5840 consensus_peers.cc:597] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:21.376917  5840 consensus_peers.cc:597] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:21.372761  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:21.379120 13168 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:21.379631 13168 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:21.380092 13168 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:21.375692  5690 rpcz_store.cc:269] 0709 12:28:02.763352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.422003 (+2658651us) service_pool.cc:224] Handling call
0709 12:28:21.364903 (+15942900us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:20.679096 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:20.679699 (+   603us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:20.679735 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:28:20.679755 (+    20us) tablet.cc:662] Decoding operations
0709 12:28:20.936500 (+256745us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:20.936626 (+   126us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:20.936664 (+    38us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:28:20.938223 (+  1559us) tablet.cc:701] Row locks acquired
0709 12:28:20.938247 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:28:20.938414 (+   167us) write_op.cc:270] Start()
0709 12:28:20.938544 (+   130us) write_op.cc:276] Timestamp: P: 1783600100938356 usec, L: 0
0709 12:28:20.938564 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:28:20.939339 (+   775us) log.cc:844] Serialized 6265 byte log entry
0709 12:28:21.060883 (+121544us) op_driver.cc:464] REPLICATION: finished
0709 12:28:21.078721 (+ 17838us) write_op.cc:301] APPLY: starting
0709 12:28:21.078838 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:21.103546 (+ 24708us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:21.116106 (+ 12560us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:21.140996 (+ 24890us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:21.141344 (+   348us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 12:28:21.141400 (+    56us) write_op.cc:312] APPLY: finished
0709 12:28:21.144262 (+  2862us) log.cc:844] Serialized 147 byte log entry
0709 12:28:21.144489 (+   227us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:21.144703 (+   214us) write_op.cc:454] Released schema lock
0709 12:28:21.144766 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:21.145219 (+   453us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:21.145249 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":738678,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":17536,"apply.run_cpu_time_us":21966,"apply.run_wall_time_us":66506,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71071,"num_ops":16,"prepare.queue_time_us":16658,"prepare.run_cpu_time_us":5927,"prepare.run_wall_time_us":294864,"raft.queue_time_us":100279,"raft.run_cpu_time_us":2516,"raft.run_wall_time_us":9044,"replication_time_us":122155,"thread_start_us":355591,"threads_started":4,"wal-append.queue_time_us":9331}]]}
I20260709 12:28:21.387077  5691 catalog_manager.cc:5697] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 13 to 22, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 22 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
W20260709 12:28:21.393339  5915 connection.cc:769] client connection to 127.2.4.193:38701 send error: Network error: sendmsg error: Connection reset by peer (error 104)
W20260709 12:28:21.404314  5915 leader_election.cc:336] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: sendmsg error: Connection reset by peer (error 104)
I20260709 12:28:21.412536  5691 catalog_manager.cc:5697] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:28:21.430107  5691 catalog_manager.cc:5697] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 20 to 22, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 22 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:21.457914  5691 catalog_manager.cc:5697] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 15 to 24. New cstate: current_term: 24 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:28:21.489451  5691 catalog_manager.cc:5697] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 18 to 25, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 25 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.500046  5691 catalog_manager.cc:5697] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 20 to 25, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 25 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.502913  5691 catalog_manager.cc:5697] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 13 to 21, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 21 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.506016  5691 catalog_manager.cc:5697] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 17 to 21, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 21 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:28:21.508025 13794 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:21.509235 13794 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:21.508617  5691 catalog_manager.cc:5697] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 17 to 22, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 22 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:28:21.511869  5691 catalog_manager.cc:5697] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 22 to 24, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 24 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:28:21.513747  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:21.514053  5691 catalog_manager.cc:5697] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:21.516374  5990 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:21.516949  5691 catalog_manager.cc:5697] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.527385 13168 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:21.530059 13794 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:21.531543 13168 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:21.531563  5691 catalog_manager.cc:5697] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 20 to 23. New cstate: current_term: 23 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.534740 13799 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.538434 13799 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:21.540982 13799 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:21.542184  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238b79ac4a244a85a83ab0d01b84e117" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:21.542757  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:21.543540  5942 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 27.
W20260709 12:28:21.544179  5990 leader_election.cc:336] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:21.544351  5915 consensus_peers.cc:597] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:21.542937  6015 raft_consensus.cc:3060] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:21.544767  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:21.545002  5990 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:21.545850 13799 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:21.546026 13794 raft_consensus.cc:2749] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:21.546420 13799 raft_consensus.cc:493] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.547025 13799 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:21.551020  5869 raft_consensus.cc:1275] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 12:28:21.552891  5990 leader_election.cc:336] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:21.554360  5691 catalog_manager.cc:5697] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 18 to 27, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 27 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:28:21.557402  5691 catalog_manager.cc:5697] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:28:21.569814  5691 catalog_manager.cc:5697] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 26 to 28, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 28 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
W20260709 12:28:21.583042  5915 consensus_peers.cc:597] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:21.583086  6016 raft_consensus.cc:1275] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 12:28:21.586621  5915 leader_election.cc:336] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:21.584682 13794 consensus_queue.cc:1048] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.009s
I20260709 12:28:21.593441  6015 raft_consensus.cc:2468] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 26.
I20260709 12:28:21.607430  5915 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:21.608457 13168 raft_consensus.cc:2804] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:21.609347 13168 raft_consensus.cc:697] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:21.608263 13500 consensus_queue.cc:1048] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:28:21.576792 13799 raft_consensus.cc:515] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:21.611588 13794 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:21.611374 13168 consensus_queue.cc:237] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:21.612488 13794 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:21.613495 13799 leader_election.cc:290] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 election: Requested vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:21.615042 13794 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:21.619207  5691 catalog_manager.cc:5697] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 14 to 26, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 26 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
W20260709 12:28:21.589844  5840 consensus_peers.cc:597] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:21.626921  5691 catalog_manager.cc:5697] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 22 to 23, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 23 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:28:21.630064  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:21.633790 13500 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:21.634661 13500 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:21.637861  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:21.638513  6019 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 25.
W20260709 12:28:21.627686  5990 leader_election.cc:336] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:21.640028  5990 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:21.640250  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:21.641599  5987 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:21.642228 13900 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:21.642617 13900 raft_consensus.cc:493] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:21.642853 13987 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:21.643042 13900 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:21.643623 13987 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:21.647390  5840 consensus_peers.cc:597] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:21.629014  5691 catalog_manager.cc:5697] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 20 to 23. New cstate: current_term: 23 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:28:21.654816 13989 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:21.649061  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bc2065e43e40c98ee821925758a3f3" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:21.658747  5942 raft_consensus.cc:3060] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:21.657517  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:21.667608 13987 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:21.670327 14005 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:21.671459 13500 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:21.683293  5942 raft_consensus.cc:2468] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 28.
I20260709 12:28:21.686272 13794 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:21.691093 13794 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:21.699023  5915 leader_election.cc:336] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:21.699079  5990 consensus_peers.cc:597] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:21.701794 13989 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:21.703766  5990 leader_election.cc:336] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:21.704396 13989 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:21.704950 13794 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:21.706485  5915 consensus_peers.cc:597] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:21.706661  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:21.709978  5691 catalog_manager.cc:5697] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:21.712111  5691 catalog_manager.cc:5697] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 17 to 20, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 20 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:28:21.714233  5691 catalog_manager.cc:5697] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 19 to 20, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 20 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:28:21.715144  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:21.716005  5869 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 23.
W20260709 12:28:21.717243  5840 leader_election.cc:336] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:21.717770  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:21.718752 13989 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:21.724012  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:21.727418 13799 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:21.728060 13799 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:21.728564 13799 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:21.735762 13900 raft_consensus.cc:515] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:21.741945  5691 catalog_manager.cc:5697] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:28:21.744112  5691 catalog_manager.cc:5697] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:21.746549  5987 leader_election.cc:304] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:21.747641 14005 raft_consensus.cc:2804] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:21.748430 14005 raft_consensus.cc:697] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 28 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
W20260709 12:28:21.749281  5840 leader_election.cc:336] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:21.749944  5840 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:21.749704 14005 consensus_queue.cc:237] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:21.767867 13900 leader_election.cc:290] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:21.768877 13955 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:21.770241  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e8ec653e484ba58498cc78f412de75" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:21.771695  6019 raft_consensus.cc:3060] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:21.788271  5915 consensus_peers.cc:597] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:21.793766  5728 debug-util.cc:398] Leaking SignalData structure 0x7b080015bcc0 after lost signal to thread 2070
W20260709 12:28:21.795555  5728 debug-util.cc:398] Leaking SignalData structure 0x7b08000526a0 after lost signal to thread 5731
W20260709 12:28:21.797708  5728 debug-util.cc:398] Leaking SignalData structure 0x7b08000c8640 after lost signal to thread 5899
W20260709 12:28:21.798013  5728 debug-util.cc:398] Leaking SignalData structure 0x7b08001f6340 after lost signal to thread 5900
W20260709 12:28:21.800136  5728 debug-util.cc:398] Leaking SignalData structure 0x7b08002b9480 after lost signal to thread 5973
W20260709 12:28:21.800442  5728 debug-util.cc:398] Leaking SignalData structure 0x7b080004a060 after lost signal to thread 5974
W20260709 12:28:21.802346  5728 debug-util.cc:398] Leaking SignalData structure 0x7b0800140420 after lost signal to thread 6048
W20260709 12:28:21.802629  5728 debug-util.cc:398] Leaking SignalData structure 0x7b080020d560 after lost signal to thread 6049
W20260709 12:28:21.803241  5728 debug-util.cc:398] Leaking SignalData structure 0x7b0800196e60 after lost signal to thread 6829
I20260709 12:28:21.805960  5691 catalog_manager.cc:5697] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 13 to 20, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 20 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:28:21.807812  5691 catalog_manager.cc:5697] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 19 to 22, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 22 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
I20260709 12:28:21.809940 13799 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:21.812127  6019 raft_consensus.cc:2468] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 26.
W20260709 12:28:21.818050  5990 leader_election.cc:336] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:21.826929 14005 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:21.827684 14005 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:21.834046 14005 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:21.834647  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:21.852176 13900 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:21.853019 13900 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:21.855366 13900 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:21.856822  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:21.857569  5869 raft_consensus.cc:3060] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:21.809304  5691 catalog_manager.cc:5697] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 14 to 24. New cstate: current_term: 24 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
I20260709 12:28:21.860275  5691 catalog_manager.cc:5697] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 20 to 22, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 22 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:21.852176 13799 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:21.872566  5990 leader_election.cc:336] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:21.873073  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:21.874621 13794 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:21.876500  5915 leader_election.cc:304] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:21.878425  5869 raft_consensus.cc:2468] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
I20260709 12:28:21.881450  5987 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:21.884375 13900 raft_consensus.cc:2804] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:21.885126 13900 raft_consensus.cc:697] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:21.885823  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:21.886678  6019 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 28.
I20260709 12:28:21.886344 13900 consensus_queue.cc:237] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:21.878939  5691 catalog_manager.cc:5697] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 20 to 29, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 29 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:21.889668  5691 catalog_manager.cc:5697] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 15 to 25, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 25 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:21.891047 13955 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.891337  5691 catalog_manager.cc:5697] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 15 to 18, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 18 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:21.892042 13799 raft_consensus.cc:2804] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:21.891985 13955 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:21.892828 13799 raft_consensus.cc:697] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:21.897006 13955 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:21.898059 13799 consensus_queue.cc:237] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:21.902418  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:21.903298  6019 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 21.
W20260709 12:28:21.907721  5915 leader_election.cc:336] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:21.914594  5840 consensus_peers.cc:597] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:21.915298  5840 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:21.930289  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:21.893294  5691 catalog_manager.cc:5697] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 19 to 24, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 24 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:21.929563  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:21.962848 13417 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:21.963613 13417 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:21.978229 13921 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:21.984581 13921 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.985100 13921 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:21.954614 13500 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:21.991225 13500 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:21.998924 13417 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:22.001259  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:21.991880 13500 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:22.006098 13987 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:22.006917 13987 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:21.966842 13903 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
W20260709 12:28:21.974411  5915 leader_election.cc:336] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:22.000371  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 51537.8us
W20260709 12:28:22.019016  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 44683.2us
I20260709 12:28:22.033756 13992 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:22.034657 13992 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:22.034986  6019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:22.035666  6019 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
I20260709 12:28:22.036962 13992 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:22.038640  5915 consensus_peers.cc:597] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:22.041363  5840 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.049064  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:22.049854  6016 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:28:22.050467 14005 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:22.051502 14005 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:22.056974  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:22.058429 13986 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:22.058899 13986 raft_consensus.cc:493] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:22.059283 13986 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:22.062397  5915 leader_election.cc:336] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.009590 13987 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:22.063190  5915 consensus_peers.cc:597] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:22.010592 13903 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:22.089035  5915 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:22.098304 13903 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:22.101734  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 80498.5us
W20260709 12:28:22.105652  5915 consensus_peers.cc:597] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:22.101465 13986 raft_consensus.cc:515] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:22.109768 13986 leader_election.cc:290] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:22.110131  5990 leader_election.cc:336] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.110832  5990 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:22.112720  5990 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111) [suppressed 286 similar messages]
I20260709 12:28:22.114068  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:22.115450 13987 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:22.115989 13987 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:22.116436 13987 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:22.118094  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:22.118945  5867 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:28:22.119009 13794 raft_consensus.cc:2749] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:22.120843  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba269f0998844ab5b461587eaa90df32" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:22.121618  6015 raft_consensus.cc:3060] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:22.132498  6015 raft_consensus.cc:2468] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:28:22.133965  5913 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:22.133162 13500 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:22.135375 14005 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:22.138217 13168 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:22.138635 13168 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:22.138995 13168 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:22.139940  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
W20260709 12:28:22.146137  5990 leader_election.cc:336] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.146687  5990 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:22.137656 13500 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:22.149508  5915 leader_election.cc:336] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.154461  6015 raft_consensus.cc:1275] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:28:22.150533 13168 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:22.159761 13168 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:22.166817  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:22.167834  5869 raft_consensus.cc:3060] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:22.150691 14005 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:22.125362 13921 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:22.174935 13921 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:22.183470 13168 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.184301 13168 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:22.186529 13168 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:22.192550  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685933c4b2364e1c90d92edb4c091799" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:22.193409  6015 raft_consensus.cc:3060] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:22.144657 13987 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:22.196038 13987 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:22.199359  5869 raft_consensus.cc:2468] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 25.
I20260709 12:28:22.208913  5913 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:22.184162  5915 leader_election.cc:304] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
W20260709 12:28:22.209971  5840 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.210456 13903 raft_consensus.cc:2804] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:22.211012 13903 raft_consensus.cc:697] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:22.212137  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa54a35cf4c54bb1a1c28658c11d174c" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:22.212105 13903 consensus_queue.cc:237] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:22.212868  6016 raft_consensus.cc:3060] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:22.214164 13168 raft_consensus.cc:2804] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:22.214695 13168 raft_consensus.cc:697] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:22.215577 13168 consensus_queue.cc:237] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:22.217835 13900 consensus_queue.cc:1048] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:28:22.218160  5915 leader_election.cc:336] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.222115  6016 raft_consensus.cc:2468] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 29.
W20260709 12:28:22.206177  5990 consensus_peers.cc:597] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:22.231432  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:22.233265  6022 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 22.
I20260709 12:28:22.234629  6023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428a50885681465a98089cb853b233b9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:22.235378  6023 raft_consensus.cc:3060] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:22.231860  5840 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.267658  6015 raft_consensus.cc:2468] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 22.
W20260709 12:28:22.273674  5915 leader_election.cc:336] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:22.286820  5915 consensus_peers.cc:597] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:22.287580  5915 leader_election.cc:336] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.297520  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:22.298681 13168 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:22.299115 13168 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:22.299438 13168 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:22.283090  5840 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:22.277220  6023 raft_consensus.cc:2468] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 25.
I20260709 12:28:22.304046 13987 raft_consensus.cc:2804] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:22.304704 13987 raft_consensus.cc:697] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 22 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:22.305784 13987 consensus_queue.cc:237] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:22.310688 13168 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:22.319886  5840 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:22.312150  5942 raft_consensus.cc:1275] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:28:22.318943  5865 raft_consensus.cc:1275] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:28:22.312058  5915 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:22.322940 13986 raft_consensus.cc:2804] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:22.323501 13986 raft_consensus.cc:697] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 29 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:22.323839 13921 raft_consensus.cc:2804] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:22.324838 13921 raft_consensus.cc:697] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:22.324527 13986 consensus_queue.cc:237] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:22.326203 13921 consensus_queue.cc:237] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:22.327508 14005 consensus_queue.cc:1048] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:22.328393 13794 consensus_queue.cc:1048] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:22.334553 13168 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:22.338291  6023 raft_consensus.cc:1275] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
W20260709 12:28:22.338491  5915 leader_election.cc:336] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.338703  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa58bdc1b834feda10d16a00cd81355" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:22.339648  6015 raft_consensus.cc:3060] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:22.342288  5915 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:22.342909  5915 leader_election.cc:336] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.374056  6015 raft_consensus.cc:2468] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:28:22.375802  5915 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:22.387554 13417 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:22.400346 13417 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:22.390519 13986 raft_consensus.cc:2804] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:22.378331 13992 consensus_queue.cc:1048] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:28:22.428508 13417 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:22.439265  5990 leader_election.cc:336] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:22.439951  5990 consensus_peers.cc:597] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:22.414822 13903 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:22.422336 13986 raft_consensus.cc:697] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
W20260709 12:28:22.459326  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 75725.4us
I20260709 12:28:22.450454 13955 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:22.454360 14017 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:22.462782 13903 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:22.463783 14005 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.479839 14017 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:22.486263 13955 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:22.487516  2067 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 6825)
I20260709 12:28:22.487335 14005 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:22.489085 14017 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:22.471148 13989 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:22.482030 14018 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.482528 13986 consensus_queue.cc:237] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:22.494875 13955 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:22.496479  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:22.497418  5940 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 18.
I20260709 12:28:22.504840 14018 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:22.506211 13989 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:22.507332 14018 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:22.508759 13989 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:22.509619  5869 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:22.510512  5869 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 25.
W20260709 12:28:22.512590  5990 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.519363  5838 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:22.526253 14005 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:22.529815 14017 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:22.530462 14017 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:22.530959 14017 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:22.529250 13903 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:22.552667  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:22.553432  6016 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 25.
I20260709 12:28:22.555308  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:22.555344  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:22.556239  6016 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 27.
I20260709 12:28:22.556252  6015 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 23.
I20260709 12:28:22.564494  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:22.568264  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:22.570957 13794 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:28:22.568815  5915 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:22.571539 13794 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:22.572450 13794 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:22.573833 14038 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:22.576046 14038 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:22.576490 14038 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:22.574434  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:22.587852  5865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:22.588893 14036 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:22.589457 14036 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 1fc57407d56241b38a94d543e073f8af)
I20260709 12:28:22.589864 14036 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:22.587102  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:22.598582 14018 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:22.599117 14018 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:22.599491 14018 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:22.612852  5869 raft_consensus.cc:1275] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:28:22.585319 13794 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:22.625916 13986 consensus_queue.cc:1048] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:28:22.628229  5915 consensus_peers.cc:597] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:22.628954  5915 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:22.629379  5915 leader_election.cc:336] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.647578 13794 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:22.612813 14038 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:22.658156 14038 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:22.657902 14036 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:22.660364 14036 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 28 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:22.670230  5990 leader_election.cc:336] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.672008 14017 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:22.673686  5865 raft_consensus.cc:2393] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 19: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
I20260709 12:28:22.674455 14018 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:22.677135  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:22.678210  5867 raft_consensus.cc:2393] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 24: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:28:22.681232  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 51342.9us
I20260709 12:28:22.694386 14018 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 election: Requested vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:22.701809  5867 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
W20260709 12:28:22.702003  5840 leader_election.cc:336] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.702745  5867 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 26: Already voted for candidate 7a0204b931394fc48ffc503716e0d111 in this term.
W20260709 12:28:22.703063  5840 leader_election.cc:336] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:22.704039  5840 consensus_peers.cc:597] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:22.711158  5942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a324a652924631995e78ee3fda8422" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:22.712378  5942 raft_consensus.cc:3060] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:22.714941 14017 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:22.728250  5913 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:22.747429  5691 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40818 (request call id 153) took 3434 ms. Trace:
I20260709 12:28:22.748001  5691 rpcz_store.cc:276] 0709 12:28:19.313482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:20.167971 (+854489us) service_pool.cc:224] Handling call
0709 12:28:22.747309 (+2579338us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:22.132308 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:22.133932 (+  1624us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:22.135743 (+  1811us) write_op.cc:435] Acquired schema lock
0709 12:28:22.135765 (+    22us) tablet.cc:662] Decoding operations
0709 12:28:22.138797 (+  3032us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:22.138918 (+   121us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:22.138945 (+    27us) tablet.cc:685] Acquiring locks for 37 operations
0709 12:28:22.141575 (+  2630us) tablet.cc:701] Row locks acquired
0709 12:28:22.146339 (+  4764us) write_op.cc:260] PREPARE: finished
0709 12:28:22.196706 (+ 50367us) write_op.cc:270] Start()
0709 12:28:22.196923 (+   217us) write_op.cc:276] Timestamp: P: 1783600102196611 usec, L: 0
0709 12:28:22.196959 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 12:28:22.197995 (+  1036us) log.cc:844] Serialized 14283 byte log entry
0709 12:28:22.265702 (+ 67707us) op_driver.cc:464] REPLICATION: finished
0709 12:28:22.284495 (+ 18793us) write_op.cc:301] APPLY: starting
0709 12:28:22.284603 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:22.307598 (+ 22995us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:22.307738 (+   140us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:22.535962 (+228224us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:22.536560 (+   598us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=74,key_file_lookups=74,delta_file_lookups=0,mrs_lookups=0
0709 12:28:22.536636 (+    76us) write_op.cc:312] APPLY: finished
0709 12:28:22.537537 (+   901us) log.cc:844] Serialized 315 byte log entry
0709 12:28:22.559726 (+ 22189us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:22.593951 (+ 34225us) write_op.cc:454] Released schema lock
0709 12:28:22.594097 (+   146us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:22.618048 (+ 23951us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:22.618094 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12220,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":18478,"apply.run_cpu_time_us":49753,"apply.run_wall_time_us":333493,"cfile_cache_hit":150,"cfile_cache_hit_bytes":158935,"num_ops":37,"prepare.queue_time_us":35319,"prepare.run_cpu_time_us":10012,"prepare.run_wall_time_us":77646,"raft.queue_time_us":39629,"raft.run_cpu_time_us":4155,"raft.run_wall_time_us":21753,"replication_time_us":68532,"thread_start_us":86486,"threads_started":4,"wal-append.queue_time_us":10116}]]}
W20260709 12:28:22.704725  5840 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.733408  5942 raft_consensus.cc:2468] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 19.
I20260709 12:28:22.759771 13500 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:22.760317  5838 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 
I20260709 12:28:22.761372 14018 raft_consensus.cc:2804] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:28:22.756953  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 52251.1us
I20260709 12:28:22.762126 14018 raft_consensus.cc:697] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:22.763371 14018 consensus_queue.cc:237] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:22.762110  5840 leader_election.cc:336] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.769834  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:22.770740  6016 raft_consensus.cc:2393] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a0204b931394fc48ffc503716e0d111 in current term 26: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:28:22.770737  5915 consensus_peers.cc:597] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:22.773177  5915 consensus_peers.cc:597] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:22.777240  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bede7b5a3c84fe28a073542219f3fe4" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:22.778155  6016 raft_consensus.cc:3060] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:22.778889  5697 catalog_manager.cc:5697] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 15 to 16, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 16 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:22.781697  5697 catalog_manager.cc:5697] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 21 to 22, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 22 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:28:22.787204  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:22.793532 13799 raft_consensus.cc:3060] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:22.793818  5697 catalog_manager.cc:5697] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:28:22.793041  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c0a9da01454a2794160d9c0ff08ea9" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:22.791838  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:22.809309 14005 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:22.797022  5697 catalog_manager.cc:5697] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 17 to 21, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 1fc57407d56241b38a94d543e073f8af (127.2.4.193). New cstate: current_term: 21 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:28:22.838976  6016 raft_consensus.cc:2468] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 28.
W20260709 12:28:22.839905  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46899.6us
I20260709 12:28:22.841594  6023 raft_consensus.cc:1275] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:22.844605 13799 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:22.855093  6023 raft_consensus.cc:1275] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:28:22.856199  6015 raft_consensus.cc:2468] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a0204b931394fc48ffc503716e0d111 in term 24.
I20260709 12:28:22.857298  5840 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 
I20260709 12:28:22.859040 13989 raft_consensus.cc:2804] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:22.859560 13989 raft_consensus.cc:697] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 28 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:22.861846 13500 consensus_queue.cc:1048] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:22.860574 13989 consensus_queue.cc:237] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:22.877842  5840 leader_election.cc:336] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:22.878484  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:22.879334 14017 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:28:22.890177  5840 leader_election.cc:336] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:22.903671  5840 leader_election.cc:336] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:22.904883  5840 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:22.905491  5840 consensus_peers.cc:597] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:22.946800  5840 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:22.947731 14017 consensus_queue.cc:1048] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:22.947443  6023 raft_consensus.cc:1275] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 29 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
W20260709 12:28:22.950975  5915 consensus_peers.cc:597] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:22.951602 14036 raft_consensus.cc:2804] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:22.952230 14036 raft_consensus.cc:697] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 24 LEADER]: Becoming Leader. State: Replica: 7a0204b931394fc48ffc503716e0d111, State: Running, Role: LEADER
I20260709 12:28:22.953230 14036 consensus_queue.cc:237] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:22.962131 13986 consensus_queue.cc:1048] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:22.981037  6023 raft_consensus.cc:1275] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:28:22.996629  6016 raft_consensus.cc:1275] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:23.008845 14017 consensus_queue.cc:1048] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:23.002650  5691 catalog_manager.cc:5697] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 26 to 28, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 28 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
W20260709 12:28:23.019896  5697 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40830 (request call id 153) took 37615 ms (client timeout 15000 ms). Trace:
W20260709 12:28:23.021863  5990 consensus_peers.cc:597] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:23.022706 13986 consensus_queue.cc:1048] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:23.022219  5691 catalog_manager.cc:5697] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 14 to 23, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 23 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:28:23.029054  5691 catalog_manager.cc:5697] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 28 to 29, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 29 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:23.030903  5691 catalog_manager.cc:5697] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 25 to 26, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 26 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:23.033233  5691 catalog_manager.cc:5697] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 21 to 23, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 23 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:23.043152  5691 catalog_manager.cc:5697] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:23.046612  5691 catalog_manager.cc:5697] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:23.020398  5697 rpcz_store.cc:269] 0709 12:27:45.420507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.429923 (+  9416us) service_pool.cc:224] Handling call
0709 12:28:23.019784 (+37589861us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:22.895826 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:22.896395 (+   569us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:22.896427 (+    32us) write_op.cc:435] Acquired schema lock
0709 12:28:22.896444 (+    17us) tablet.cc:662] Decoding operations
0709 12:28:22.897293 (+   849us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:22.897370 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:22.897397 (+    27us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:28:22.907854 (+ 10457us) tablet.cc:701] Row locks acquired
0709 12:28:22.918097 (+ 10243us) write_op.cc:260] PREPARE: finished
0709 12:28:22.918278 (+   181us) write_op.cc:270] Start()
0709 12:28:22.918414 (+   136us) write_op.cc:276] Timestamp: P: 1783600102918210 usec, L: 0
0709 12:28:22.918436 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:28:22.919212 (+   776us) log.cc:844] Serialized 1681 byte log entry
0709 12:28:22.957866 (+ 38654us) op_driver.cc:464] REPLICATION: finished
0709 12:28:22.959880 (+  2014us) write_op.cc:301] APPLY: starting
0709 12:28:22.959970 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:22.962422 (+  2452us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:22.962519 (+    97us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:22.965254 (+  2735us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:22.965433 (+   179us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 12:28:22.965482 (+    49us) write_op.cc:312] APPLY: finished
0709 12:28:22.968862 (+  3380us) log.cc:844] Serialized 48 byte log entry
0709 12:28:22.969994 (+  1132us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:22.970105 (+   111us) write_op.cc:454] Released schema lock
0709 12:28:22.970163 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:22.970567 (+   404us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:22.970596 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":148,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":1100,"apply.run_cpu_time_us":6695,"apply.run_wall_time_us":11259,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16736,"num_ops":4,"prepare.queue_time_us":14905,"prepare.run_cpu_time_us":3606,"prepare.run_wall_time_us":35755,"replication_time_us":39201,"thread_start_us":17111,"threads_started":1}]]}
I20260709 12:28:23.052791  5691 catalog_manager.cc:5697] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 24 to 25, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 25 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:23.056730  5691 catalog_manager.cc:5697] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 21 to 24, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 24 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:23.058607  5691 catalog_manager.cc:5697] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 23 to 24, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 24 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:23.065300  5691 catalog_manager.cc:5697] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 22 to 23, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 23 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:23.072291  5915 consensus_peers.cc:597] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:23.072391  5840 consensus_peers.cc:597] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:23.066890  5691 catalog_manager.cc:5697] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 22 to 27, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 27 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:28:23.078327  5691 catalog_manager.cc:5697] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 18 to 22, leader changed from 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 22 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:28:23.099512  5691 catalog_manager.cc:5697] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 25 to 26, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 26 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:23.101274  5691 catalog_manager.cc:5697] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 22 to 23, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 23 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } }
W20260709 12:28:23.249910  5990 consensus_peers.cc:597] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:23.272796  6016 raft_consensus.cc:1275] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:28:23.274766 14036 consensus_queue.cc:1048] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:23.285703  5941 raft_consensus.cc:1275] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:28:23.289891  5915 consensus_peers.cc:597] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:23.291916 13955 consensus_queue.cc:1048] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:28:23.316887  5915 consensus_peers.cc:597] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:23.362721  5990 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:23.363560  5990 consensus_peers.cc:597] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:23.374104  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 43613.1us
W20260709 12:28:23.375155  5840 consensus_peers.cc:597] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:23.414542  5691 catalog_manager.cc:5697] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 18 to 19, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 19 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } }
W20260709 12:28:23.432862  5692 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40812 (request call id 145) took 59587 ms (client timeout 15000 ms). Trace:
W20260709 12:28:23.433405  5692 rpcz_store.cc:269] 0709 12:27:23.856394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.863322 (+  6928us) service_pool.cc:224] Handling call
0709 12:28:23.431739 (+59568417us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3880,"reported_tablets":44}
W20260709 12:28:23.440438  5990 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:23.453866  5840 consensus_peers.cc:597] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:23.454910  5840 consensus_peers.cc:597] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:23.488699  2067 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 6825)
W20260709 12:28:23.495911  5840 consensus_peers.cc:597] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:23.513737  5990 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:23.531033  6015 raft_consensus.cc:1275] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Refusing update from remote peer 7a0204b931394fc48ffc503716e0d111: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:28:23.570041 14036 consensus_queue.cc:1048] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:28:23.591116  5840 consensus_peers.cc:597] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:23.610469  5694 catalog_manager.cc:5697] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:23.612634  5694 catalog_manager.cc:5697] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
W20260709 12:28:23.615065  5840 consensus_peers.cc:597] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:23.633913  5693 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40830 (request call id 150) took 114525 ms (client timeout 15000 ms). Trace:
W20260709 12:28:23.634483  5693 rpcz_store.cc:269] 0709 12:26:29.108111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.108310 (+   199us) service_pool.cc:224] Handling call
0709 12:28:23.632494 (+114524184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29623,"reported_tablets":44}
I20260709 12:28:23.630362  5694 catalog_manager.cc:5697] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 24 to 27, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 27 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:23.632642  5840 consensus_peers.cc:597] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:23.639212  5694 catalog_manager.cc:5697] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 16 to 19, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 19 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:23.652659  5694 catalog_manager.cc:5697] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 16 to 21, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 21 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:23.654665  5694 catalog_manager.cc:5697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:28:23.656407  5694 catalog_manager.cc:5697] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 17 to 20, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 20 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:28:23.658303  5694 catalog_manager.cc:5697] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } }
I20260709 12:28:23.663053  5694 catalog_manager.cc:5697] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 21 to 26, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 26 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:23.780030  5694 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40830 (request call id 154) took 11909 ms (client timeout 15000 ms). Trace:
W20260709 12:28:23.785904  5694 rpcz_store.cc:269] 0709 12:28:11.871149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.871485 (+   336us) service_pool.cc:224] Handling call
0709 12:28:23.779949 (+11908464us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:23.676672 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:23.677410 (+   738us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:23.677455 (+    45us) write_op.cc:435] Acquired schema lock
0709 12:28:23.677477 (+    22us) tablet.cc:662] Decoding operations
0709 12:28:23.678575 (+  1098us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:23.678647 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:23.678680 (+    33us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:28:23.679560 (+   880us) tablet.cc:701] Row locks acquired
0709 12:28:23.679583 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:28:23.679766 (+   183us) write_op.cc:270] Start()
0709 12:28:23.679911 (+   145us) write_op.cc:276] Timestamp: P: 1783600103679711 usec, L: 0
0709 12:28:23.679948 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 12:28:23.680753 (+   805us) log.cc:844] Serialized 3587 byte log entry
0709 12:28:23.683948 (+  3195us) op_driver.cc:464] REPLICATION: finished
0709 12:28:23.684463 (+   515us) write_op.cc:301] APPLY: starting
0709 12:28:23.684559 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:23.690712 (+  6153us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:23.690850 (+   138us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:23.718828 (+ 27978us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:23.719114 (+   286us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 12:28:23.719172 (+    58us) write_op.cc:312] APPLY: finished
0709 12:28:23.719531 (+   359us) log.cc:844] Serialized 88 byte log entry
0709 12:28:23.720664 (+  1133us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:23.720844 (+   180us) write_op.cc:454] Released schema lock
0709 12:28:23.720927 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:23.749878 (+ 28951us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:23.749919 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":49387,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":287,"apply.run_cpu_time_us":14181,"apply.run_wall_time_us":45860,"cfile_cache_hit":36,"cfile_cache_hit_bytes":37656,"num_ops":9,"prepare.queue_time_us":174,"prepare.run_cpu_time_us":4504,"prepare.run_wall_time_us":21526,"replication_time_us":3852}]]}
I20260709 12:28:23.782357  5688 catalog_manager.cc:5697] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "1fc57407d56241b38a94d543e073f8af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
W20260709 12:28:23.822921  5990 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:23.889978  5688 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40812 (request call id 146) took 35498 ms (client timeout 15000 ms). Trace:
W20260709 12:28:23.907579  5688 rpcz_store.cc:269] 0709 12:27:48.391591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.399170 (+  7579us) service_pool.cc:224] Handling call
0709 12:28:23.889881 (+35490711us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:23.824240 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:23.824872 (+   632us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:23.824910 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:28:23.824929 (+    19us) tablet.cc:662] Decoding operations
0709 12:28:23.825314 (+   385us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:23.825374 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:23.825405 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:28:23.825606 (+   201us) tablet.cc:701] Row locks acquired
0709 12:28:23.825646 (+    40us) write_op.cc:260] PREPARE: finished
0709 12:28:23.825798 (+   152us) write_op.cc:270] Start()
0709 12:28:23.825911 (+   113us) write_op.cc:276] Timestamp: P: 1783600103825748 usec, L: 0
0709 12:28:23.825932 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:28:23.826641 (+   709us) log.cc:844] Serialized 534 byte log entry
0709 12:28:23.828840 (+  2199us) op_driver.cc:464] REPLICATION: finished
0709 12:28:23.829292 (+   452us) write_op.cc:301] APPLY: starting
0709 12:28:23.829380 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:23.830256 (+   876us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:23.830355 (+    99us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:23.831090 (+   735us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:23.831256 (+   166us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:28:23.831302 (+    46us) write_op.cc:312] APPLY: finished
0709 12:28:23.831505 (+   203us) log.cc:844] Serialized 24 byte log entry
0709 12:28:23.831681 (+   176us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:23.831763 (+    82us) write_op.cc:454] Released schema lock
0709 12:28:23.831807 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:23.832145 (+   338us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:23.832171 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":612657,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":243,"apply.run_cpu_time_us":2833,"apply.run_wall_time_us":2833,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4184,"num_ops":1,"prepare.queue_time_us":134,"prepare.run_cpu_time_us":2694,"prepare.run_wall_time_us":2694,"replication_time_us":2791}]]}
W20260709 12:28:23.942763  5915 consensus_peers.cc:597] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:23.985179  5840 consensus_peers.cc:597] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:23.985841  5840 consensus_peers.cc:597] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:23.990417  5689 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40818 (request call id 151) took 52623 ms (client timeout 15000 ms). Trace:
W20260709 12:28:24.017740  5689 rpcz_store.cc:269] 0709 12:27:31.367233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.404238 (+ 37005us) service_pool.cc:224] Handling call
0709 12:28:23.990276 (+52586038us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1806438,"reported_tablets":44}
I20260709 12:28:24.100899  5696 catalog_manager.cc:5697] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 reported cstate change: term changed from 22 to 24, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 24 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:24.116415  5840 consensus_peers.cc:597] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:24.150290  5915 consensus_peers.cc:597] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:24.154356  5990 consensus_peers.cc:597] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:24.181483  5695 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40830 (request call id 149) took 147730 ms (client timeout 15000 ms). Trace:
W20260709 12:28:24.183511  5695 rpcz_store.cc:269] 0709 12:25:56.447073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.447329 (+   256us) service_pool.cc:224] Handling call
0709 12:28:24.177068 (+147729739us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31094,"reported_tablets":43}
W20260709 12:28:24.188488  5840 consensus_peers.cc:597] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:24.257296  5915 consensus_peers.cc:597] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:24.258257  5688 catalog_manager.cc:5697] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 24 to 28. New cstate: current_term: 28 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:24.261642  5688 catalog_manager.cc:5697] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:24.286511  5840 consensus_peers.cc:597] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:24.297009  5688 catalog_manager.cc:5697] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 22 to 28. New cstate: current_term: 28 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
W20260709 12:28:24.318696  5915 consensus_peers.cc:597] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:24.325778  5696 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40812 (request call id 147) took 9680 ms. Trace:
I20260709 12:28:24.315241  5688 catalog_manager.cc:5697] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 21 to 24, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 24 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:28:24.334761  5688 catalog_manager.cc:5697] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 21 to 23, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 23 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:24.331290  5696 rpcz_store.cc:276] 0709 12:28:14.645734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.934982 (+3289248us) service_pool.cc:224] Handling call
0709 12:28:24.325544 (+6390562us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:24.180600 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:24.181220 (+   620us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:24.181257 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:28:24.181275 (+    18us) tablet.cc:662] Decoding operations
0709 12:28:24.181665 (+   390us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:24.181725 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:24.181750 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:28:24.181948 (+   198us) tablet.cc:701] Row locks acquired
0709 12:28:24.181959 (+    11us) write_op.cc:260] PREPARE: finished
0709 12:28:24.182114 (+   155us) write_op.cc:270] Start()
0709 12:28:24.182232 (+   118us) write_op.cc:276] Timestamp: P: 1783600104182063 usec, L: 0
0709 12:28:24.182252 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:28:24.182934 (+   682us) log.cc:844] Serialized 534 byte log entry
0709 12:28:24.193273 (+ 10339us) op_driver.cc:464] REPLICATION: finished
0709 12:28:24.193769 (+   496us) write_op.cc:301] APPLY: starting
0709 12:28:24.193866 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:24.194810 (+   944us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:24.194930 (+   120us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:24.195881 (+   951us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:24.196047 (+   166us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:28:24.196096 (+    49us) write_op.cc:312] APPLY: finished
0709 12:28:24.196315 (+   219us) log.cc:844] Serialized 24 byte log entry
0709 12:28:24.196511 (+   196us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:24.196616 (+   105us) write_op.cc:454] Released schema lock
0709 12:28:24.196658 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:24.197037 (+   379us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:24.197064 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":39204,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":285,"apply.run_cpu_time_us":3244,"apply.run_wall_time_us":3244,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4184,"num_ops":1,"prepare.queue_time_us":169,"prepare.run_cpu_time_us":2680,"prepare.run_wall_time_us":2682,"replication_time_us":10848}]]}
I20260709 12:28:24.339299  5688 catalog_manager.cc:5697] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 19 to 22, leader changed from 6add331326ca47038308a447bf798fa7 (127.2.4.196) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 22 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:28:24.341133  5688 catalog_manager.cc:5697] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 22 to 25, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 7a0204b931394fc48ffc503716e0d111 (127.2.4.194). New cstate: current_term: 25 leader_uuid: "7a0204b931394fc48ffc503716e0d111" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } }
I20260709 12:28:24.349332  5688 catalog_manager.cc:5697] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 21 to 26, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 26 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:24.489459  2067 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 6825)
W20260709 12:28:24.541222  5915 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:24.573441  5915 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:24.643903  5915 consensus_peers.cc:597] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:24.692013  5915 consensus_peers.cc:597] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:24.733911  5990 consensus_peers.cc:597] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:24.745293  5915 consensus_peers.cc:597] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:24.770915  5915 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:24.777208 14057 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.778061 14057 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:24.780653 14057 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:24.793259  5990 leader_election.cc:336] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:24.811041  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111" is_pre_election: true
I20260709 12:28:24.812055  5866 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 26.
I20260709 12:28:24.821949  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:24.823688 14057 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:24.824152 14057 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.824524 14057 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:24.859534  5990 consensus_peers.cc:597] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:24.870306  5915 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:24.898007 14057 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:24.937094 14057 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 election: Requested vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:24.947171  5840 consensus_peers.cc:597] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:24.947553  5866 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "7a0204b931394fc48ffc503716e0d111"
I20260709 12:28:24.950285  5866 raft_consensus.cc:3060] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:24.929484  5915 consensus_peers.cc:597] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:24.968375  5866 raft_consensus.cc:2468] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 27.
W20260709 12:28:24.978520  5990 leader_election.cc:336] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:24.981976  5987 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af
I20260709 12:28:24.984678 14059 raft_consensus.cc:2804] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:24.985363 14059 raft_consensus.cc:697] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 27 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:24.986464 14059 consensus_queue.cc:237] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:24.999656  5915 consensus_peers.cc:597] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:25.051916  5688 catalog_manager.cc:5697] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 25 to 27, leader changed from 1fc57407d56241b38a94d543e073f8af (127.2.4.193) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 27 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:25.112326  5915 consensus_peers.cc:597] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:25.199425  5915 consensus_peers.cc:597] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:25.257496  5840 consensus_peers.cc:597] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:25.318025  5915 consensus_peers.cc:597] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:25.348807  5990 consensus_peers.cc:597] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:25.416873  5869 raft_consensus.cc:1275] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
W20260709 12:28:25.421939  5990 consensus_peers.cc:597] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:25.431365 14060 consensus_queue.cc:1048] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:28:25.490195  2067 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 6825)
W20260709 12:28:25.602012  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 88018.4us
W20260709 12:28:25.623229  5990 consensus_peers.cc:597] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.656978  5990 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:25.709980  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 81104.9us
W20260709 12:28:25.777885  5990 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:25.791308  5840 consensus_peers.cc:597] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:25.805801  5840 consensus_peers.cc:597] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:25.811262  5990 consensus_peers.cc:597] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.845301  5990 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:25.870378  5840 consensus_peers.cc:597] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:25.938479  5915 consensus_peers.cc:597] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:25.950093  5915 consensus_peers.cc:597] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:26.061151  5915 consensus_peers.cc:597] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:26.088671  5840 consensus_peers.cc:597] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:26.203356  5840 consensus_peers.cc:597] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:26.219153  5840 consensus_peers.cc:597] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:26.226408  5840 consensus_peers.cc:597] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:26.313369  5840 consensus_peers.cc:597] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:26.351267  5915 consensus_peers.cc:597] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:26.401194  5990 consensus_peers.cc:597] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:26.404803  5840 consensus_peers.cc:597] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:26.406405  5990 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:26.491230  2067 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 6825)
W20260709 12:28:26.615895  5840 consensus_peers.cc:597] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:26.711545  5840 consensus_peers.cc:597] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:26.822872  5915 consensus_peers.cc:597] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:26.870982  5915 consensus_peers.cc:597] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:26.875325  5840 consensus_peers.cc:597] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:26.911486  5915 consensus_peers.cc:597] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:26.958642  5840 consensus_peers.cc:597] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:27.089090  5915 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:27.090266  5915 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:27.132860  5840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111) [suppressed 450 similar messages]
W20260709 12:28:27.139146  5915 consensus_peers.cc:597] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:27.158756  5840 consensus_peers.cc:597] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:27.171063  5915 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:27.199177  5990 consensus_peers.cc:597] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:27.307605  5915 consensus_peers.cc:597] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:27.335709  5915 consensus_peers.cc:597] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:27.382679  5915 consensus_peers.cc:597] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:27.427292  5915 consensus_peers.cc:597] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:27.428488  5990 consensus_peers.cc:597] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:27.491858  2067 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 6825)
W20260709 12:28:27.507472  5915 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:27.621028  5915 consensus_peers.cc:597] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:27.715247  5915 consensus_peers.cc:597] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:27.720117  5990 consensus_peers.cc:597] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:27.881795  5840 consensus_peers.cc:597] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:27.947431  5990 consensus_peers.cc:597] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:27.983021  5915 consensus_peers.cc:597] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:28.024572  5840 consensus_peers.cc:597] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:28.044361  5990 consensus_peers.cc:597] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:28.110000  5990 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:28.173847  5990 consensus_peers.cc:597] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:28.217008  5990 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:28.298678  5840 consensus_peers.cc:597] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:28.303789  5915 consensus_peers.cc:597] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:28.344092  5990 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:28.435974  5840 consensus_peers.cc:597] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:28.462824  5840 consensus_peers.cc:597] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:28.492933  2067 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 6825)
W20260709 12:28:28.699203  5915 consensus_peers.cc:597] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:28.762907  5840 consensus_peers.cc:597] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:28.766165  5915 consensus_peers.cc:597] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:28.830955  5840 consensus_peers.cc:597] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:28.930186  5915 consensus_peers.cc:597] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:28.937863  5915 consensus_peers.cc:597] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:28.970767  5840 consensus_peers.cc:597] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:28.982486  5990 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:29.009788  5990 consensus_peers.cc:597] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:29.012203  5840 consensus_peers.cc:597] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:29.104687  5840 consensus_peers.cc:597] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:29.190994  5840 consensus_peers.cc:597] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:29.344643  5840 consensus_peers.cc:597] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:29.346347  5840 consensus_peers.cc:597] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:29.367430  5915 consensus_peers.cc:597] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:29.368060  5915 consensus_peers.cc:597] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:29.410323  5840 consensus_peers.cc:597] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:29.437978  5915 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:29.494168  2067 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 6825)
W20260709 12:28:29.497340  5915 consensus_peers.cc:597] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:29.723779  5990 consensus_peers.cc:597] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:29.736656  5915 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:29.756747  5915 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:29.777587  5915 consensus_peers.cc:597] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:29.836315  5840 consensus_peers.cc:597] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:29.839903  5915 consensus_peers.cc:597] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:29.840782  5915 consensus_peers.cc:597] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:29.988888  5915 consensus_peers.cc:597] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:30.033607  5990 consensus_peers.cc:597] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:30.079975  5915 consensus_peers.cc:597] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:30.112650  5840 consensus_peers.cc:597] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:30.120947  5990 consensus_peers.cc:597] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:30.147761  5915 consensus_peers.cc:597] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:30.294399  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 65547.4us
W20260709 12:28:30.297073  5915 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:30.297757  5915 consensus_peers.cc:597] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:30.497920  2067 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 6825)
W20260709 12:28:30.504763  5990 consensus_peers.cc:597] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:30.518198  5990 consensus_peers.cc:597] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:30.693151  5990 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:30.694064  5990 consensus_peers.cc:597] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:30.695363  5840 consensus_peers.cc:597] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:30.713203  5990 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:30.778692  5915 consensus_peers.cc:597] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:30.845494  5840 consensus_peers.cc:597] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:30.872648  5840 consensus_peers.cc:597] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:30.973096  5990 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:31.053287  5840 consensus_peers.cc:597] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:31.161517  5840 consensus_peers.cc:597] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:31.198948  5990 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:31.256804  5915 consensus_peers.cc:597] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:31.327286  5915 consensus_peers.cc:597] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:31.434705  5915 consensus_peers.cc:597] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:31.569042  2067 tablet_server.cc:196] TabletServer@127.2.4.193:38701 shutdown complete.
W20260709 12:28:31.588913  5840 consensus_peers.cc:597] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:31.590157  5840 consensus_peers.cc:597] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:31.609962  5840 consensus_peers.cc:597] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:31.611075  5840 consensus_peers.cc:597] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:31.630268  5915 consensus_peers.cc:597] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:31.674985  5840 consensus_peers.cc:597] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:31.698599  5840 consensus_peers.cc:597] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:31.726652  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 105714us
W20260709 12:28:31.747296  5990 consensus_peers.cc:597] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:31.765187  5840 consensus_peers.cc:597] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:31.820046  5915 consensus_peers.cc:597] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:31.923491  5915 consensus_peers.cc:597] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:32.117374  2067 tablet_server.cc:179] TabletServer@127.2.4.194:0 shutting down...
W20260709 12:28:32.135582  5915 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111) [suppressed 429 similar messages]
W20260709 12:28:32.261868  5990 consensus_peers.cc:597] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:32.311990  5838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 79416.1us
W20260709 12:28:32.343357  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 285324us
W20260709 12:28:32.372231  5840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 48732.8us
W20260709 12:28:32.392074  5840 consensus_peers.cc:597] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:32.443317  5987 consensus_peers.cc:597] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.464258  5913 consensus_peers.cc:597] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.465066  5913 consensus_peers.cc:597] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.465770  5913 consensus_peers.cc:597] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.466429  5913 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.467079  5913 consensus_peers.cc:597] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.484306  5913 consensus_peers.cc:597] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.485157  5913 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.485878  5913 consensus_peers.cc:597] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.487869  5913 consensus_peers.cc:597] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.489693  5913 consensus_peers.cc:597] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.489977  5987 consensus_peers.cc:597] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.490388  5913 consensus_peers.cc:597] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.490787  5987 consensus_peers.cc:597] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.495436  5987 consensus_peers.cc:597] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.496107  5987 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.496702  5987 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.496346  5913 consensus_peers.cc:597] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.497289  5987 consensus_peers.cc:597] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.498018  5987 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.498719  5987 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.509433  5913 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.519196  5915 consensus_peers.cc:597] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:32.520560  5915 consensus_peers.cc:597] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:32.521394  5915 consensus_peers.cc:597] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:32.522241  5915 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:32.522838  5915 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:32.523602  5915 consensus_peers.cc:597] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:32.549463  5915 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:32.551882  5913 consensus_peers.cc:597] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:32.550119  5915 consensus_peers.cc:597] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:32.565394  5915 consensus_peers.cc:597] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:32.578280  5840 consensus_peers.cc:597] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:32.587329  5913 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:32.615589  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:32.616878  2067 tablet_replica.cc:333] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.618186  2067 raft_consensus.cc:2243] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:28:32.621003  2067 raft_consensus.cc:2272] T 8d0c096d28fc46d3be987a091fcb5533 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:32.622604  5990 consensus_peers.cc:597] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:32.626359  2067 tablet_replica.cc:333] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.627558  2067 raft_consensus.cc:2243] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:32.628330  2067 raft_consensus.cc:2272] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7a0204b931394fc48ffc503716e0d111 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.635926  2067 tablet_replica.cc:333] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.637465  2067 raft_consensus.cc:2243] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:32.638917  2067 raft_consensus.cc:2272] T 48c0a9da01454a2794160d9c0ff08ea9 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.642663  2067 tablet_replica.cc:333] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.643728  2067 raft_consensus.cc:2243] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:32.645967  2067 raft_consensus.cc:2272] T 595c6563cdd74f47bda68ead57106a8b P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.650728  2067 tablet_replica.cc:333] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.651726  2067 raft_consensus.cc:2243] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:32.652526  2067 raft_consensus.cc:2272] T d45107f59eee48f79ae2b6006319cd62 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.656100  2067 tablet_replica.cc:333] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.657079  2067 raft_consensus.cc:2243] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:32.658617  2067 raft_consensus.cc:2272] T 1ba7af1a4a744df6bc8c20ec72e21519 P 7a0204b931394fc48ffc503716e0d111 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.662020  2067 tablet_replica.cc:333] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.662945  2067 raft_consensus.cc:2243] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:28:32.664450  2067 raft_consensus.cc:2272] T cfa3fcaa83734d509f6b31c3874873be P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.667788  2067 tablet_replica.cc:333] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.668681  2067 raft_consensus.cc:2243] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:28:32.670045  2067 raft_consensus.cc:2272] T c3c69fa4d49547f8a9e76c055c3ae7be P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.673537  2067 tablet_replica.cc:333] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.674513  2067 raft_consensus.cc:2243] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:32.675922  2067 raft_consensus.cc:2272] T 4ef7a99e80494a989eda0b1a37482656 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.685113  2067 tablet_replica.cc:333] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.687583  2067 raft_consensus.cc:2243] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:32.692811  2067 raft_consensus.cc:2272] T 60934281a651439a87cd8c11702f67d2 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.710759  2067 tablet_replica.cc:333] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.724752  2067 raft_consensus.cc:2243] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:32.733101  2067 raft_consensus.cc:2272] T 0006f949032c4034b4f4872ee332386e P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.736724  2067 tablet_replica.cc:333] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.737697  2067 raft_consensus.cc:2243] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:32.738996  2067 raft_consensus.cc:2272] T d04ce59a08f3427581e5e6b5d87ad757 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.743808  2067 tablet_replica.cc:333] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.745047  2067 raft_consensus.cc:2243] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:32.746196  2067 raft_consensus.cc:2272] T c0e0b097c72c4c748e146f8303c90f0c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.755890  2067 tablet_replica.cc:333] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.756672  2067 raft_consensus.cc:2243] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:28:32.758769  2067 raft_consensus.cc:2272] T 52af756fdd9b411c8c4b494e2552a3fb P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.777034  2067 tablet_replica.cc:333] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.778079  2067 raft_consensus.cc:2243] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:32.778995  2067 raft_consensus.cc:2272] T f3c3647dbbb74fbd86870b12d1409976 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:32.799961  5990 consensus_peers.cc:597] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:32.790243  2067 tablet_replica.cc:333] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.809132  2067 raft_consensus.cc:2243] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:32.810518  2067 raft_consensus.cc:2272] T bc2de53e733442a69e97d39affab60ba P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.820079  2067 tablet_replica.cc:333] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.823819  2067 raft_consensus.cc:2243] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:32.844044  2067 raft_consensus.cc:2272] T 860d4f6b513041829ed9cd83880edc5c P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.862061  2067 tablet_replica.cc:333] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.863358  2067 raft_consensus.cc:2243] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 27 LEADER]: Raft consensus shutting down.
W20260709 12:28:32.862056  5915 consensus_peers.cc:597] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:32.865939  2067 raft_consensus.cc:2272] T 6ad5821a38aa41d7abd76509a12a97df P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.872278  2067 tablet_replica.cc:333] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.873219  2067 raft_consensus.cc:2243] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:32.879246  2067 raft_consensus.cc:2272] T e55fd31b2e414885a9bd5c9b40e04e19 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.884223  2067 tablet_replica.cc:333] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.887264  2067 raft_consensus.cc:2243] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:32.889842  2067 raft_consensus.cc:2272] T 402bd52b71ce4044b0ec86bf6f8d73fc P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.922634  2067 tablet_replica.cc:333] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.923554  2067 raft_consensus.cc:2243] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:32.928115  2067 raft_consensus.cc:2272] T d4765e1fc00c485086dc883bea4b87f7 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:32.931053  5840 consensus_peers.cc:597] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:32.932770  2067 tablet_replica.cc:333] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.934168  2067 raft_consensus.cc:2243] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:32.935060  2067 raft_consensus.cc:2272] T 30da8f1ded30432a87f01684d88a9969 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.939378  2067 tablet_replica.cc:333] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.940776  2067 raft_consensus.cc:2243] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:32.941565  2067 raft_consensus.cc:2272] T d001526e46b24f429f57c91e4a270036 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.944991  2067 tablet_replica.cc:333] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.946002  2067 raft_consensus.cc:2243] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:32.946939  2067 raft_consensus.cc:2272] T 6ed8b895df664a3b8f7934c37fd8f181 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.960860 14102 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:32.968307 14102 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:32.973381  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:32.974867  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "187e00b075464d9c855ce2b87615ac70" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:32.974557 14102 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:32.976650  5840 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:32.978220 14102 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:32.989696  2067 tablet_replica.cc:333] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.991029  2067 raft_consensus.cc:2243] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:32.991890  2067 raft_consensus.cc:2272] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:32.995404  2067 tablet_replica.cc:333] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:32.996349  2067 raft_consensus.cc:2243] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:32.998088  2067 raft_consensus.cc:2272] T 685933c4b2364e1c90d92edb4c091799 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.025445  2067 tablet_replica.cc:333] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
W20260709 12:28:33.044440  5915 consensus_peers.cc:597] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:33.057927  2067 raft_consensus.cc:2243] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:33.058797  2067 raft_consensus.cc:2272] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:33.103719  5990 consensus_peers.cc:597] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:33.105157  5839 raft_consensus.cc:687] T 256f8c690fb2456f92b4035fc3bf5e23 P 7a0204b931394fc48ffc503716e0d111: failed to submit failure detected task: Service unavailable: Thread pool token was shut down
I20260709 12:28:33.106846 14069 raft_consensus.cc:493] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:33.107420 14069 raft_consensus.cc:515] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:33.107753 14088 raft_consensus.cc:493] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6add331326ca47038308a447bf798fa7)
I20260709 12:28:33.108296 14088 raft_consensus.cc:515] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:33.109825 14069 leader_election.cc:290] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:33.110323 14088 leader_election.cc:290] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:33.112623  2067 tablet_replica.cc:333] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:33.118456  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048b40ab425f451c983713b7df96c732" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:33.119230  2067 raft_consensus.cc:2243] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:33.121441  2067 raft_consensus.cc:2272] T e9bb01883059454e80b19d5333aad71e P 7a0204b931394fc48ffc503716e0d111 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.121357  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42697dffe5a495ea1ba4a111df5ba39" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:33.134195  5840 leader_election.cc:336] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:33.134760  5840 leader_election.cc:304] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:33.142163 14102 raft_consensus.cc:2749] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:33.141758  2067 tablet_replica.cc:333] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
W20260709 12:28:33.143317  5840 leader_election.cc:336] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:33.143766  5840 leader_election.cc:304] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 6add331326ca47038308a447bf798fa7
I20260709 12:28:33.144485 14102 raft_consensus.cc:2749] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:33.144258  2067 raft_consensus.cc:2243] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:33.146433  2067 raft_consensus.cc:2272] T 59cab2d7e5234efd9a97b8377f1919f7 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.150300  2067 tablet_replica.cc:333] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:33.151217  2067 raft_consensus.cc:2243] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:28:33.155207  2067 raft_consensus.cc:2272] T 428a50885681465a98089cb853b233b9 P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.159106  2067 tablet_replica.cc:333] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
W20260709 12:28:33.159227  5915 consensus_peers.cc:597] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:33.160260  2067 raft_consensus.cc:2243] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:28:33.161826  2067 raft_consensus.cc:2272] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.165410  2067 tablet_replica.cc:333] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:33.166498  2067 raft_consensus.cc:2243] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:28:33.167980  2067 raft_consensus.cc:2272] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.171325  2067 tablet_replica.cc:333] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:33.172276  2067 raft_consensus.cc:2243] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:33.173110  2067 raft_consensus.cc:2272] T 8e301d9d5b914f3b9ec07dea273d5126 P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.176420  2067 tablet_replica.cc:333] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:33.177446  2067 raft_consensus.cc:2243] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:33.178910  2067 raft_consensus.cc:2272] T a80a3917fd9748bc8ae4817757081f09 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.182502  2067 tablet_replica.cc:333] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:33.183434  2067 raft_consensus.cc:2243] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:28:33.184626  2067 raft_consensus.cc:2272] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 7a0204b931394fc48ffc503716e0d111 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.188277  2067 tablet_replica.cc:333] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:33.189514  2067 raft_consensus.cc:2243] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:33.190479  2067 raft_consensus.cc:2272] T e42697dffe5a495ea1ba4a111df5ba39 P 7a0204b931394fc48ffc503716e0d111 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.196614  2067 tablet_replica.cc:333] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:33.199169  2067 raft_consensus.cc:2243] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:33.201038  2067 raft_consensus.cc:2272] T 9bede7b5a3c84fe28a073542219f3fe4 P 7a0204b931394fc48ffc503716e0d111 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.204943  2067 tablet_replica.cc:333] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:33.205935  2067 raft_consensus.cc:2243] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:28:33.207883  2067 raft_consensus.cc:2272] T 18a324a652924631995e78ee3fda8422 P 7a0204b931394fc48ffc503716e0d111 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.211612  2067 tablet_replica.cc:333] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:33.212585  2067 raft_consensus.cc:2243] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:33.214012  2067 raft_consensus.cc:2272] T ebb084ec08f44cb2a935c184afafadb0 P 7a0204b931394fc48ffc503716e0d111 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.217927  2067 tablet_replica.cc:333] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:33.218943  2067 raft_consensus.cc:2243] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:33.220247  2067 raft_consensus.cc:2272] T e657475654794984a13c42a668f3158b P 7a0204b931394fc48ffc503716e0d111 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.222225 14102 raft_consensus.cc:493] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:33.222781 14102 raft_consensus.cc:515] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:33.226257 14102 leader_election.cc:290] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:33.226481  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbd0e138da244d48f133d23b4aa1eda" candidate_uuid: "7a0204b931394fc48ffc503716e0d111" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:33.229513  2067 tablet_replica.cc:333] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:33.230454  2067 raft_consensus.cc:2243] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:28:33.232160  2067 raft_consensus.cc:2272] T 7a3145d5d96e41518c7bf41999289ac3 P 7a0204b931394fc48ffc503716e0d111 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:33.234323  5840 leader_election.cc:336] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:33.234828  5840 leader_election.cc:304] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a0204b931394fc48ffc503716e0d111; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:33.235529 14102 raft_consensus.cc:2749] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:33.236291  2067 tablet_replica.cc:333] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:33.237370  2067 raft_consensus.cc:2243] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:33.238199  2067 raft_consensus.cc:2272] T edbd0e138da244d48f133d23b4aa1eda P 7a0204b931394fc48ffc503716e0d111 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.241958  2067 tablet_replica.cc:333] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:33.242687  2067 raft_consensus.cc:2243] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:33.243474  2067 raft_consensus.cc:2272] T 048b40ab425f451c983713b7df96c732 P 7a0204b931394fc48ffc503716e0d111 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:33.246645  2067 tablet_replica.cc:333] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111: stopping tablet replica
I20260709 12:28:33.247285  2067 raft_consensus.cc:2243] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:33.247910  2067 raft_consensus.cc:2272] T 187e00b075464d9c855ce2b87615ac70 P 7a0204b931394fc48ffc503716e0d111 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:33.251137  5990 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:33.291505  5990 consensus_peers.cc:597] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:33.314042  5915 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:33.429790  5990 consensus_peers.cc:597] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:33.467181  5990 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:33.486176  5990 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:33.737092  5915 consensus_peers.cc:597] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:28:33.800873 14098 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:33.801532 14098 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:33.802466 14084 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:33.803057 14084 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:33.803866 14098 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:33.805065 14084 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:33.812777  5987 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:33.813380  5987 leader_election.cc:336] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:33.829854  5990 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:33.830377  5990 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:33.834838 14084 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:33.835881  5990 leader_election.cc:336] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:33.837877  5990 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:33.840790 14084 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:33.874366  5990 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:33.899505 14113 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:33.900167 14113 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:33.902879 14113 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:33.919929 14084 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:33.920594 14084 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:33.923223 14084 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:33.933331 14118 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:33.934445 14118 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:33.936792 14118 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:33.940516  5913 leader_election.cc:336] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:33.944286  5987 leader_election.cc:336] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:33.953912  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:33.980248 14113 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:33.980895 14113 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:33.983182 14113 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:33.983752 14113 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:33.984228 14113 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:33.992005 14084 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.034169 14084 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:34.035455 14113 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:34.014072  5915 leader_election.cc:336] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:34.036165 14084 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:34.029932  5913 leader_election.cc:336] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.027874  5987 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:34.038470 14084 raft_consensus.cc:2749] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:34.036103  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.068424  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:34.069164  6016 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 27.
I20260709 12:28:34.073151 14118 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:34.080613 14098 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.081249 14098 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:34.083319 14098 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:34.085956  5987 leader_election.cc:336] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.094422  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:34.095294  5941 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 27.
I20260709 12:28:34.100148  5941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
W20260709 12:28:34.100564  5913 leader_election.cc:336] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.104488 14118 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.105439 14118 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:34.107743 14118 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:34.114780  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.116254 14113 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:34.116681 14113 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.117070 14113 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:34.135686  5987 leader_election.cc:336] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:34.144964  5915 leader_election.cc:336] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:34.149121  5913 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 47863.6us
I20260709 12:28:34.145583  5915 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:34.150971  5913 leader_election.cc:336] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.152637 14122 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:34.184340 14122 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.185039 14122 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:34.189337 14122 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:34.271548  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.292061 14113 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:34.311772 14084 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:34.312358 14084 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.312716 14084 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:34.313499 14098 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.314147 14098 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:34.316424 14098 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:34.333336 14098 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:28:34.334053  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 62610.3us
I20260709 12:28:34.333961 14098 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:34.334928  5987 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:34.336277 14098 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:34.337009 14098 raft_consensus.cc:2749] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:34.379554  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:43329, user_credentials={real_user=slave}} blocked reactor thread for 44733.3us
I20260709 12:28:34.380484 14129 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.346057 14123 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.381208 14129 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:34.391695 14129 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:34.391788 14123 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:34.399098 14118 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.391891 14113 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:34.381208 14084 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:34.407207 14128 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.408097 14124 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.408272 14123 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:34.400013 14118 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:34.415275  5990 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:34.415930  5990 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:34.418361 14123 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.418995 14123 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:34.419268 14124 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:34.416117 14128 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:34.422837 14084 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:34.422845 14128 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:34.421619 14124 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:34.437491  5915 consensus_peers.cc:597] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:34.438387  5915 leader_election.cc:336] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:34.438853  5915 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.440047  6022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:34.440929 14118 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:34.440971  6022 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 27.
I20260709 12:28:34.443342 14123 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:34.454512 14127 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:34.457031 14113 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.457512  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:34.457692 14113 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:34.458617  6016 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 28: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
I20260709 12:28:34.460072 14113 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:34.461982  5987 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.462602  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.463158  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:34.463569 14124 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:34.463740  5987 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.463960  5947 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 23.
I20260709 12:28:34.464408  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.465276 14124 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:34.471881  5987 leader_election.cc:336] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:34.472587  5987 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:34.473106  5987 leader_election.cc:336] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:34.473659  5987 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.475275 14131 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.477301 14131 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:34.476557  5913 leader_election.cc:336] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:34.484508  5913 leader_election.cc:336] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.483026 14132 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.486074 14131 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:34.486737 14132 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:34.488777  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.489780 14124 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:34.489773  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.490414 14124 raft_consensus.cc:493] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.490847 14124 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:34.490896 14118 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:34.491401 14118 raft_consensus.cc:493] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.491485 14132 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:34.491828 14118 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:34.496017  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
I20260709 12:28:34.496876  5947 raft_consensus.cc:2393] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 6add331326ca47038308a447bf798fa7 in current term 28: Already voted for candidate 7eabb96dfdd1449c94ad1261ded46c3c in this term.
I20260709 12:28:34.499586 14124 raft_consensus.cc:515] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:34.501353 14118 raft_consensus.cc:515] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:34.503100  5987 consensus_peers.cc:597] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:34.503389 14118 leader_election.cc:290] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:34.504237  5913 leader_election.cc:336] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.505517 14124 leader_election.cc:290] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:34.506520  5913 leader_election.cc:336] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.506616  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:34.507611  5913 consensus_peers.cc:597] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:34.508746  5913 leader_election.cc:336] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.509442 14124 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:34.506616  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:34.511852  5987 leader_election.cc:336] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.513092 14118 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:28:34.514606  5990 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:34.515185  5990 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:34.516335  5990 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:34.516778  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.517779 14134 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:34.509581  5913 leader_election.cc:336] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.518688 14134 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:34.521706  5913 consensus_peers.cc:597] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:34.522208  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:34.522567  2067 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 5896)
W20260709 12:28:34.522789  5913 leader_election.cc:336] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.523017  6016 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:28:34.524835  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:34.528273  6018 raft_consensus.cc:2393] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eabb96dfdd1449c94ad1261ded46c3c in current term 24: Already voted for candidate 6add331326ca47038308a447bf798fa7 in this term.
W20260709 12:28:34.530975  5990 consensus_peers.cc:597] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:34.531164  5915 leader_election.cc:336] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:34.532215  5915 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.533161 14132 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:34.533679  5915 consensus_peers.cc:597] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:34.536841  5913 leader_election.cc:336] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.536778  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80a3917fd9748bc8ae4817757081f09" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c"
W20260709 12:28:34.537616  5913 leader_election.cc:336] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.537681  5947 raft_consensus.cc:3060] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:34.541047  5913 leader_election.cc:336] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.547725  5947 raft_consensus.cc:2468] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6add331326ca47038308a447bf798fa7 in term 24.
I20260709 12:28:34.549489  5987 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.550921 14124 raft_consensus.cc:2804] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:34.551510 14124 raft_consensus.cc:697] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Becoming Leader. State: Replica: 6add331326ca47038308a447bf798fa7, State: Running, Role: LEADER
I20260709 12:28:34.552528 14124 consensus_queue.cc:237] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:34.560070 14134 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.560853 14134 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:34.563776 14134 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:34.566694  5990 consensus_peers.cc:597] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:34.568202  5987 leader_election.cc:336] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.568272  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad5821a38aa41d7abd76509a12a97df" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:34.569180  6018 raft_consensus.cc:3060] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:34.566655  5947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "6add331326ca47038308a447bf798fa7" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" is_pre_election: true
I20260709 12:28:34.571494  5987 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:34.572628 14124 raft_consensus.cc:2749] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:34.575147  5915 consensus_peers.cc:597] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:34.575901  5915 leader_election.cc:336] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:34.576339  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.577301 14118 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:34.578898  5915 consensus_peers.cc:597] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:34.579738  5915 leader_election.cc:336] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:34.580204  5915 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.581143 14118 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:34.583352  5915 consensus_peers.cc:597] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:34.586745  6018 raft_consensus.cc:2468] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 28.
I20260709 12:28:34.589457  5915 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.592466  5696 catalog_manager.cc:5697] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 reported cstate change: term changed from 23 to 24, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 6add331326ca47038308a447bf798fa7 (127.2.4.196). New cstate: current_term: 24 leader_uuid: "6add331326ca47038308a447bf798fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:34.600441  5987 consensus_peers.cc:597] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:34.602413 14132 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:34.603123 14132 raft_consensus.cc:493] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.603062  5915 leader_election.cc:304] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 6add331326ca47038308a447bf798fa7, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.603932 14132 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:34.604820 14118 raft_consensus.cc:2749] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:34.629504  5915 leader_election.cc:304] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.630877 14118 raft_consensus.cc:2804] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:34.632290 14118 raft_consensus.cc:697] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:34.634196 14118 consensus_queue.cc:237] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:34.643174  5915 leader_election.cc:336] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:34.643878  5915 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.638324 14132 raft_consensus.cc:515] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:34.645754 14131 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:34.645912  5915 leader_election.cc:336] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:34.646694  5915 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.647564 14132 leader_election.cc:290] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:34.648257 14132 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:34.649116 14124 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.650094 14124 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:34.663579  5990 leader_election.cc:336] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:34.667701  5913 leader_election.cc:336] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:34.668398  5913 consensus_peers.cc:597] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:34.675300 14124 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:34.680719  5987 leader_election.cc:336] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.682909  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.684005 14134 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:34.696859 14118 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.697576 14118 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:34.699643  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60934281a651439a87cd8c11702f67d2" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:34.699990 14118 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:34.700381  6018 raft_consensus.cc:3060] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:34.706754  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
W20260709 12:28:34.707631  5913 leader_election.cc:336] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.708101  6016 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 23.
I20260709 12:28:34.717778  5915 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:34.718008  5987 consensus_peers.cc:597] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:34.726706 14123 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:34.727200 14123 raft_consensus.cc:493] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:34.727581 14123 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:34.744997  6018 raft_consensus.cc:2468] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 25.
I20260709 12:28:34.748862  5915 leader_election.cc:304] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.749892 14132 raft_consensus.cc:2804] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:28:34.751540  5915 consensus_peers.cc:597] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:34.750424 14132 raft_consensus.cc:697] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:34.752857 14132 consensus_queue.cc:237] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:34.768764  5688 catalog_manager.cc:5697] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 27 to 28, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 28 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:34.779036 14123 raft_consensus.cc:515] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:34.785471 14123 leader_election.cc:290] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:34.787313  6018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb084ec08f44cb2a935c184afafadb0" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:34.791235  6018 raft_consensus.cc:3060] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:34.797878  5913 leader_election.cc:336] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:34.800757  6018 raft_consensus.cc:2468] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 24.
I20260709 12:28:34.802557  5915 leader_election.cc:304] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:34.803984 14123 raft_consensus.cc:2804] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:34.804596 14123 raft_consensus.cc:697] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:34.805572 14123 consensus_queue.cc:237] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:34.813529  5987 consensus_peers.cc:597] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:34.824909  5915 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:34.826079  5915 consensus_peers.cc:597] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:34.826483  5987 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:34.907405  5915 consensus_peers.cc:597] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:34.911780  5688 catalog_manager.cc:5697] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 24 to 25, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 25 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:34.914832  5688 catalog_manager.cc:5697] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 23 to 24, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 24 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:34.936853  5913 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:34.944851  5913 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:34.966591  5915 consensus_peers.cc:597] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:34.967701  5913 consensus_peers.cc:597] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:34.970055  5987 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:34.970832  5987 consensus_peers.cc:597] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:34.993613  5913 consensus_peers.cc:597] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:35.001343  5947 raft_consensus.cc:1275] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Refusing update from remote peer 6add331326ca47038308a447bf798fa7: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:28:35.003310 14124 consensus_queue.cc:1048] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:28:35.028556  5987 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:35.038240  5913 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:35.041321  5915 consensus_peers.cc:597] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:35.051518  5915 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:35.055291  6018 raft_consensus.cc:1275] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:28:35.057893 14123 consensus_queue.cc:1048] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:28:35.076215  5987 consensus_peers.cc:597] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:35.076930  5990 consensus_peers.cc:597] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:35.133054  5913 consensus_peers.cc:597] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:35.147773  5987 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:35.154467  5913 consensus_peers.cc:597] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:35.155377  5913 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:35.155849  5913 consensus_peers.cc:597] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:35.161983  5915 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:35.167907  6018 raft_consensus.cc:1275] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:35.178656 14132 consensus_queue.cc:1048] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:28:35.221298  5913 consensus_peers.cc:597] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:35.231984  5913 consensus_peers.cc:597] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:35.244556  5913 consensus_peers.cc:597] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:35.252641  5913 consensus_peers.cc:597] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:35.298908  5913 consensus_peers.cc:597] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:35.342677  5990 consensus_peers.cc:597] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:35.369659 14134 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:35.370432 14134 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:35.384402  5990 leader_election.cc:336] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:35.397416 14134 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:35.402133  5987 leader_election.cc:336] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:35.403016  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:35.405961 14124 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:35.433203  6016 raft_consensus.cc:1275] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:28:35.440572 14147 consensus_queue.cc:1048] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:28:35.524086  2067 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 5896)
W20260709 12:28:35.565112  5915 consensus_peers.cc:597] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:35.587749  5913 consensus_peers.cc:597] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:35.591604  5990 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:35.612154  5915 consensus_peers.cc:597] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:35.618537 14147 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:35.619243 14147 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:35.628746  5913 leader_election.cc:336] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:35.629873 14147 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:35.650996  5915 consensus_peers.cc:597] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:35.668306  5915 leader_election.cc:336] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:35.668955  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:35.669941 14147 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:35.735486  5915 consensus_peers.cc:597] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:28:35.838444 14147 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:35.839087 14147 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:35.841343 14147 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:35.847775  5915 leader_election.cc:336] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:35.848716  5913 leader_election.cc:336] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:35.849227  5913 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:35.850163 14147 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:35.882206  5990 consensus_peers.cc:597] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:35.892411  5990 consensus_peers.cc:597] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:35.944701 14124 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:35.945518 14124 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:35.951563 14124 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:35.952448  5990 consensus_peers.cc:597] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:35.953420  5987 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:35.965890  5990 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:35.966459  5990 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:35.972923 14124 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:35.988516  5915 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:35.997334  5915 consensus_peers.cc:597] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:36.020040 14131 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:36.020663 14131 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:36.023326 14131 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
I20260709 12:28:36.026259  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7" is_pre_election: true
I20260709 12:28:36.027128  6016 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 28.
W20260709 12:28:36.030901  5913 leader_election.cc:336] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:36.032402  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:36.033396 14131 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:36.033984 14131 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:36.034363 14131 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:36.048400 14131 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:36.067034  6016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0c096d28fc46d3be987a091fcb5533" candidate_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "6add331326ca47038308a447bf798fa7"
I20260709 12:28:36.067764  6016 raft_consensus.cc:3060] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:36.099684  5990 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:28:36.102501 14131 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 election: Requested vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 6add331326ca47038308a447bf798fa7 (127.2.4.196:34835)
W20260709 12:28:36.119684  5913 leader_election.cc:336] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:36.081948  6016 raft_consensus.cc:2468] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eabb96dfdd1449c94ad1261ded46c3c in term 29.
I20260709 12:28:36.123896  5915 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6add331326ca47038308a447bf798fa7, 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:36.124982 14153 raft_consensus.cc:2804] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:36.125687 14153 raft_consensus.cc:697] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 29 LEADER]: Becoming Leader. State: Replica: 7eabb96dfdd1449c94ad1261ded46c3c, State: Running, Role: LEADER
I20260709 12:28:36.126777 14153 consensus_queue.cc:237] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:36.158568  5688 catalog_manager.cc:5697] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c reported cstate change: term changed from 27 to 29, leader changed from 7a0204b931394fc48ffc503716e0d111 (127.2.4.194) to 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195). New cstate: current_term: 29 leader_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:36.229058 14153 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:36.232218 14153 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:36.234905 14153 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:36.264984  5915 leader_election.cc:336] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:36.266855  5913 leader_election.cc:336] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:36.267450  5913 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:36.268616 14123 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:36.294553 14123 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:36.295361 14123 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:36.298745 14123 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:36.306547  5913 leader_election.cc:336] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:36.317130  5915 leader_election.cc:336] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:36.317834  5915 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:36.318861 14123 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:36.334499 14152 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:36.335186 14152 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:36.337467 14152 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:36.349234  5990 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:36.351712  5987 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:36.352147  5987 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:36.352807 14152 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:36.367307  5990 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:36.384881 14153 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:36.385880 14153 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:36.391793 14153 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:36.397583  5915 leader_election.cc:336] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:36.400070 14152 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:36.400820 14152 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:36.405262 14152 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:36.407008  5913 leader_election.cc:336] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:36.415292  5913 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:36.416180 14153 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:36.410530  5990 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:36.417997  5987 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:36.418810  5987 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:36.427663 14124 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:36.519695  5990 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:36.524876  2067 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 5896)
I20260709 12:28:36.596274 14152 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:36.597083 14152 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:36.599440 14124 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:36.600144 14124 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:36.602377 14124 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:36.605337 14152 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:36.631304  5990 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:36.635985  5990 leader_election.cc:336] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:36.636437  5987 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:36.644693  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:36.657286  5987 leader_election.cc:336] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:36.663552  5987 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
W20260709 12:28:36.646255  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.196:34835, user_credentials={real_user=slave}} blocked reactor thread for 74646.5us
I20260709 12:28:36.668519  6016 raft_consensus.cc:1275] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 29 FOLLOWER]: Refusing update from remote peer 7eabb96dfdd1449c94ad1261ded46c3c: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:28:36.657974 14152 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:36.682701 14147 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:36.683605 14147 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:36.686528 14147 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:36.689095 14152 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:36.696450 14147 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:28:36.698146  5915 consensus_peers.cc:597] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:36.704788 14152 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:36.705616 14152 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:36.708508 14152 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:36.697273 14147 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:36.734499 14153 consensus_queue.cc:1048] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:36.742969 14147 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:36.747493  5987 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:36.750113  5990 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:36.750658  5990 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:36.751446 14124 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:36.757349  5913 consensus_peers.cc:597] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:36.758369  5913 leader_election.cc:336] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:36.761912  5915 consensus_peers.cc:597] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:36.761945 14153 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:36.766592 14153 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:36.775569 14153 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:36.780191  5915 leader_election.cc:336] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:36.780807  5915 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:36.782989 14131 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:36.787052  5915 leader_election.cc:336] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:36.787655  5915 leader_election.cc:336] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:36.822242  5915 consensus_peers.cc:597] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:36.825690  5913 leader_election.cc:336] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:36.843705  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:36.844657 14147 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:36.845768  5913 consensus_peers.cc:597] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:36.869465  5913 leader_election.cc:336] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:36.870055  5913 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:36.875422 14131 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:36.880693  5915 consensus_peers.cc:597] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:36.920796  5915 consensus_peers.cc:597] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:36.970366  5987 consensus_peers.cc:597] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.058372  5913 consensus_peers.cc:597] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.075166  5987 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.092941  5990 consensus_peers.cc:597] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:37.104655  5987 consensus_peers.cc:597] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.117091  5987 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.130613  5913 consensus_peers.cc:597] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.136560  5913 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111) [suppressed 664 similar messages]
W20260709 12:28:37.156260  5913 consensus_peers.cc:597] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.195140  5990 consensus_peers.cc:597] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:37.298405  5987 consensus_peers.cc:597] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.348662  5987 consensus_peers.cc:597] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:37.525478  2067 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 5896)
W20260709 12:28:37.629514  5915 consensus_peers.cc:597] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:37.633355  5990 consensus_peers.cc:597] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:37.641726  5913 consensus_peers.cc:597] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.642889  5913 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.652341  5987 consensus_peers.cc:597] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.657305  5987 consensus_peers.cc:597] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.662524  5913 consensus_peers.cc:597] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:37.670523  5913 consensus_peers.cc:597] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.671131  5913 consensus_peers.cc:597] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.671612  5913 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.677455  5915 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:37.678151  5915 consensus_peers.cc:597] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:37.679059  5915 consensus_peers.cc:597] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:37.679541  5915 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:37.687690  5915 consensus_peers.cc:597] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:37.688333  5913 consensus_peers.cc:597] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:37.689111  5915 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:37.689683  5915 consensus_peers.cc:597] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:37.733742  5913 consensus_peers.cc:597] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:37.778625  5987 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.790357  5913 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:37.796157 14168 raft_consensus.cc:493] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:37.796806 14168 raft_consensus.cc:515] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:37.799392 14168 leader_election.cc:290] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:37.803321  5990 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:37.805572  5987 leader_election.cc:336] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:37.806200  5987 leader_election.cc:304] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:37.806912 14168 raft_consensus.cc:2749] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:37.920627  5913 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.921406  5990 consensus_peers.cc:597] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:37.943761  5913 consensus_peers.cc:597] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:37.999714  5987 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:38.024403  5913 consensus_peers.cc:597] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:38.024930  5913 consensus_peers.cc:597] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:38.146616  5913 consensus_peers.cc:597] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:38.247690  5990 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:28:38.255106  5990 consensus_peers.cc:597] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:38.260201  5915 consensus_peers.cc:597] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:38.268628 14168 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:38.269409 14168 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:38.279529 14168 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:38.285243  5915 consensus_peers.cc:597] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:38.290542  5915 consensus_peers.cc:597] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:38.291973  5915 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:38.295689 14162 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:38.296309 14162 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:38.286629  5987 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:38.302047 14162 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:38.314952  5915 leader_election.cc:336] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:38.321806  5913 leader_election.cc:336] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:38.322372  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:38.323693 14170 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:38.328908  5990 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:38.330425  5990 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:38.332404 14168 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:38.340806 14168 raft_consensus.cc:493] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:38.341904 14168 raft_consensus.cc:515] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:38.345299 14168 leader_election.cc:290] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:38.358108  5990 consensus_peers.cc:597] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:38.358875  5990 leader_election.cc:336] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:38.362116  5915 consensus_peers.cc:597] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:38.367753  5970 debug-util.cc:398] Leaking SignalData structure 0x7b08001413e0 after lost signal to thread 2070
W20260709 12:28:38.369318  5970 debug-util.cc:398] Leaking SignalData structure 0x7b0800125140 after lost signal to thread 5731
W20260709 12:28:38.371093  5970 debug-util.cc:398] Leaking SignalData structure 0x7b0800197400 after lost signal to thread 5973
W20260709 12:28:38.372596  5970 debug-util.cc:398] Leaking SignalData structure 0x7b08000962a0 after lost signal to thread 6048
W20260709 12:28:38.375379  5987 leader_election.cc:336] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:38.375948  5987 leader_election.cc:304] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:38.377895 14168 raft_consensus.cc:2749] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:38.509264 14170 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:38.510006 14170 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:38.514218 14170 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:38.519546  5915 leader_election.cc:336] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:38.525996  5913 leader_election.cc:336] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:38.526381  2067 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 5896)
I20260709 12:28:38.526562  5913 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:38.527552 14170 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:38.565653 14162 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:38.566393 14162 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
W20260709 12:28:38.580672  5915 leader_election.cc:336] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:38.579193 14162 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:38.589780  5913 leader_election.cc:336] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:38.591259  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:38.592294 14162 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:38.604907  5990 consensus_peers.cc:597] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:38.626005  5915 consensus_peers.cc:597] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:38.653759  5990 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:28:38.812613 14147 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:38.813855 14162 raft_consensus.cc:493] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:38.813470 14147 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:38.814517 14162 raft_consensus.cc:515] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:38.816752 14162 leader_election.cc:290] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:38.824141  5913 leader_election.cc:336] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:38.824970  5913 leader_election.cc:336] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:38.825911 14147 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:38.870101  5915 consensus_peers.cc:597] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:38.876206  5915 leader_election.cc:336] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:38.876873  5915 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:38.909137 14147 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:38.911670  5915 leader_election.cc:336] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:38.913458  5915 leader_election.cc:304] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:38.917052 14162 raft_consensus.cc:2749] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:38.955598  5990 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:28:39.084427 14147 raft_consensus.cc:493] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:39.085103 14147 raft_consensus.cc:515] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:39.087476 14147 leader_election.cc:290] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:39.126158  5913 leader_election.cc:336] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:39.171077  5990 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:39.201234  5915 consensus_peers.cc:597] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:39.205592  5915 leader_election.cc:336] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:39.207948  5915 leader_election.cc:304] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:39.211220 14170 raft_consensus.cc:2749] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:39.266211  5913 consensus_peers.cc:597] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:39.304219  5915 consensus_peers.cc:597] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:39.451445  5987 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:39.479547  5987 consensus_peers.cc:597] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:39.488286  5913 consensus_peers.cc:597] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:39.497774 14175 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:39.498818 14175 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:39.505903 14175 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
I20260709 12:28:39.523778 14171 raft_consensus.cc:493] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
W20260709 12:28:39.527647  2067 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 5896)
I20260709 12:28:39.527347 14171 raft_consensus.cc:515] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:39.529901 14171 leader_election.cc:290] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:39.530750  5987 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:39.542683  5990 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:39.543233  5990 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:39.545521 14171 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:39.562433  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 44706.7us
W20260709 12:28:39.581404  5987 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:39.583693  5990 leader_election.cc:336] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:39.587641  5990 leader_election.cc:304] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:39.603890 14175 raft_consensus.cc:2749] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:39.611799  5913 consensus_peers.cc:597] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:39.618804  5915 consensus_peers.cc:597] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:39.686863  5987 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:39.707547  5987 consensus_peers.cc:597] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:39.742336 14175 raft_consensus.cc:493] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:39.743199 14175 raft_consensus.cc:515] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
W20260709 12:28:39.754164  5987 consensus_peers.cc:597] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:39.759461 14175 leader_election.cc:290] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:39.760363  5987 consensus_peers.cc:597] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:39.774649  5915 consensus_peers.cc:597] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:39.777602  5915 consensus_peers.cc:597] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:39.781041  5987 leader_election.cc:336] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:39.786821  5990 leader_election.cc:336] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:39.791873  5990 leader_election.cc:304] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:39.798040 14175 raft_consensus.cc:2749] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:39.801045 14170 raft_consensus.cc:493] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:39.802156 14170 raft_consensus.cc:515] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:39.804862 14170 leader_election.cc:290] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:39.824079  5987 consensus_peers.cc:597] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:39.834772  5913 leader_election.cc:336] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:39.865820  5987 consensus_peers.cc:597] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:39.869863  5913 consensus_peers.cc:597] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:39.867172  5915 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:28:39.878018 14147 raft_consensus.cc:493] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:39.878650 14147 raft_consensus.cc:515] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:39.884644 14147 leader_election.cc:290] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:39.905234  5913 leader_election.cc:336] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:39.964836  5990 consensus_peers.cc:597] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:39.979596  5915 consensus_peers.cc:597] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:40.018545  5990 consensus_peers.cc:597] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:40.029335 14175 raft_consensus.cc:493] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:40.030088 14175 raft_consensus.cc:515] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:40.032848 14175 leader_election.cc:290] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:40.046908  5987 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:40.071403 14174 raft_consensus.cc:493] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:40.072218 14174 raft_consensus.cc:515] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:40.074587 14174 leader_election.cc:290] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:40.092249  5915 leader_election.cc:336] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:40.092788  5915 leader_election.cc:304] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:40.093740 14177 raft_consensus.cc:2749] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:40.093981  5915 leader_election.cc:336] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:40.094506  5915 leader_election.cc:304] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:40.105710 14177 raft_consensus.cc:2749] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:40.137249  5915 consensus_peers.cc:597] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:40.359036  5990 leader_election.cc:336] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:40.360095  5990 leader_election.cc:304] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:40.361339 14175 raft_consensus.cc:2749] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:40.368176  5987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:38005, user_credentials={real_user=slave}} blocked reactor thread for 259596us
W20260709 12:28:40.541499  5915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 401688us
W20260709 12:28:40.551685  5915 leader_election.cc:336] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:40.556763  5913 consensus_peers.cc:597] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:40.557453  5913 leader_election.cc:336] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:40.557902  5913 leader_election.cc:304] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:40.559603 14177 raft_consensus.cc:2749] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:40.559681 14147 raft_consensus.cc:493] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:40.561726 14147 raft_consensus.cc:515] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:40.564146 14147 leader_election.cc:290] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:40.566792  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 207856us
W20260709 12:28:40.602728  5915 consensus_peers.cc:597] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:40.620306 14147 raft_consensus.cc:493] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:40.620888 14147 raft_consensus.cc:515] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:40.623086 14147 leader_election.cc:290] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:40.637472  5987 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:40.639417  5915 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:40.649133  5915 consensus_peers.cc:597] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:40.649354  5913 consensus_peers.cc:597] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:40.650393  5913 consensus_peers.cc:597] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:40.657493  5915 leader_election.cc:336] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:40.658296  5913 consensus_peers.cc:597] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:40.662322  5915 consensus_peers.cc:597] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:40.665818  5913 consensus_peers.cc:597] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:40.663769  5990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:38701, user_credentials={real_user=slave}} blocked reactor thread for 46162us
W20260709 12:28:40.666400  5913 consensus_peers.cc:597] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:40.666852  5913 consensus_peers.cc:597] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:40.667061  5990 consensus_peers.cc:597] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:40.670945  5990 consensus_peers.cc:597] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:40.667281  5913 consensus_peers.cc:597] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:40.673964  5913 consensus_peers.cc:597] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:40.674572  5913 consensus_peers.cc:597] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:40.675040  5913 consensus_peers.cc:597] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:40.675586  5913 leader_election.cc:336] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:40.678527 14180 raft_consensus.cc:493] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:40.679107 14180 raft_consensus.cc:515] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } }
I20260709 12:28:40.681311 14180 leader_election.cc:290] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:40.684167  5915 leader_election.cc:336] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:40.684638  5915 leader_election.cc:304] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:40.685405 14170 raft_consensus.cc:2749] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:40.685768  5913 leader_election.cc:336] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:40.686451  5915 consensus_peers.cc:597] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:28:40.686496  5913 leader_election.cc:304] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:40.687875 14182 raft_consensus.cc:2749] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:40.691743  5915 leader_election.cc:336] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:40.693485  5913 leader_election.cc:336] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:40.693938  5913 leader_election.cc:304] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eabb96dfdd1449c94ad1261ded46c3c; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:40.694674 14182 raft_consensus.cc:2749] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:40.694929  5913 consensus_peers.cc:597] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:40.736114 14181 raft_consensus.cc:493] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:40.736838 14181 raft_consensus.cc:515] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:40.740573 14181 leader_election.cc:290] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:40.745926  5990 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:40.754568  5987 leader_election.cc:336] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:40.755095  5987 leader_election.cc:304] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:40.755894 14181 raft_consensus.cc:2749] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:40.763095  2067 tablet_server.cc:196] TabletServer@127.2.4.194:0 shutdown complete.
I20260709 12:28:40.880102  2067 tablet_server.cc:179] TabletServer@127.2.4.195:0 shutting down...
W20260709 12:28:40.896721  5987 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:40.972095  5987 consensus_peers.cc:597] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 -> Peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Couldn't send request to peer 7eabb96dfdd1449c94ad1261ded46c3c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:40.973315  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:40.974506  2067 tablet_replica.cc:333] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:40.975450  2067 raft_consensus.cc:2243] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:40.976816  2067 raft_consensus.cc:2272] T 99b9212afb8b459bb96908f8259bb8c0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:40.979805  2067 tablet_replica.cc:333] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:40.980674  2067 raft_consensus.cc:2243] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:40.982070  2067 raft_consensus.cc:2272] T 8d0c096d28fc46d3be987a091fcb5533 P 7eabb96dfdd1449c94ad1261ded46c3c [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:40.985047  2067 tablet_replica.cc:333] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:40.986006  2067 raft_consensus.cc:2243] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:40.987747  2067 raft_consensus.cc:2272] T 6a1efdcc52fa4f98acdbe3c7f5c38570 P 7eabb96dfdd1449c94ad1261ded46c3c [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:40.991225  2067 tablet_replica.cc:333] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:40.992187  2067 raft_consensus.cc:2243] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:40.993772  2067 raft_consensus.cc:2272] T fee70b2d75314f788ad873b1950fb890 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:40.994124  5990 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:28:40.996977  2067 tablet_replica.cc:333] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:40.997920  2067 raft_consensus.cc:2243] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:40.998900  2067 raft_consensus.cc:2272] T d45107f59eee48f79ae2b6006319cd62 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.001402  2067 tablet_replica.cc:333] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.002170  2067 raft_consensus.cc:2243] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.003130  2067 raft_consensus.cc:2272] T 4ef7a99e80494a989eda0b1a37482656 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.005522  2067 tablet_replica.cc:333] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.006246  2067 raft_consensus.cc:2243] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.007292  2067 raft_consensus.cc:2272] T 0006f949032c4034b4f4872ee332386e P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.010010  2067 tablet_replica.cc:333] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.011227  2067 raft_consensus.cc:2243] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:41.012761  2067 raft_consensus.cc:2272] T d04ce59a08f3427581e5e6b5d87ad757 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.017156  2067 tablet_replica.cc:333] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.018333  2067 raft_consensus.cc:2243] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.019675  2067 raft_consensus.cc:2272] T 256f8c690fb2456f92b4035fc3bf5e23 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.023173  2067 tablet_replica.cc:333] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.024217  2067 raft_consensus.cc:2243] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.025832  2067 raft_consensus.cc:2272] T d6803fb2b32049f3935eeb6331dcde39 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.028877  2067 tablet_replica.cc:333] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.029876  2067 raft_consensus.cc:2243] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.030843  2067 raft_consensus.cc:2272] T c0e0b097c72c4c748e146f8303c90f0c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.033288  2067 tablet_replica.cc:333] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.034209  2067 raft_consensus.cc:2243] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:41.034848  2067 raft_consensus.cc:2272] T 52af756fdd9b411c8c4b494e2552a3fb P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.037556  2067 tablet_replica.cc:333] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.038545  2067 raft_consensus.cc:2243] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:41.039149  2067 raft_consensus.cc:2272] T cfa3fcaa83734d509f6b31c3874873be P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.042114  2067 tablet_replica.cc:333] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.042905  2067 raft_consensus.cc:2243] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.044178  2067 raft_consensus.cc:2272] T aa54a35cf4c54bb1a1c28658c11d174c P 7eabb96dfdd1449c94ad1261ded46c3c [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.046962  2067 tablet_replica.cc:333] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.047897  2067 raft_consensus.cc:2243] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.048920  2067 raft_consensus.cc:2272] T 6ed8b895df664a3b8f7934c37fd8f181 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.052161  2067 tablet_replica.cc:333] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
W20260709 12:28:41.052879  5990 consensus_peers.cc:597] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:41.052956  2067 raft_consensus.cc:2243] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.054390  2067 raft_consensus.cc:2272] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:41.055070  5915 consensus_peers.cc:597] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:41.056661  2067 tablet_replica.cc:333] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.057435  2067 raft_consensus.cc:2243] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:41.058043  2067 raft_consensus.cc:2272] T 860d4f6b513041829ed9cd83880edc5c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.060201  2067 tablet_replica.cc:333] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.060837  2067 raft_consensus.cc:2243] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:41.061609  2067 raft_consensus.cc:2272] T 7d95abc0e3a9452eb13b2625af34008e P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.064836  2067 tablet_replica.cc:333] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.065719  2067 raft_consensus.cc:2243] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.066991  2067 raft_consensus.cc:2272] T f3c3647dbbb74fbd86870b12d1409976 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:41.068717  5990 consensus_peers.cc:597] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:41.069968  2067 tablet_replica.cc:333] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.070755  2067 raft_consensus.cc:2243] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.071841  2067 raft_consensus.cc:2272] T d4765e1fc00c485086dc883bea4b87f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:41.072700  5915 consensus_peers.cc:597] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:28:41.074173  2067 tablet_replica.cc:333] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.075493  2067 raft_consensus.cc:2243] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:41.076206  2067 raft_consensus.cc:2272] T 30da8f1ded30432a87f01684d88a9969 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.079047  2067 tablet_replica.cc:333] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.079710  2067 raft_consensus.cc:2243] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.080961  2067 raft_consensus.cc:2272] T d001526e46b24f429f57c91e4a270036 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.083729  2067 tablet_replica.cc:333] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.084357  2067 raft_consensus.cc:2243] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.085315  2067 raft_consensus.cc:2272] T 02e8ec653e484ba58498cc78f412de75 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.088052  2067 tablet_replica.cc:333] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.088771  2067 raft_consensus.cc:2243] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.090296  2067 raft_consensus.cc:2272] T 6ad5821a38aa41d7abd76509a12a97df P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.092839  2067 tablet_replica.cc:333] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.093436  2067 raft_consensus.cc:2243] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.094671  2067 raft_consensus.cc:2272] T 85d9b48bdd4046d788b310a73871c31c P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.097501  2067 tablet_replica.cc:333] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.098397  2067 raft_consensus.cc:2243] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.099627  2067 raft_consensus.cc:2272] T 4fa58bdc1b834feda10d16a00cd81355 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:41.100098  5987 consensus_peers.cc:597] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 -> Peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Couldn't send request to peer 7eabb96dfdd1449c94ad1261ded46c3c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:41.102154  2067 tablet_replica.cc:333] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.103035  2067 raft_consensus.cc:2243] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.104561  2067 raft_consensus.cc:2272] T edbd0e138da244d48f133d23b4aa1eda P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.107424  2067 tablet_replica.cc:333] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.108157  2067 raft_consensus.cc:2243] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.109241  2067 raft_consensus.cc:2272] T 59cab2d7e5234efd9a97b8377f1919f7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:41.110535  5987 consensus_peers.cc:597] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 -> Peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Couldn't send request to peer 7eabb96dfdd1449c94ad1261ded46c3c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:41.111733  2067 tablet_replica.cc:333] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.112332  2067 raft_consensus.cc:2243] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.113226  2067 raft_consensus.cc:2272] T ba269f0998844ab5b461587eaa90df32 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.115387  2067 tablet_replica.cc:333] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.116185  2067 raft_consensus.cc:2243] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:41.116801  2067 raft_consensus.cc:2272] T 45edb6642b84468aa6fc19a7fd2ddf6c P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.119671  2067 tablet_replica.cc:333] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
W20260709 12:28:41.119925  5987 consensus_peers.cc:597] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 -> Peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Couldn't send request to peer 7eabb96dfdd1449c94ad1261ded46c3c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:41.120362  2067 raft_consensus.cc:2243] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:41.121002  2067 raft_consensus.cc:2272] T 07d59a4ebda543dfbfa20847a8e9e4a7 P 7eabb96dfdd1449c94ad1261ded46c3c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.123178  2067 tablet_replica.cc:333] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.123773  2067 raft_consensus.cc:2243] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.124706  2067 raft_consensus.cc:2272] T 60934281a651439a87cd8c11702f67d2 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.127398  2067 tablet_replica.cc:333] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.128340  2067 raft_consensus.cc:2243] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.129487  2067 raft_consensus.cc:2272] T 2dcbedb48e03420dbf7f8646e65b1886 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.131819  2067 tablet_replica.cc:333] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.132370  2067 raft_consensus.cc:2243] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.133347  2067 raft_consensus.cc:2272] T 1ab14137002240c2909503f8b4e55c79 P 7eabb96dfdd1449c94ad1261ded46c3c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.135483  2067 tablet_replica.cc:333] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.136065  2067 raft_consensus.cc:2243] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.137315  2067 raft_consensus.cc:2272] T 75e8568a9892438c91e30ddc62fef67c P 7eabb96dfdd1449c94ad1261ded46c3c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.139807  2067 tablet_replica.cc:333] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.140362  2067 raft_consensus.cc:2243] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.141215  2067 raft_consensus.cc:2272] T 187e00b075464d9c855ce2b87615ac70 P 7eabb96dfdd1449c94ad1261ded46c3c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.143289  2067 tablet_replica.cc:333] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.143806  2067 raft_consensus.cc:2243] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.144811  2067 raft_consensus.cc:2272] T 238b79ac4a244a85a83ab0d01b84e117 P 7eabb96dfdd1449c94ad1261ded46c3c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.146901  2067 tablet_replica.cc:333] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.147373  2067 raft_consensus.cc:2243] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:41.147780  2067 raft_consensus.cc:2272] T 681f64c726104caf875ae7d4329d0676 P 7eabb96dfdd1449c94ad1261ded46c3c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.149819  2067 tablet_replica.cc:333] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.150306  2067 raft_consensus.cc:2243] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:41.150736  2067 raft_consensus.cc:2272] T 7a3145d5d96e41518c7bf41999289ac3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.152766  2067 tablet_replica.cc:333] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.153294  2067 raft_consensus.cc:2243] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.154173  2067 raft_consensus.cc:2272] T 8e301d9d5b914f3b9ec07dea273d5126 P 7eabb96dfdd1449c94ad1261ded46c3c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.156172  2067 tablet_replica.cc:333] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.156890  2067 raft_consensus.cc:2243] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:41.157485  2067 raft_consensus.cc:2272] T 41bc2065e43e40c98ee821925758a3f3 P 7eabb96dfdd1449c94ad1261ded46c3c [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.160060  2067 tablet_replica.cc:333] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.160749  2067 raft_consensus.cc:2243] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:41.161574  2067 raft_consensus.cc:2272] T a80a3917fd9748bc8ae4817757081f09 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.164019  2067 tablet_replica.cc:333] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.164601  2067 raft_consensus.cc:2243] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:41.165148  2067 raft_consensus.cc:2272] T 18a324a652924631995e78ee3fda8422 P 7eabb96dfdd1449c94ad1261ded46c3c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:41.167631  2067 tablet_replica.cc:333] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c: stopping tablet replica
I20260709 12:28:41.168190  2067 raft_consensus.cc:2243] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:41.169142  2067 raft_consensus.cc:2272] T ebb084ec08f44cb2a935c184afafadb0 P 7eabb96dfdd1449c94ad1261ded46c3c [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:41.189047  5990 consensus_peers.cc:597] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:28:41.189935  5987 consensus_peers.cc:597] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 -> Peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Couldn't send request to peer 7eabb96dfdd1449c94ad1261ded46c3c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:41.397691 14181 raft_consensus.cc:493] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7a0204b931394fc48ffc503716e0d111)
I20260709 12:28:41.398322 14181 raft_consensus.cc:515] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } } peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } }
I20260709 12:28:41.400288 14181 leader_election.cc:290] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701), 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005)
W20260709 12:28:41.404277  5990 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
W20260709 12:28:41.404840  5987 leader_election.cc:336] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:41.405230  5987 leader_election.cc:304] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7a0204b931394fc48ffc503716e0d111
I20260709 12:28:41.405902 14181 raft_consensus.cc:2749] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:41.485208  5686 maintenance_manager.cc:419] P 03f70467f9c34de887520aec76068d6f: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033395
I20260709 12:28:41.571970  5685 maintenance_manager.cc:643] P 03f70467f9c34de887520aec76068d6f: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.084s	user 0.032s	sys 0.014s Metrics: {"bytes_written":6572,"delete_count":0,"lbm_write_time_us":1219,"lbm_writes_lt_1ms":6,"reinsert_count":0,"thread_start_us":736,"threads_started":1,"update_count":275,"wal-append.queue_time_us":1130}
I20260709 12:28:41.574913  5686 maintenance_manager.cc:419] P 03f70467f9c34de887520aec76068d6f: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.290129
W20260709 12:28:41.578949  6045 debug-util.cc:398] Leaking SignalData structure 0x7b080027a680 after lost signal to thread 2070
W20260709 12:28:41.581511  6045 debug-util.cc:398] Leaking SignalData structure 0x7b08002b2040 after lost signal to thread 5731
W20260709 12:28:41.584602  6045 debug-util.cc:398] Leaking SignalData structure 0x7b08002b7de0 after lost signal to thread 5973
W20260709 12:28:41.586725  6045 debug-util.cc:398] Leaking SignalData structure 0x7b080024d520 after lost signal to thread 6048
W20260709 12:28:41.632393  5990 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:28:41.667826  5685 maintenance_manager.cc:643] P 03f70467f9c34de887520aec76068d6f: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.092s	user 0.065s	sys 0.025s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":26666,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":2346,"dirs.run_cpu_time_us":743,"dirs.run_wall_time_us":1479,"lbm_read_time_us":1903,"lbm_reads_lt_1ms":23,"lbm_write_time_us":1508,"lbm_writes_lt_1ms":11,"mutex_wait_us":105,"peak_mem_usage":296496,"reinsert_count":0,"thread_start_us":2586,"threads_started":2,"update_count":275}
W20260709 12:28:41.718156  5990 consensus_peers.cc:597] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:28:41.841040  5987 consensus_peers.cc:597] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:41.842801  2067 tablet_server.cc:196] TabletServer@127.2.4.195:0 shutdown complete.
W20260709 12:28:41.876271  5990 consensus_peers.cc:597] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 -> Peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Couldn't send request to peer 1fc57407d56241b38a94d543e073f8af. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:41.915606  5987 consensus_peers.cc:597] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:41.916389  5987 consensus_peers.cc:597] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:42.045957  2067 tablet_server.cc:179] TabletServer@127.2.4.196:0 shutting down...
W20260709 12:28:42.089188  5987 consensus_peers.cc:597] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:42.093868 14175 raft_consensus.cc:493] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:42.098868 14175 raft_consensus.cc:515] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:42.104590 14175 leader_election.cc:290] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:42.109308 14186 raft_consensus.cc:493] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:42.110347 14187 raft_consensus.cc:493] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:42.110337 14186 raft_consensus.cc:515] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:42.111123 14187 raft_consensus.cc:515] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:42.112847 14186 leader_election.cc:290] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:42.113272 14187 leader_election.cc:290] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:42.113453 14186 raft_consensus.cc:493] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:42.114092 14186 raft_consensus.cc:515] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:42.116385 14186 leader_election.cc:290] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:42.116892 14186 raft_consensus.cc:493] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:42.117449 14186 raft_consensus.cc:515] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
W20260709 12:28:42.118168  5987 leader_election.cc:336] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Network error: Client connection negotiation failed: client connection to 127.2.4.195:43329: connect: Connection refused (error 111)
W20260709 12:28:42.121335  5987 leader_election.cc:336] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Network error: Client connection negotiation failed: client connection to 127.2.4.195:43329: connect: Connection refused (error 111)
W20260709 12:28:42.122010  5987 leader_election.cc:336] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Network error: Client connection negotiation failed: client connection to 127.2.4.195:43329: connect: Connection refused (error 111)
W20260709 12:28:42.122555  5987 leader_election.cc:336] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Network error: Client connection negotiation failed: client connection to 127.2.4.195:43329: connect: Connection refused (error 111)
W20260709 12:28:42.124934  5987 leader_election.cc:336] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:42.125423  5987 leader_election.cc:304] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:42.126380 14175 raft_consensus.cc:2749] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:42.127429  5987 leader_election.cc:336] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
I20260709 12:28:42.127878  5987 leader_election.cc:304] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:42.127955 14186 leader_election.cc:290] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
I20260709 12:28:42.128927 14186 raft_consensus.cc:2749] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:42.134828 14186 raft_consensus.cc:493] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:42.135708 14186 raft_consensus.cc:515] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:42.138617 14186 leader_election.cc:290] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
W20260709 12:28:42.145015  5987 leader_election.cc:336] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:42.145957  5987 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111) [suppressed 532 similar messages]
W20260709 12:28:42.147423  5987 leader_election.cc:336] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Network error: Client connection negotiation failed: client connection to 127.2.4.195:43329: connect: Connection refused (error 111)
I20260709 12:28:42.148025  5987 leader_election.cc:304] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
W20260709 12:28:42.148690  5990 leader_election.cc:336] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:42.149260 14192 raft_consensus.cc:2749] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:42.149529  5990 leader_election.cc:304] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:42.150027  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:42.150888 14192 raft_consensus.cc:2749] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:42.151595  2067 tablet_replica.cc:333] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.153357  2067 raft_consensus.cc:2243] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.155046  2067 raft_consensus.cc:2272] T 99b9212afb8b459bb96908f8259bb8c0 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.155411 14192 raft_consensus.cc:493] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
W20260709 12:28:42.156533  5987 leader_election.cc:336] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Network error: Client connection negotiation failed: client connection to 127.2.4.195:43329: connect: Connection refused (error 111)
I20260709 12:28:42.156349 14192 raft_consensus.cc:515] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:42.157617 14186 raft_consensus.cc:493] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:42.158389 14186 raft_consensus.cc:515] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a0204b931394fc48ffc503716e0d111" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 38005 } } peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } }
I20260709 12:28:42.159408 14192 leader_election.cc:290] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:42.160072  2067 tablet_replica.cc:333] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.160429 14192 raft_consensus.cc:493] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7eabb96dfdd1449c94ad1261ded46c3c)
I20260709 12:28:42.161110 14192 raft_consensus.cc:515] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eabb96dfdd1449c94ad1261ded46c3c" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 43329 } } peers { permanent_uuid: "6add331326ca47038308a447bf798fa7" member_type: VOTER last_known_addr { host: "127.2.4.196" port: 34835 } } peers { permanent_uuid: "1fc57407d56241b38a94d543e073f8af" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 38701 } }
I20260709 12:28:42.161823  2067 raft_consensus.cc:2243] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:28:42.163837 14192 leader_election.cc:290] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329), 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701)
I20260709 12:28:42.164124  2067 raft_consensus.cc:2272] T 595c6563cdd74f47bda68ead57106a8b P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.161105 14186 leader_election.cc:290] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005), 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329)
W20260709 12:28:42.171803  5987 leader_election.cc:336] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Network error: Client connection negotiation failed: client connection to 127.2.4.195:43329: connect: Connection refused (error 111)
W20260709 12:28:42.172556  5987 leader_election.cc:336] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Network error: Client connection negotiation failed: client connection to 127.2.4.195:43329: connect: Connection refused (error 111)
W20260709 12:28:42.173148  5987 leader_election.cc:336] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7eabb96dfdd1449c94ad1261ded46c3c (127.2.4.195:43329): Network error: Client connection negotiation failed: client connection to 127.2.4.195:43329: connect: Connection refused (error 111)
W20260709 12:28:42.173991  5987 leader_election.cc:336] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111)
W20260709 12:28:42.174290  5990 leader_election.cc:336] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:42.174656  2067 tablet_replica.cc:333] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.174474  5987 leader_election.cc:304] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 7a0204b931394fc48ffc503716e0d111, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:42.175217  5990 leader_election.cc:304] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:42.175968 14192 raft_consensus.cc:2749] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:42.176213 14186 raft_consensus.cc:2749] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:42.176458  2067 raft_consensus.cc:2243] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:42.177783  5990 leader_election.cc:336] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:42.177942  2067 raft_consensus.cc:2272] T 0006f949032c4034b4f4872ee332386e P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.181027  5990 leader_election.cc:304] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:42.184163 14186 raft_consensus.cc:2749] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:42.185371  2067 tablet_replica.cc:333] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.186591  2067 raft_consensus.cc:2243] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.187470  2067 raft_consensus.cc:2272] T 8d0c096d28fc46d3be987a091fcb5533 P 6add331326ca47038308a447bf798fa7 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:42.188515  5990 leader_election.cc:336] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:42.189248  5990 leader_election.cc:304] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:42.190336 14186 raft_consensus.cc:2749] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:42.190608  5990 leader_election.cc:336] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1fc57407d56241b38a94d543e073f8af (127.2.4.193:38701): Network error: Client connection negotiation failed: client connection to 127.2.4.193:38701: connect: Connection refused (error 111)
I20260709 12:28:42.191311  5990 leader_election.cc:304] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6add331326ca47038308a447bf798fa7; no voters: 1fc57407d56241b38a94d543e073f8af, 7eabb96dfdd1449c94ad1261ded46c3c
I20260709 12:28:42.191677  2067 tablet_replica.cc:333] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.192283 14186 raft_consensus.cc:2749] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:42.193015  2067 raft_consensus.cc:2243] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.194597  2067 raft_consensus.cc:2272] T 48c0a9da01454a2794160d9c0ff08ea9 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.197530  2067 tablet_replica.cc:333] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.198421  2067 raft_consensus.cc:2243] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.199353  2067 raft_consensus.cc:2272] T ba269f0998844ab5b461587eaa90df32 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.202416  2067 tablet_replica.cc:333] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.203127  2067 raft_consensus.cc:2243] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.204093  2067 raft_consensus.cc:2272] T 428a50885681465a98089cb853b233b9 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.207154  2067 tablet_replica.cc:333] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.207899  2067 raft_consensus.cc:2243] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.208791  2067 raft_consensus.cc:2272] T fee70b2d75314f788ad873b1950fb890 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.211848  2067 tablet_replica.cc:333] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.212803  2067 raft_consensus.cc:2243] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.214030  2067 raft_consensus.cc:2272] T d6803fb2b32049f3935eeb6331dcde39 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.217842  2067 tablet_replica.cc:333] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.218592  2067 raft_consensus.cc:2243] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.219596  2067 raft_consensus.cc:2272] T c0e0b097c72c4c748e146f8303c90f0c P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.222291  2067 tablet_replica.cc:333] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.222985  2067 raft_consensus.cc:2243] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:28:42.224671  2067 raft_consensus.cc:2272] T bc2de53e733442a69e97d39affab60ba P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.228183  2067 tablet_replica.cc:333] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.228885  2067 raft_consensus.cc:2243] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.230360  2067 raft_consensus.cc:2272] T c3c69fa4d49547f8a9e76c055c3ae7be P 6add331326ca47038308a447bf798fa7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.233078  2067 tablet_replica.cc:333] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.233871  2067 raft_consensus.cc:2243] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.234612  2067 raft_consensus.cc:2272] T aa54a35cf4c54bb1a1c28658c11d174c P 6add331326ca47038308a447bf798fa7 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.237437  2067 tablet_replica.cc:333] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.238195  2067 raft_consensus.cc:2243] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:28:42.239926  2067 raft_consensus.cc:2272] T e657475654794984a13c42a668f3158b P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.242667  2067 tablet_replica.cc:333] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.243321  2067 raft_consensus.cc:2243] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.244046  2067 raft_consensus.cc:2272] T d4765e1fc00c485086dc883bea4b87f7 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.246631  2067 tablet_replica.cc:333] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.247330  2067 raft_consensus.cc:2243] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:42.248699  2067 raft_consensus.cc:2272] T 1ba7af1a4a744df6bc8c20ec72e21519 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.251353  2067 tablet_replica.cc:333] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.251987  2067 raft_consensus.cc:2243] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:42.253125  2067 raft_consensus.cc:2272] T 7d95abc0e3a9452eb13b2625af34008e P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.255999  2067 tablet_replica.cc:333] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.256629  2067 raft_consensus.cc:2243] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.257503  2067 raft_consensus.cc:2272] T 02e8ec653e484ba58498cc78f412de75 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.260064  2067 tablet_replica.cc:333] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.260735  2067 raft_consensus.cc:2243] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.261368  2067 raft_consensus.cc:2272] T 6ad5821a38aa41d7abd76509a12a97df P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.264070  2067 tablet_replica.cc:333] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.264725  2067 raft_consensus.cc:2243] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.265540  2067 raft_consensus.cc:2272] T e55fd31b2e414885a9bd5c9b40e04e19 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.267902  2067 tablet_replica.cc:333] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.268536  2067 raft_consensus.cc:2243] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:42.270596  2067 raft_consensus.cc:2272] T 402bd52b71ce4044b0ec86bf6f8d73fc P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.273226  2067 tablet_replica.cc:333] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.273900  2067 raft_consensus.cc:2243] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:42.275123  2067 raft_consensus.cc:2272] T 681f64c726104caf875ae7d4329d0676 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.277734  2067 tablet_replica.cc:333] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.278366  2067 raft_consensus.cc:2243] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.279621  2067 raft_consensus.cc:2272] T 238b79ac4a244a85a83ab0d01b84e117 P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.282330  2067 tablet_replica.cc:333] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.284505  2067 raft_consensus.cc:2243] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.285074  2067 raft_consensus.cc:2272] T 256f8c690fb2456f92b4035fc3bf5e23 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.287520  2067 tablet_replica.cc:333] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.288160  2067 raft_consensus.cc:2243] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:28:42.289773  2067 raft_consensus.cc:2272] T e9bb01883059454e80b19d5333aad71e P 6add331326ca47038308a447bf798fa7 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.293282  2067 tablet_replica.cc:333] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.294234  2067 raft_consensus.cc:2243] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.295639  2067 raft_consensus.cc:2272] T 6ed8b895df664a3b8f7934c37fd8f181 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:42.295506  5987 consensus_peers.cc:597] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 -> Peer 7a0204b931394fc48ffc503716e0d111 (127.2.4.194:38005): Couldn't send request to peer 7a0204b931394fc48ffc503716e0d111. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:38005: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:42.299032  2067 tablet_replica.cc:333] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.299806  2067 raft_consensus.cc:2243] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.300813  2067 raft_consensus.cc:2272] T 9a7ca03d5d3d4b13bee5481a90dc7bdc P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.304117  2067 tablet_replica.cc:333] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.305125  2067 raft_consensus.cc:2243] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.306352  2067 raft_consensus.cc:2272] T 85d9b48bdd4046d788b310a73871c31c P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.310124  2067 tablet_replica.cc:333] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.310774  2067 raft_consensus.cc:2243] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.311499  2067 raft_consensus.cc:2272] T 4fa58bdc1b834feda10d16a00cd81355 P 6add331326ca47038308a447bf798fa7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.314276  2067 tablet_replica.cc:333] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.314942  2067 raft_consensus.cc:2243] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.315749  2067 raft_consensus.cc:2272] T 685933c4b2364e1c90d92edb4c091799 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.325604  2067 tablet_replica.cc:333] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.326475  2067 raft_consensus.cc:2243] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:42.327662  2067 raft_consensus.cc:2272] T 30da8f1ded30432a87f01684d88a9969 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.331472  2067 tablet_replica.cc:333] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.332146  2067 raft_consensus.cc:2243] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.332806  2067 raft_consensus.cc:2272] T 60934281a651439a87cd8c11702f67d2 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.335621  2067 tablet_replica.cc:333] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.336220  2067 raft_consensus.cc:2243] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.337015  2067 raft_consensus.cc:2272] T 2dcbedb48e03420dbf7f8646e65b1886 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.339586  2067 tablet_replica.cc:333] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.340291  2067 raft_consensus.cc:2243] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.341085  2067 raft_consensus.cc:2272] T 1ab14137002240c2909503f8b4e55c79 P 6add331326ca47038308a447bf798fa7 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.343947  2067 tablet_replica.cc:333] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.344587  2067 raft_consensus.cc:2243] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.345279  2067 raft_consensus.cc:2272] T 75e8568a9892438c91e30ddc62fef67c P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.348228  2067 tablet_replica.cc:333] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.348872  2067 raft_consensus.cc:2243] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:42.350291  2067 raft_consensus.cc:2272] T 048b40ab425f451c983713b7df96c732 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.352988  2067 tablet_replica.cc:333] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.353615  2067 raft_consensus.cc:2243] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.355058  2067 raft_consensus.cc:2272] T 187e00b075464d9c855ce2b87615ac70 P 6add331326ca47038308a447bf798fa7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.358297  2067 tablet_replica.cc:333] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.359035  2067 raft_consensus.cc:2243] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.359865  2067 raft_consensus.cc:2272] T 8e301d9d5b914f3b9ec07dea273d5126 P 6add331326ca47038308a447bf798fa7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.363248  2067 tablet_replica.cc:333] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.364126  2067 raft_consensus.cc:2243] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:42.365532  2067 raft_consensus.cc:2272] T 41bc2065e43e40c98ee821925758a3f3 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.368353  2067 tablet_replica.cc:333] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.368971  2067 raft_consensus.cc:2243] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:42.370299  2067 raft_consensus.cc:2272] T a80a3917fd9748bc8ae4817757081f09 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.372901  2067 tablet_replica.cc:333] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.373574  2067 raft_consensus.cc:2243] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.374338  2067 raft_consensus.cc:2272] T 926e6a2e59d94f64ba48bae2ca5f3f55 P 6add331326ca47038308a447bf798fa7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.377404  2067 tablet_replica.cc:333] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.378033  2067 raft_consensus.cc:2243] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.379048  2067 raft_consensus.cc:2272] T ebb084ec08f44cb2a935c184afafadb0 P 6add331326ca47038308a447bf798fa7 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.381948  2067 tablet_replica.cc:333] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.382472  2067 raft_consensus.cc:2243] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:28:42.383836  2067 raft_consensus.cc:2272] T e42697dffe5a495ea1ba4a111df5ba39 P 6add331326ca47038308a447bf798fa7 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.386416  2067 tablet_replica.cc:333] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7: stopping tablet replica
I20260709 12:28:42.386916  2067 raft_consensus.cc:2243] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:42.387523  2067 raft_consensus.cc:2272] T 9bede7b5a3c84fe28a073542219f3fe4 P 6add331326ca47038308a447bf798fa7 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:42.976922  2067 tablet_server.cc:196] TabletServer@127.2.4.196:0 shutdown complete.
I20260709 12:28:43.066037  2067 master.cc:562] Master@127.2.4.254:41049 shutting down...
I20260709 12:28:43.106240  2067 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:43.106997  2067 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:43.107303  2067 tablet_replica.cc:333] T 00000000000000000000000000000000 P 03f70467f9c34de887520aec76068d6f: stopping tablet replica
I20260709 12:28:43.223409  2067 master.cc:584] Master@127.2.4.254:41049 shutdown complete.
I20260709 12:28:43.274560  2067 test_util.cc:182] -----------------------------------------------
I20260709 12:28:43.274808  2067 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599727094521-2067-0
[  FAILED  ] LeaderRebalancerTest.RestartTserver (251566 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:28:43.278409  2067 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.4.254:35633
I20260709 12:28:43.279564  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:43.286679 14193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:43.287076 14194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:43.287688 14196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:43.289345  2067 server_base.cc:1061] running on GCE node
I20260709 12:28:43.294173  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:43.294435  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:43.294556  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600123294545 us; error 0 us; skew 500 ppm
I20260709 12:28:43.295127  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:43.298398  2067 webserver.cc:533] Webserver started at http://127.2.4.254:36531/ using document root <none> and password file <none>
I20260709 12:28:43.298983  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:43.299226  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:43.299561  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:43.301110  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/master-0-root/instance:
uuid: "bee45cca01884435ac7cf61afe80bed3"
format_stamp: "Formatted at 2026-07-09 12:28:43 on dist-test-slave-4qvp"
I20260709 12:28:43.306917  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.004s
I20260709 12:28:43.311319 14201 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:43.312475  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:28:43.312945  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/master-0-root
uuid: "bee45cca01884435ac7cf61afe80bed3"
format_stamp: "Formatted at 2026-07-09 12:28:43 on dist-test-slave-4qvp"
I20260709 12:28:43.313328  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:43.344529  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:43.346005  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:43.393476  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.254:35633
I20260709 12:28:43.393764 14252 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.254:35633 every 8 connection(s)
I20260709 12:28:43.398080 14253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.411481 14253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3: Bootstrap starting.
I20260709 12:28:43.417893 14253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.423413 14253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3: No bootstrap required, opened a new log
I20260709 12:28:43.426443 14253 raft_consensus.cc:359] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee45cca01884435ac7cf61afe80bed3" member_type: VOTER }
I20260709 12:28:43.426959 14253 raft_consensus.cc:385] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.427215 14253 raft_consensus.cc:740] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bee45cca01884435ac7cf61afe80bed3, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.427958 14253 consensus_queue.cc:260] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee45cca01884435ac7cf61afe80bed3" member_type: VOTER }
I20260709 12:28:43.428521 14253 raft_consensus.cc:399] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:28:43.428774 14253 raft_consensus.cc:493] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:28:43.429014 14253 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:43.435497 14253 raft_consensus.cc:515] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee45cca01884435ac7cf61afe80bed3" member_type: VOTER }
I20260709 12:28:43.436106 14253 leader_election.cc:304] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bee45cca01884435ac7cf61afe80bed3; no voters: 
I20260709 12:28:43.437491 14253 leader_election.cc:290] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:28:43.438205 14256 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:43.439783 14256 raft_consensus.cc:697] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [term 1 LEADER]: Becoming Leader. State: Replica: bee45cca01884435ac7cf61afe80bed3, State: Running, Role: LEADER
I20260709 12:28:43.440598 14256 consensus_queue.cc:237] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee45cca01884435ac7cf61afe80bed3" member_type: VOTER }
I20260709 12:28:43.441308 14253 sys_catalog.cc:565] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:28:43.447706 14258 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bee45cca01884435ac7cf61afe80bed3. Latest consensus state: current_term: 1 leader_uuid: "bee45cca01884435ac7cf61afe80bed3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee45cca01884435ac7cf61afe80bed3" member_type: VOTER } }
I20260709 12:28:43.448083 14257 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bee45cca01884435ac7cf61afe80bed3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee45cca01884435ac7cf61afe80bed3" member_type: VOTER } }
I20260709 12:28:43.448578 14258 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:43.448647 14257 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:43.451032 14263 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:28:43.456912 14263 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:28:43.461395  2067 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:28:43.467885 14263 catalog_manager.cc:1383] Generated new cluster ID: 37d37505a34f4cbfbc39f4f87119b6c4
I20260709 12:28:43.468292 14263 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:28:43.486582 14263 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:28:43.488178 14263 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:28:43.509212 14263 catalog_manager.cc:6070] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3: Generated new TSK 0
I20260709 12:28:43.510129 14263 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:28:43.529125  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:43.536300 14274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:43.537417 14275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:43.540329 14277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:43.541252  2067 server_base.cc:1061] running on GCE node
I20260709 12:28:43.543953  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:43.544183  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:43.544418  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600123544375 us; error 0 us; skew 500 ppm
I20260709 12:28:43.545039  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:43.547974  2067 webserver.cc:533] Webserver started at http://127.2.4.193:41745/ using document root <none> and password file <none>
I20260709 12:28:43.548545  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:43.548789  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:43.549103  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:43.550493  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-0-root/instance:
uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
format_stamp: "Formatted at 2026-07-09 12:28:43 on dist-test-slave-4qvp"
I20260709 12:28:43.556026  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:28:43.560633 14282 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:43.561844  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:28:43.562269  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-0-root
uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
format_stamp: "Formatted at 2026-07-09 12:28:43 on dist-test-slave-4qvp"
I20260709 12:28:43.562750  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:43.581899  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:43.583348  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:43.585120  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:43.588686  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:43.588904  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:43.589228  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:43.589445  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:43.643399  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.193:43175
I20260709 12:28:43.643630 14344 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.193:43175 every 8 connection(s)
I20260709 12:28:43.649652  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:43.659577 14349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:43.664031 14350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:43.665098  2067 server_base.cc:1061] running on GCE node
W20260709 12:28:43.666965 14352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:43.668498  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:43.668846  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:43.669071  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600123669053 us; error 0 us; skew 500 ppm
I20260709 12:28:43.669917  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:43.673406  2067 webserver.cc:533] Webserver started at http://127.2.4.194:37509/ using document root <none> and password file <none>
I20260709 12:28:43.674151  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:43.674453  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:43.674713 14345 heartbeater.cc:344] Connected to a master server at 127.2.4.254:35633
I20260709 12:28:43.674880  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:43.675292 14345 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:43.676374 14345 heartbeater.cc:507] Master 127.2.4.254:35633 requested a full tablet report, sending...
I20260709 12:28:43.676702  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-1-root/instance:
uuid: "a153a1160e9240439004094da9aa01ab"
format_stamp: "Formatted at 2026-07-09 12:28:43 on dist-test-slave-4qvp"
I20260709 12:28:43.679579 14218 ts_manager.cc:194] Registered new tserver with Master: e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175)
I20260709 12:28:43.682396 14218 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52450
I20260709 12:28:43.683097  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:28:43.688516 14357 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:43.689618  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:28:43.690263  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-1-root
uuid: "a153a1160e9240439004094da9aa01ab"
format_stamp: "Formatted at 2026-07-09 12:28:43 on dist-test-slave-4qvp"
I20260709 12:28:43.690707  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:43.702943  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:43.704226  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:43.705853  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:43.708395  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:43.708659  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:43.708925  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:43.709102  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:43.763077  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.194:36325
I20260709 12:28:43.763263 14419 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.194:36325 every 8 connection(s)
I20260709 12:28:43.770356  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:43.782152 14423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:43.782178 14424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:43.794077 14426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:43.796110  2067 server_base.cc:1061] running on GCE node
I20260709 12:28:43.797358  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:28:43.797679 14420 heartbeater.cc:344] Connected to a master server at 127.2.4.254:35633
W20260709 12:28:43.797709  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:43.798274  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600123798251 us; error 0 us; skew 500 ppm
I20260709 12:28:43.798341 14420 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:43.799005  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:43.799722 14420 heartbeater.cc:507] Master 127.2.4.254:35633 requested a full tablet report, sending...
I20260709 12:28:43.803591  2067 webserver.cc:533] Webserver started at http://127.2.4.195:33731/ using document root <none> and password file <none>
I20260709 12:28:43.804440  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:43.804373 14218 ts_manager.cc:194] Registered new tserver with Master: a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:43.805087  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:43.805457  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:43.806717  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-2-root/instance:
uuid: "abe2b40322a94e769d80927662234903"
format_stamp: "Formatted at 2026-07-09 12:28:43 on dist-test-slave-4qvp"
I20260709 12:28:43.807192 14218 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52466
I20260709 12:28:43.813071  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:28:43.818588 14431 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:43.819957  2067 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:28:43.820290  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-2-root
uuid: "abe2b40322a94e769d80927662234903"
format_stamp: "Formatted at 2026-07-09 12:28:43 on dist-test-slave-4qvp"
I20260709 12:28:43.820600  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:43.840341  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:43.841953  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:43.844575  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:43.848533  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:43.849011  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260709 12:28:43.849354  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:43.849567  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:43.928601  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.195:37025
I20260709 12:28:43.928670 14493 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.195:37025 every 8 connection(s)
I20260709 12:28:43.947386 14494 heartbeater.cc:344] Connected to a master server at 127.2.4.254:35633
I20260709 12:28:43.948058 14494 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:43.949273 14494 heartbeater.cc:507] Master 127.2.4.254:35633 requested a full tablet report, sending...
I20260709 12:28:43.953658 14218 ts_manager.cc:194] Registered new tserver with Master: abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:43.953887  2067 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.02040568s
I20260709 12:28:43.955996 14218 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52468
I20260709 12:28:43.988039 14218 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52482:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:43.991727 14218 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:44.075651 14385 tablet_service.cc:1511] Processing CreateTablet for tablet d6dcc07ba9784ae98d3974c6abb1b53b (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:28:44.077593 14385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6dcc07ba9784ae98d3974c6abb1b53b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.088707 14459 tablet_service.cc:1511] Processing CreateTablet for tablet d6dcc07ba9784ae98d3974c6abb1b53b (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:28:44.088812 14456 tablet_service.cc:1511] Processing CreateTablet for tablet 7aeb6f75770b4fd5bf7692ba6e2ffa92 (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:28:44.090267 14459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6dcc07ba9784ae98d3974c6abb1b53b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.090629 14383 tablet_service.cc:1511] Processing CreateTablet for tablet 354b6360e1104e169aaee9531ae2c3a6 (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:28:44.091797 14456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7aeb6f75770b4fd5bf7692ba6e2ffa92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.092167 14383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 354b6360e1104e169aaee9531ae2c3a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.095016 14382 tablet_service.cc:1511] Processing CreateTablet for tablet 7aeb6f75770b4fd5bf7692ba6e2ffa92 (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:28:44.088418 14384 tablet_service.cc:1511] Processing CreateTablet for tablet 625d6ab82ccb4bfca8333bbf41e4ce0f (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:28:44.096302 14457 tablet_service.cc:1511] Processing CreateTablet for tablet 354b6360e1104e169aaee9531ae2c3a6 (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:28:44.095016 14458 tablet_service.cc:1511] Processing CreateTablet for tablet 625d6ab82ccb4bfca8333bbf41e4ce0f (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:28:44.097903 14457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 354b6360e1104e169aaee9531ae2c3a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.099110 14458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 625d6ab82ccb4bfca8333bbf41e4ce0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.101202 14455 tablet_service.cc:1511] Processing CreateTablet for tablet 527f315df63447e3bc065eb01506adec (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:28:44.102877 14455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 527f315df63447e3bc065eb01506adec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.106945 14381 tablet_service.cc:1511] Processing CreateTablet for tablet 527f315df63447e3bc065eb01506adec (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:28:44.106945 14454 tablet_service.cc:1511] Processing CreateTablet for tablet 8ad19be826494e9fa81d8b26a929ff4c (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:28:44.108700 14453 tablet_service.cc:1511] Processing CreateTablet for tablet 9eec14432a4543eea89701c25066887b (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:28:44.110489 14454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ad19be826494e9fa81d8b26a929ff4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.114106 14453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eec14432a4543eea89701c25066887b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.114500 14452 tablet_service.cc:1511] Processing CreateTablet for tablet 320069d83c9f4143869dd90f6d253134 (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:28:44.115998 14452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 320069d83c9f4143869dd90f6d253134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.117072 14380 tablet_service.cc:1511] Processing CreateTablet for tablet 8ad19be826494e9fa81d8b26a929ff4c (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:28:44.122923 14310 tablet_service.cc:1511] Processing CreateTablet for tablet d6dcc07ba9784ae98d3974c6abb1b53b (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:28:44.123528 14451 tablet_service.cc:1511] Processing CreateTablet for tablet 5e6555f9875c4743a1680538e4c03821 (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:28:44.124814 14310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6dcc07ba9784ae98d3974c6abb1b53b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.125260 14451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e6555f9875c4743a1680538e4c03821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.125970 14380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ad19be826494e9fa81d8b26a929ff4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.127543 14381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 527f315df63447e3bc065eb01506adec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.127528 14304 tablet_service.cc:1511] Processing CreateTablet for tablet 9eec14432a4543eea89701c25066887b (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:28:44.129187 14304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eec14432a4543eea89701c25066887b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.130092 14384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 625d6ab82ccb4bfca8333bbf41e4ce0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.129769 14450 tablet_service.cc:1511] Processing CreateTablet for tablet 36f62e820fa642a1bcd4095a04b01648 (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:28:44.131389 14450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36f62e820fa642a1bcd4095a04b01648. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.132894 14306 tablet_service.cc:1511] Processing CreateTablet for tablet 527f315df63447e3bc065eb01506adec (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:28:44.130951 14305 tablet_service.cc:1511] Processing CreateTablet for tablet 8ad19be826494e9fa81d8b26a929ff4c (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:28:44.134086 14382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7aeb6f75770b4fd5bf7692ba6e2ffa92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.134467 14306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 527f315df63447e3bc065eb01506adec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.138173 14305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ad19be826494e9fa81d8b26a929ff4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.138955 14379 tablet_service.cc:1511] Processing CreateTablet for tablet 9eec14432a4543eea89701c25066887b (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:28:44.139451 14307 tablet_service.cc:1511] Processing CreateTablet for tablet 7aeb6f75770b4fd5bf7692ba6e2ffa92 (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:28:44.140705 14379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eec14432a4543eea89701c25066887b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.141283 14307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7aeb6f75770b4fd5bf7692ba6e2ffa92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.152441 14303 tablet_service.cc:1511] Processing CreateTablet for tablet 320069d83c9f4143869dd90f6d253134 (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:28:44.143723 14308 tablet_service.cc:1511] Processing CreateTablet for tablet 354b6360e1104e169aaee9531ae2c3a6 (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:28:44.146170 14309 tablet_service.cc:1511] Processing CreateTablet for tablet 625d6ab82ccb4bfca8333bbf41e4ce0f (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:28:44.154716 14376 tablet_service.cc:1511] Processing CreateTablet for tablet 36f62e820fa642a1bcd4095a04b01648 (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:28:44.155287 14302 tablet_service.cc:1511] Processing CreateTablet for tablet 5e6555f9875c4743a1680538e4c03821 (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:28:44.156584 14376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36f62e820fa642a1bcd4095a04b01648. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.143602 14378 tablet_service.cc:1511] Processing CreateTablet for tablet 320069d83c9f4143869dd90f6d253134 (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:28:44.158672 14378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 320069d83c9f4143869dd90f6d253134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.158347 14301 tablet_service.cc:1511] Processing CreateTablet for tablet 36f62e820fa642a1bcd4095a04b01648 (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:28:44.152521 14377 tablet_service.cc:1511] Processing CreateTablet for tablet 5e6555f9875c4743a1680538e4c03821 (DEFAULT_TABLE table=test-workload [id=db185f49126845ea94ea8d6ec87b70cd]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:28:44.154767 14308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 354b6360e1104e169aaee9531ae2c3a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.162216 14377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e6555f9875c4743a1680538e4c03821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.162915 14309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 625d6ab82ccb4bfca8333bbf41e4ce0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.170535 14303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 320069d83c9f4143869dd90f6d253134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.175266 14302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e6555f9875c4743a1680538e4c03821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.178128 14301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36f62e820fa642a1bcd4095a04b01648. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.334838 14509 tablet_bootstrap.cc:492] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:44.335628 14508 tablet_bootstrap.cc:492] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903: Bootstrap starting.
I20260709 12:28:44.344216 14510 tablet_bootstrap.cc:492] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab: Bootstrap starting.
I20260709 12:28:44.354665 14510 tablet_bootstrap.cc:654] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.354772 14509 tablet_bootstrap.cc:654] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.354772 14508 tablet_bootstrap.cc:654] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.384320 14508 tablet_bootstrap.cc:492] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903: No bootstrap required, opened a new log
I20260709 12:28:44.384825 14508 ts_tablet_manager.cc:1403] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.020s
I20260709 12:28:44.385550 14510 tablet_bootstrap.cc:492] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab: No bootstrap required, opened a new log
I20260709 12:28:44.386650 14510 ts_tablet_manager.cc:1403] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab: Time spent bootstrapping tablet: real 0.043s	user 0.020s	sys 0.021s
I20260709 12:28:44.385550 14509 tablet_bootstrap.cc:492] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8: No bootstrap required, opened a new log
I20260709 12:28:44.387356 14509 ts_tablet_manager.cc:1403] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.053s	user 0.017s	sys 0.026s
I20260709 12:28:44.387604 14508 raft_consensus.cc:359] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.388442 14508 raft_consensus.cc:385] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.388783 14508 raft_consensus.cc:740] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.389714 14508 consensus_queue.cc:260] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.390038 14510 raft_consensus.cc:359] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.390911 14510 raft_consensus.cc:385] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.391606 14510 raft_consensus.cc:740] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a153a1160e9240439004094da9aa01ab, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.390792 14509 raft_consensus.cc:359] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.392469 14509 raft_consensus.cc:385] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.392774 14510 consensus_queue.cc:260] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.393030 14509 raft_consensus.cc:740] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.395337 14494 heartbeater.cc:499] Master 127.2.4.254:35633 was elected leader, sending a full tablet report...
I20260709 12:28:44.394906 14509 consensus_queue.cc:260] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.395852 14508 ts_tablet_manager.cc:1434] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.003s
I20260709 12:28:44.396983 14508 tablet_bootstrap.cc:492] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903: Bootstrap starting.
I20260709 12:28:44.403103 14345 heartbeater.cc:499] Master 127.2.4.254:35633 was elected leader, sending a full tablet report...
I20260709 12:28:44.403219 14510 ts_tablet_manager.cc:1434] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.005s
I20260709 12:28:44.403702 14509 ts_tablet_manager.cc:1434] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.016s	user 0.015s	sys 0.001s
I20260709 12:28:44.404843 14509 tablet_bootstrap.cc:492] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:44.408537 14420 heartbeater.cc:499] Master 127.2.4.254:35633 was elected leader, sending a full tablet report...
I20260709 12:28:44.409765 14510 tablet_bootstrap.cc:492] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab: Bootstrap starting.
I20260709 12:28:44.411543 14509 tablet_bootstrap.cc:654] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.417822 14510 tablet_bootstrap.cc:654] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.418334 14509 tablet_bootstrap.cc:492] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8: No bootstrap required, opened a new log
I20260709 12:28:44.419037 14509 ts_tablet_manager.cc:1403] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.014s	user 0.004s	sys 0.009s
I20260709 12:28:44.422225 14509 raft_consensus.cc:359] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.422982 14509 raft_consensus.cc:385] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.423286 14509 raft_consensus.cc:740] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.423573 14508 tablet_bootstrap.cc:654] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.424237 14509 consensus_queue.cc:260] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.426780 14509 ts_tablet_manager.cc:1434] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:28:44.428154 14509 tablet_bootstrap.cc:492] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:44.431430 14510 tablet_bootstrap.cc:492] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab: No bootstrap required, opened a new log
I20260709 12:28:44.432278 14510 ts_tablet_manager.cc:1403] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.009s
W20260709 12:28:44.434760 14495 tablet.cc:2406] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:28:44.436161 14510 raft_consensus.cc:359] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.437142 14510 raft_consensus.cc:385] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.437511 14510 raft_consensus.cc:740] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a153a1160e9240439004094da9aa01ab, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.437671 14509 tablet_bootstrap.cc:654] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.438827 14510 consensus_queue.cc:260] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.440716 14514 raft_consensus.cc:493] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.441327 14514 raft_consensus.cc:515] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.441994 14510 ts_tablet_manager.cc:1434] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:28:44.443408 14510 tablet_bootstrap.cc:492] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab: Bootstrap starting.
I20260709 12:28:44.451251 14510 tablet_bootstrap.cc:654] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.457012 14514 leader_election.cc:290] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:44.472630 14320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "320069d83c9f4143869dd90f6d253134" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" is_pre_election: true
W20260709 12:28:44.478475 14434 leader_election.cc:343] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:44.481300 14266 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:44.542913 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "320069d83c9f4143869dd90f6d253134" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
I20260709 12:28:44.544894 14516 raft_consensus.cc:493] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.545820 14516 raft_consensus.cc:515] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
W20260709 12:28:44.550561 14435 leader_election.cc:343] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a153a1160e9240439004094da9aa01ab (127.2.4.194:36325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:44.551481 14435 leader_election.cc:304] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe2b40322a94e769d80927662234903; no voters: a153a1160e9240439004094da9aa01ab, e621e3baa9aa42ab963c4f3af3bca8b8
I20260709 12:28:44.552693 14514 raft_consensus.cc:2749] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:44.552887 14516 leader_election.cc:290] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a153a1160e9240439004094da9aa01ab (127.2.4.194:36325), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:44.562956 14508 tablet_bootstrap.cc:492] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903: No bootstrap required, opened a new log
I20260709 12:28:44.563541 14508 ts_tablet_manager.cc:1403] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903: Time spent bootstrapping tablet: real 0.167s	user 0.049s	sys 0.083s
I20260709 12:28:44.566923 14508 raft_consensus.cc:359] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.567984 14510 tablet_bootstrap.cc:492] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab: No bootstrap required, opened a new log
I20260709 12:28:44.567476 14509 tablet_bootstrap.cc:492] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8: No bootstrap required, opened a new log
I20260709 12:28:44.569605 14510 ts_tablet_manager.cc:1403] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab: Time spent bootstrapping tablet: real 0.126s	user 0.041s	sys 0.056s
I20260709 12:28:44.570456 14509 ts_tablet_manager.cc:1403] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.143s	user 0.046s	sys 0.078s
I20260709 12:28:44.572129 14508 raft_consensus.cc:385] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.572520 14508 raft_consensus.cc:740] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.573474 14510 raft_consensus.cc:359] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.574834 14510 raft_consensus.cc:385] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.574491 14509 raft_consensus.cc:359] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.575538 14509 raft_consensus.cc:385] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.575361 14510 raft_consensus.cc:740] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a153a1160e9240439004094da9aa01ab, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.576028 14509 raft_consensus.cc:740] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.576874 14509 consensus_queue.cc:260] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.576941 14508 consensus_queue.cc:260] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.577836 14510 consensus_queue.cc:260] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.579325 14509 ts_tablet_manager.cc:1434] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:28:44.580433 14509 tablet_bootstrap.cc:492] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:44.580607 14510 ts_tablet_manager.cc:1434] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 12:28:44.587657 14509 tablet_bootstrap.cc:654] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.588356 14508 ts_tablet_manager.cc:1434] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.003s
I20260709 12:28:44.589692 14508 tablet_bootstrap.cc:492] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903: Bootstrap starting.
I20260709 12:28:44.594161 14510 tablet_bootstrap.cc:492] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab: Bootstrap starting.
I20260709 12:28:44.605365 14508 tablet_bootstrap.cc:654] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.605443 14509 tablet_bootstrap.cc:492] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8: No bootstrap required, opened a new log
I20260709 12:28:44.606258 14509 ts_tablet_manager.cc:1403] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.026s	user 0.024s	sys 0.000s
I20260709 12:28:44.606897 14510 tablet_bootstrap.cc:654] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.609978 14509 raft_consensus.cc:359] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.611176 14509 raft_consensus.cc:385] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.611845 14509 raft_consensus.cc:740] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.613039 14509 consensus_queue.cc:260] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.616557 14509 ts_tablet_manager.cc:1434] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 12:28:44.617894 14509 tablet_bootstrap.cc:492] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:44.619532 14510 tablet_bootstrap.cc:492] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab: No bootstrap required, opened a new log
I20260709 12:28:44.620193 14510 ts_tablet_manager.cc:1403] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.006s
I20260709 12:28:44.624310 14510 raft_consensus.cc:359] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.625131 14510 raft_consensus.cc:385] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.625460 14510 raft_consensus.cc:740] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a153a1160e9240439004094da9aa01ab, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.625435 14509 tablet_bootstrap.cc:654] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.627067 14516 raft_consensus.cc:493] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.627614 14516 raft_consensus.cc:515] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.629921 14516 leader_election.cc:290] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a153a1160e9240439004094da9aa01ab (127.2.4.194:36325), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:44.626502 14510 consensus_queue.cc:260] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.632287 14510 ts_tablet_manager.cc:1434] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.002s
I20260709 12:28:44.632440 14469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "527f315df63447e3bc065eb01506adec" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903" is_pre_election: true
I20260709 12:28:44.633525 14510 tablet_bootstrap.cc:492] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab: Bootstrap starting.
I20260709 12:28:44.633236 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "527f315df63447e3bc065eb01506adec" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
I20260709 12:28:44.633729 14469 raft_consensus.cc:2468] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in term 0.
I20260709 12:28:44.635440 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f62e820fa642a1bcd4095a04b01648" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
I20260709 12:28:44.635941 14468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f62e820fa642a1bcd4095a04b01648" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903" is_pre_election: true
W20260709 12:28:44.637187 14286 leader_election.cc:343] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a153a1160e9240439004094da9aa01ab (127.2.4.194:36325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:44.638119 14286 leader_election.cc:343] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a153a1160e9240439004094da9aa01ab (127.2.4.194:36325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:44.638911 14286 leader_election.cc:304] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: a153a1160e9240439004094da9aa01ab
I20260709 12:28:44.639827 14516 raft_consensus.cc:2804] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:44.640221 14516 raft_consensus.cc:493] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.640547 14516 raft_consensus.cc:3060] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.641063 14510 tablet_bootstrap.cc:654] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:44.650421 14286 leader_election.cc:343] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe2b40322a94e769d80927662234903 (127.2.4.195:37025): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:44.651028 14286 leader_election.cc:304] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e621e3baa9aa42ab963c4f3af3bca8b8; no voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903
I20260709 12:28:44.654875 14516 raft_consensus.cc:515] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.657835 14516 leader_election.cc:290] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 election: Requested vote from peers a153a1160e9240439004094da9aa01ab (127.2.4.194:36325), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:44.658613 14516 raft_consensus.cc:2749] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:44.663450 14508 tablet_bootstrap.cc:492] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903: No bootstrap required, opened a new log
I20260709 12:28:44.666141 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "527f315df63447e3bc065eb01506adec" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:44.667357 14508 ts_tablet_manager.cc:1403] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903: Time spent bootstrapping tablet: real 0.078s	user 0.040s	sys 0.024s
I20260709 12:28:44.667287 14468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "527f315df63447e3bc065eb01506adec" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903"
I20260709 12:28:44.667980 14510 tablet_bootstrap.cc:492] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab: No bootstrap required, opened a new log
I20260709 12:28:44.668092 14468 raft_consensus.cc:3060] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.668895 14510 ts_tablet_manager.cc:1403] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.001s
W20260709 12:28:44.668331 14286 leader_election.cc:343] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a153a1160e9240439004094da9aa01ab (127.2.4.194:36325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:44.671057 14508 raft_consensus.cc:359] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.671725 14509 tablet_bootstrap.cc:492] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8: No bootstrap required, opened a new log
I20260709 12:28:44.671904 14508 raft_consensus.cc:385] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.672336 14508 raft_consensus.cc:740] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.672446 14509 ts_tablet_manager.cc:1403] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.015s
I20260709 12:28:44.672703 14510 raft_consensus.cc:359] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.673240 14508 consensus_queue.cc:260] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.673918 14510 raft_consensus.cc:385] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.675918 14508 ts_tablet_manager.cc:1434] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:28:44.676537 14509 raft_consensus.cc:359] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } }
I20260709 12:28:44.677107 14508 tablet_bootstrap.cc:492] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903: Bootstrap starting.
I20260709 12:28:44.677420 14509 raft_consensus.cc:385] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.677820 14509 raft_consensus.cc:740] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.678059 14468 raft_consensus.cc:2468] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in term 1.
I20260709 12:28:44.678860 14509 consensus_queue.cc:260] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } }
I20260709 12:28:44.679600 14286 leader_election.cc:304] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: a153a1160e9240439004094da9aa01ab
I20260709 12:28:44.681310 14516 raft_consensus.cc:2804] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:44.681527 14509 ts_tablet_manager.cc:1434] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:28:44.681902 14516 raft_consensus.cc:697] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 LEADER]: Becoming Leader. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Running, Role: LEADER
I20260709 12:28:44.682862 14509 tablet_bootstrap.cc:492] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:44.682880 14516 consensus_queue.cc:237] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.684584 14508 tablet_bootstrap.cc:654] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.691205 14509 tablet_bootstrap.cc:654] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.674402 14510 raft_consensus.cc:740] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a153a1160e9240439004094da9aa01ab, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.693864 14510 consensus_queue.cc:260] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.697986 14510 ts_tablet_manager.cc:1434] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.006s
I20260709 12:28:44.700827 14515 raft_consensus.cc:493] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.701753 14515 raft_consensus.cc:515] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.704224 14509 tablet_bootstrap.cc:492] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8: No bootstrap required, opened a new log
I20260709 12:28:44.704957 14509 ts_tablet_manager.cc:1403] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.001s
I20260709 12:28:44.699693 14510 tablet_bootstrap.cc:492] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab: Bootstrap starting.
I20260709 12:28:44.708859 14509 raft_consensus.cc:359] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.709780 14509 raft_consensus.cc:385] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.710160 14509 raft_consensus.cc:740] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.711287 14509 consensus_queue.cc:260] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.716357 14509 ts_tablet_manager.cc:1434] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 12:28:44.717684 14509 tablet_bootstrap.cc:492] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:44.719653 14515 leader_election.cc:290] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:44.722383 14508 tablet_bootstrap.cc:492] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903: No bootstrap required, opened a new log
I20260709 12:28:44.725708 14509 tablet_bootstrap.cc:654] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.723021 14508 ts_tablet_manager.cc:1403] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903: Time spent bootstrapping tablet: real 0.046s	user 0.028s	sys 0.014s
I20260709 12:28:44.727587 14510 tablet_bootstrap.cc:654] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.729898 14508 raft_consensus.cc:359] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } }
I20260709 12:28:44.730815 14508 raft_consensus.cc:385] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.731140 14508 raft_consensus.cc:740] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.732467 14508 consensus_queue.cc:260] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } }
I20260709 12:28:44.736404 14508 ts_tablet_manager.cc:1434] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:28:44.735206 14217 catalog_manager.cc:5697] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:44.739583 14515 raft_consensus.cc:493] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.740553 14515 raft_consensus.cc:515] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.739411 14516 raft_consensus.cc:493] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.741112 14508 tablet_bootstrap.cc:492] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903: Bootstrap starting.
I20260709 12:28:44.743947 14530 raft_consensus.cc:493] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.744617 14515 leader_election.cc:290] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:44.745406 14530 raft_consensus.cc:515] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.749310 14530 leader_election.cc:290] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a153a1160e9240439004094da9aa01ab (127.2.4.194:36325), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:44.750319 14516 raft_consensus.cc:515] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } }
I20260709 12:28:44.751755 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "354b6360e1104e169aaee9531ae2c3a6" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
I20260709 12:28:44.752794 14395 raft_consensus.cc:2468] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in term 0.
I20260709 12:28:44.754539 14286 leader_election.cc:304] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:44.755786 14468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "354b6360e1104e169aaee9531ae2c3a6" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903" is_pre_election: true
I20260709 12:28:44.756632 14468 raft_consensus.cc:2468] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in term 0.
I20260709 12:28:44.757001 14508 tablet_bootstrap.cc:654] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.758782 14530 raft_consensus.cc:2804] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:44.759291 14530 raft_consensus.cc:493] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.759716 14530 raft_consensus.cc:3060] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.772003 14530 raft_consensus.cc:515] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.776408 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "354b6360e1104e169aaee9531ae2c3a6" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:44.777299 14395 raft_consensus.cc:3060] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.781116 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6dcc07ba9784ae98d3974c6abb1b53b" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
W20260709 12:28:44.783059 14286 leader_election.cc:343] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a153a1160e9240439004094da9aa01ab (127.2.4.194:36325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:44.785073 14516 leader_election.cc:290] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a153a1160e9240439004094da9aa01ab (127.2.4.194:36325), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:44.786033 14468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6dcc07ba9784ae98d3974c6abb1b53b" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903" is_pre_election: true
I20260709 12:28:44.787073 14468 raft_consensus.cc:2468] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in term 0.
I20260709 12:28:44.789100 14286 leader_election.cc:304] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: a153a1160e9240439004094da9aa01ab
I20260709 12:28:44.802074 14395 raft_consensus.cc:2468] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in term 1.
I20260709 12:28:44.805773 14468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "354b6360e1104e169aaee9531ae2c3a6" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903"
I20260709 12:28:44.812973 14468 raft_consensus.cc:3060] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.808390 14516 raft_consensus.cc:2804] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:44.809000 14320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e6555f9875c4743a1680538e4c03821" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" is_pre_election: true
I20260709 12:28:44.810392 14319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" is_pre_election: true
I20260709 12:28:44.818142 14516 raft_consensus.cc:493] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.819288 14516 raft_consensus.cc:3060] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:44.819466 14360 leader_election.cc:343] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:44.820420 14286 leader_election.cc:304] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:44.821944 14515 raft_consensus.cc:493] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.822564 14515 raft_consensus.cc:515] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.824891 14515 leader_election.cc:290] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:44.826097 14320 raft_consensus.cc:2468] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 0.
I20260709 12:28:44.828913 14360 leader_election.cc:304] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:44.829587 14469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e6555f9875c4743a1680538e4c03821" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903" is_pre_election: true
I20260709 12:28:44.830190 14467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903" is_pre_election: true
I20260709 12:28:44.830644 14320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec14432a4543eea89701c25066887b" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" is_pre_election: true
I20260709 12:28:44.831459 14508 tablet_bootstrap.cc:492] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903: No bootstrap required, opened a new log
I20260709 12:28:44.831722 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec14432a4543eea89701c25066887b" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903" is_pre_election: true
W20260709 12:28:44.832334 14360 leader_election.cc:343] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:44.832137 14361 leader_election.cc:343] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe2b40322a94e769d80927662234903 (127.2.4.195:37025): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:44.832090 14508 ts_tablet_manager.cc:1403] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903: Time spent bootstrapping tablet: real 0.091s	user 0.028s	sys 0.042s
W20260709 12:28:44.833870 14361 leader_election.cc:343] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe2b40322a94e769d80927662234903 (127.2.4.195:37025): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:44.834324 14361 leader_election.cc:304] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a153a1160e9240439004094da9aa01ab; no voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8
I20260709 12:28:44.832420 14515 raft_consensus.cc:2804] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:44.835829 14515 raft_consensus.cc:493] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.836210 14515 raft_consensus.cc:3060] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.811141 14530 leader_election.cc:290] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 election: Requested vote from peers a153a1160e9240439004094da9aa01ab (127.2.4.194:36325), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:44.838276 14530 raft_consensus.cc:2804] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:44.839092 14530 raft_consensus.cc:697] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 LEADER]: Becoming Leader. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Running, Role: LEADER
I20260709 12:28:44.838061 14508 raft_consensus.cc:359] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.839390 14516 raft_consensus.cc:515] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } }
I20260709 12:28:44.840655 14530 consensus_queue.cc:237] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.842273 14516 leader_election.cc:290] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 election: Requested vote from peers a153a1160e9240439004094da9aa01ab (127.2.4.194:36325), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:44.842787 14508 raft_consensus.cc:385] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.843256 14508 raft_consensus.cc:740] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.844448 14508 consensus_queue.cc:260] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.846622 14510 tablet_bootstrap.cc:492] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab: No bootstrap required, opened a new log
I20260709 12:28:44.847543 14510 ts_tablet_manager.cc:1403] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab: Time spent bootstrapping tablet: real 0.148s	user 0.009s	sys 0.007s
I20260709 12:28:44.850525 14508 ts_tablet_manager.cc:1434] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.003s
I20260709 12:28:44.852845 14508 tablet_bootstrap.cc:492] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903: Bootstrap starting.
I20260709 12:28:44.828935 14509 tablet_bootstrap.cc:492] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8: No bootstrap required, opened a new log
I20260709 12:28:44.855922 14509 ts_tablet_manager.cc:1403] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.139s	user 0.039s	sys 0.058s
I20260709 12:28:44.860168 14508 tablet_bootstrap.cc:654] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.862962 14510 raft_consensus.cc:359] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } }
I20260709 12:28:44.864260 14510 raft_consensus.cc:385] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.867965 14510 raft_consensus.cc:740] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a153a1160e9240439004094da9aa01ab, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.868455 14543 raft_consensus.cc:2749] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:44.866060 14509 raft_consensus.cc:359] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
W20260709 12:28:44.870262 14361 leader_election.cc:343] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe2b40322a94e769d80927662234903 (127.2.4.195:37025): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:44.870911 14361 leader_election.cc:304] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a153a1160e9240439004094da9aa01ab; no voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8
I20260709 12:28:44.870743 14510 consensus_queue.cc:260] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } }
I20260709 12:28:44.867415 14468 raft_consensus.cc:2468] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in term 1.
I20260709 12:28:44.870265 14509 raft_consensus.cc:385] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:28:44.866760 14286 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.194:36325, user_credentials={real_user=slave}} blocked reactor thread for 46441.6us
I20260709 12:28:44.866914 14515 raft_consensus.cc:515] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.876382 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6dcc07ba9784ae98d3974c6abb1b53b" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903"
I20260709 12:28:44.877384 14466 raft_consensus.cc:3060] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.877508 14320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e6555f9875c4743a1680538e4c03821" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:44.876051 14508 tablet_bootstrap.cc:492] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903: No bootstrap required, opened a new log
I20260709 12:28:44.878471 14320 raft_consensus.cc:3060] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.879177 14508 ts_tablet_manager.cc:1403] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903: Time spent bootstrapping tablet: real 0.027s	user 0.005s	sys 0.008s
I20260709 12:28:44.873709 14509 raft_consensus.cc:740] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.885744 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6dcc07ba9784ae98d3974c6abb1b53b" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:44.887400 14395 raft_consensus.cc:3060] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.892289 14544 raft_consensus.cc:2749] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:44.896421 14466 raft_consensus.cc:2468] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in term 1.
I20260709 12:28:44.897148 14320 raft_consensus.cc:2468] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 1.
I20260709 12:28:44.898063 14286 leader_election.cc:304] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:44.899137 14542 raft_consensus.cc:2804] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:44.899652 14542 raft_consensus.cc:697] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 LEADER]: Becoming Leader. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Running, Role: LEADER
I20260709 12:28:44.900534 14542 consensus_queue.cc:237] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } }
I20260709 12:28:44.902496 14509 consensus_queue.cc:260] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.903635 14468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e6555f9875c4743a1680538e4c03821" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903"
I20260709 12:28:44.904582 14509 ts_tablet_manager.cc:1434] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.001s
W20260709 12:28:44.905980 14361 leader_election.cc:343] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe2b40322a94e769d80927662234903 (127.2.4.195:37025): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:44.907974 14508 raft_consensus.cc:359] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.908876 14508 raft_consensus.cc:385] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.909180 14508 raft_consensus.cc:740] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.909734 14509 tablet_bootstrap.cc:492] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:44.910602 14360 leader_election.cc:304] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a153a1160e9240439004094da9aa01ab, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: abe2b40322a94e769d80927662234903
I20260709 12:28:44.911737 14544 raft_consensus.cc:2804] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:44.912204 14544 raft_consensus.cc:697] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [term 1 LEADER]: Becoming Leader. State: Replica: a153a1160e9240439004094da9aa01ab, State: Running, Role: LEADER
I20260709 12:28:44.913012 14544 consensus_queue.cc:237] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.913846 14395 raft_consensus.cc:2468] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in term 1.
I20260709 12:28:44.891290 14515 leader_election.cc:290] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:44.886186 14543 raft_consensus.cc:493] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.918401 14543 raft_consensus.cc:515] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.921651 14320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad19be826494e9fa81d8b26a929ff4c" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" is_pre_election: true
W20260709 12:28:44.926174 14360 leader_election.cc:343] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:44.933951 14510 ts_tablet_manager.cc:1434] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab: Time spent starting tablet: real 0.086s	user 0.021s	sys 0.029s
I20260709 12:28:44.937109 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad19be826494e9fa81d8b26a929ff4c" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903" is_pre_election: true
I20260709 12:28:44.936328 14216 catalog_manager.cc:5697] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab reported cstate change: term changed from 0 to 1, leader changed from <none> to a153a1160e9240439004094da9aa01ab (127.2.4.194). New cstate: current_term: 1 leader_uuid: "a153a1160e9240439004094da9aa01ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:44.938917 14361 leader_election.cc:343] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe2b40322a94e769d80927662234903 (127.2.4.195:37025): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:44.928064 14508 consensus_queue.cc:260] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.939456 14361 leader_election.cc:304] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a153a1160e9240439004094da9aa01ab; no voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8
I20260709 12:28:44.946465 14510 tablet_bootstrap.cc:492] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab: Bootstrap starting.
I20260709 12:28:44.946606 14548 raft_consensus.cc:2749] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:44.948061 14217 catalog_manager.cc:5697] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:44.935329 14543 leader_election.cc:290] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:44.953233 14508 ts_tablet_manager.cc:1434] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903: Time spent starting tablet: real 0.073s	user 0.006s	sys 0.004s
I20260709 12:28:44.954943 14509 tablet_bootstrap.cc:654] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.955032 14508 tablet_bootstrap.cc:492] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903: Bootstrap starting.
I20260709 12:28:44.967084 14510 tablet_bootstrap.cc:654] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.969805 14542 raft_consensus.cc:493] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.970522 14542 raft_consensus.cc:515] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.972950 14542 leader_election.cc:290] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a153a1160e9240439004094da9aa01ab (127.2.4.194:36325), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:44.973821 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "320069d83c9f4143869dd90f6d253134" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
I20260709 12:28:44.974174 14508 tablet_bootstrap.cc:654] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.974535 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "320069d83c9f4143869dd90f6d253134" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903" is_pre_election: true
I20260709 12:28:44.975600 14466 raft_consensus.cc:2468] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in term 0.
W20260709 12:28:44.975819 14286 leader_election.cc:343] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a153a1160e9240439004094da9aa01ab (127.2.4.194:36325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:44.977661 14286 leader_election.cc:304] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: a153a1160e9240439004094da9aa01ab
I20260709 12:28:44.978552 14542 raft_consensus.cc:2804] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:44.978950 14542 raft_consensus.cc:493] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.979558 14542 raft_consensus.cc:3060] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.972436 14218 catalog_manager.cc:5697] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:44.980301 14510 tablet_bootstrap.cc:492] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab: No bootstrap required, opened a new log
I20260709 12:28:44.981170 14510 ts_tablet_manager.cc:1403] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.006s
I20260709 12:28:44.984395 14510 raft_consensus.cc:359] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.985203 14510 raft_consensus.cc:385] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.986717 14509 tablet_bootstrap.cc:492] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8: No bootstrap required, opened a new log
I20260709 12:28:44.987344 14509 ts_tablet_manager.cc:1403] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.005s
I20260709 12:28:44.987428 14508 tablet_bootstrap.cc:492] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903: No bootstrap required, opened a new log
I20260709 12:28:44.988148 14508 ts_tablet_manager.cc:1403] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.001s
I20260709 12:28:44.990212 14509 raft_consensus.cc:359] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.991004 14509 raft_consensus.cc:385] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.991283 14509 raft_consensus.cc:740] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.991662 14508 raft_consensus.cc:359] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.992587 14508 raft_consensus.cc:385] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.992141 14509 consensus_queue.cc:260] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.992949 14508 raft_consensus.cc:740] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.993049 14542 raft_consensus.cc:515] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.985504 14510 raft_consensus.cc:740] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a153a1160e9240439004094da9aa01ab, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.994912 14509 ts_tablet_manager.cc:1434] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:44.994676 14510 consensus_queue.cc:260] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:44.997382 14510 ts_tablet_manager.cc:1434] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 12:28:44.997495 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "320069d83c9f4143869dd90f6d253134" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903"
I20260709 12:28:44.998324 14466 raft_consensus.cc:3060] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.998217 14508 consensus_queue.cc:260] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.000859 14508 ts_tablet_manager.cc:1434] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 12:28:45.003625 14508 tablet_bootstrap.cc:492] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903: Bootstrap starting.
I20260709 12:28:45.004773 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "320069d83c9f4143869dd90f6d253134" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:45.005694 14510 tablet_bootstrap.cc:492] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab: Bootstrap starting.
W20260709 12:28:45.006397 14286 leader_election.cc:343] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a153a1160e9240439004094da9aa01ab (127.2.4.194:36325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:45.006934 14542 leader_election.cc:290] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 election: Requested vote from peers a153a1160e9240439004094da9aa01ab (127.2.4.194:36325), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:45.008026 14466 raft_consensus.cc:2468] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in term 1.
I20260709 12:28:45.009155 14509 tablet_bootstrap.cc:492] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:45.012946 14510 tablet_bootstrap.cc:654] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.014377 14286 leader_election.cc:304] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: a153a1160e9240439004094da9aa01ab
I20260709 12:28:45.015916 14542 raft_consensus.cc:2804] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:45.016538 14542 raft_consensus.cc:697] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 LEADER]: Becoming Leader. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Running, Role: LEADER
I20260709 12:28:45.017683 14542 consensus_queue.cc:237] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.024487 14508 tablet_bootstrap.cc:654] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.025679 14509 tablet_bootstrap.cc:654] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.038460 14509 tablet_bootstrap.cc:492] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8: No bootstrap required, opened a new log
I20260709 12:28:45.038762 14508 tablet_bootstrap.cc:492] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903: No bootstrap required, opened a new log
I20260709 12:28:45.037997 14215 catalog_manager.cc:5697] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:45.039016 14509 ts_tablet_manager.cc:1403] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.005s
I20260709 12:28:45.039458 14508 ts_tablet_manager.cc:1403] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.010s
I20260709 12:28:45.041481 14509 raft_consensus.cc:359] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.042294 14509 raft_consensus.cc:385] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.042541 14509 raft_consensus.cc:740] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.042809 14508 raft_consensus.cc:359] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.043787 14508 raft_consensus.cc:385] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.044210 14508 raft_consensus.cc:740] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.043946 14509 consensus_queue.cc:260] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.045571 14508 consensus_queue.cc:260] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.047580 14509 ts_tablet_manager.cc:1434] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:28:45.048708 14509 tablet_bootstrap.cc:492] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:45.048986 14508 ts_tablet_manager.cc:1434] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:28:45.050194 14508 tablet_bootstrap.cc:492] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903: Bootstrap starting.
I20260709 12:28:45.054625 14510 tablet_bootstrap.cc:492] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab: No bootstrap required, opened a new log
I20260709 12:28:45.055495 14510 ts_tablet_manager.cc:1403] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.003s
I20260709 12:28:45.065393 14509 tablet_bootstrap.cc:654] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.066148 14508 tablet_bootstrap.cc:654] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.067595 14510 raft_consensus.cc:359] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.068605 14510 raft_consensus.cc:385] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.068948 14510 raft_consensus.cc:740] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a153a1160e9240439004094da9aa01ab, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.069953 14510 consensus_queue.cc:260] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.073442 14510 ts_tablet_manager.cc:1434] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 12:28:45.073748 14508 tablet_bootstrap.cc:492] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903: No bootstrap required, opened a new log
I20260709 12:28:45.073596 14509 tablet_bootstrap.cc:492] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8: No bootstrap required, opened a new log
I20260709 12:28:45.074472 14509 ts_tablet_manager.cc:1403] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.007s
I20260709 12:28:45.074649 14510 tablet_bootstrap.cc:492] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab: Bootstrap starting.
I20260709 12:28:45.074472 14508 ts_tablet_manager.cc:1403] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.003s
I20260709 12:28:45.077672 14509 raft_consensus.cc:359] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.078619 14509 raft_consensus.cc:385] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.078995 14509 raft_consensus.cc:740] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.079737 14509 consensus_queue.cc:260] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.080907 14510 tablet_bootstrap.cc:654] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.082306 14509 ts_tablet_manager.cc:1434] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:28:45.085134 14508 raft_consensus.cc:359] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.086081 14508 raft_consensus.cc:385] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.086354 14508 raft_consensus.cc:740] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.087685 14510 tablet_bootstrap.cc:492] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab: No bootstrap required, opened a new log
I20260709 12:28:45.087256 14508 consensus_queue.cc:260] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.088301 14510 ts_tablet_manager.cc:1403] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.005s
I20260709 12:28:45.089920 14508 ts_tablet_manager.cc:1434] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:28:45.091176 14508 tablet_bootstrap.cc:492] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903: Bootstrap starting.
I20260709 12:28:45.090905 14510 raft_consensus.cc:359] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.091742 14510 raft_consensus.cc:385] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.092029 14510 raft_consensus.cc:740] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a153a1160e9240439004094da9aa01ab, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.092767 14510 consensus_queue.cc:260] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.094915 14510 ts_tablet_manager.cc:1434] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:28:45.096473 14510 tablet_bootstrap.cc:492] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab: Bootstrap starting.
I20260709 12:28:45.098822 14508 tablet_bootstrap.cc:654] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.104382 14510 tablet_bootstrap.cc:654] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.115286 14510 tablet_bootstrap.cc:492] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab: No bootstrap required, opened a new log
I20260709 12:28:45.115286 14508 tablet_bootstrap.cc:492] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903: No bootstrap required, opened a new log
I20260709 12:28:45.115919 14508 ts_tablet_manager.cc:1403] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.004s
I20260709 12:28:45.115911 14510 ts_tablet_manager.cc:1403] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.008s
I20260709 12:28:45.118937 14508 raft_consensus.cc:359] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.119079 14510 raft_consensus.cc:359] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.119763 14508 raft_consensus.cc:385] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.119808 14510 raft_consensus.cc:385] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.120183 14508 raft_consensus.cc:740] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.120260 14510 raft_consensus.cc:740] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a153a1160e9240439004094da9aa01ab, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.121025 14508 consensus_queue.cc:260] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.121025 14510 consensus_queue.cc:260] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.123459 14510 ts_tablet_manager.cc:1434] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:28:45.123992 14508 ts_tablet_manager.cc:1434] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:28:45.155367 14543 raft_consensus.cc:493] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.155824 14395 raft_consensus.cc:3060] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.156241 14543 raft_consensus.cc:515] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.159541 14543 leader_election.cc:290] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:45.160275 14320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "320069d83c9f4143869dd90f6d253134" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" is_pre_election: true
I20260709 12:28:45.160344 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "320069d83c9f4143869dd90f6d253134" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903" is_pre_election: true
I20260709 12:28:45.161412 14466 raft_consensus.cc:2393] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a153a1160e9240439004094da9aa01ab in current term 1: Already voted for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in this term.
I20260709 12:28:45.162736 14361 leader_election.cc:304] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a153a1160e9240439004094da9aa01ab; no voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8
I20260709 12:28:45.163690 14543 raft_consensus.cc:3060] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.168437 14542 consensus_queue.cc:1048] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:45.173949 14543 raft_consensus.cc:2749] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:45.183589 14516 consensus_queue.cc:1048] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:45.198474 14514 raft_consensus.cc:493] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.199059 14514 raft_consensus.cc:515] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.201131 14514 leader_election.cc:290] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:45.202136 14320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e6555f9875c4743a1680538e4c03821" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" is_pre_election: true
I20260709 12:28:45.202494 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e6555f9875c4743a1680538e4c03821" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
I20260709 12:28:45.203114 14320 raft_consensus.cc:2393] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe2b40322a94e769d80927662234903 in current term 1: Already voted for candidate a153a1160e9240439004094da9aa01ab in this term.
I20260709 12:28:45.204727 14434 leader_election.cc:304] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe2b40322a94e769d80927662234903; no voters: a153a1160e9240439004094da9aa01ab, e621e3baa9aa42ab963c4f3af3bca8b8
I20260709 12:28:45.206199 14514 raft_consensus.cc:3060] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.214385 14514 raft_consensus.cc:2749] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:45.238636 14514 raft_consensus.cc:493] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.239171 14514 raft_consensus.cc:515] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.241025 14514 leader_election.cc:290] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:45.242409 14320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" is_pre_election: true
I20260709 12:28:45.242365 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
I20260709 12:28:45.243553 14320 raft_consensus.cc:2468] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 0.
I20260709 12:28:45.243695 14395 raft_consensus.cc:2468] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 0.
I20260709 12:28:45.244980 14434 leader_election.cc:304] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:45.245921 14514 raft_consensus.cc:2804] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:45.246526 14514 raft_consensus.cc:493] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:45.246951 14514 raft_consensus.cc:3060] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.254432 14514 raft_consensus.cc:515] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.256527 14514 leader_election.cc:290] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:45.257737 14320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:45.257835 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:45.258545 14320 raft_consensus.cc:3060] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.258692 14395 raft_consensus.cc:3060] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.265094 14395 raft_consensus.cc:2468] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 1.
I20260709 12:28:45.266153 14320 raft_consensus.cc:2468] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 1.
I20260709 12:28:45.266289 14435 leader_election.cc:304] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: 
I20260709 12:28:45.267328 14514 raft_consensus.cc:2804] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:45.268445 14514 raft_consensus.cc:697] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Becoming Leader. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:45.269232 14514 consensus_queue.cc:237] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.278452 14215 catalog_manager.cc:5697] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 reported cstate change: term changed from 0 to 1, leader changed from <none> to abe2b40322a94e769d80927662234903 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "abe2b40322a94e769d80927662234903" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:45.322336 14514 raft_consensus.cc:493] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.323024 14514 raft_consensus.cc:515] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.325539 14514 leader_election.cc:290] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:45.326264 14320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad19be826494e9fa81d8b26a929ff4c" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" is_pre_election: true
I20260709 12:28:45.326362 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad19be826494e9fa81d8b26a929ff4c" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
I20260709 12:28:45.327123 14320 raft_consensus.cc:2468] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 0.
I20260709 12:28:45.327279 14395 raft_consensus.cc:2468] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 0.
I20260709 12:28:45.328675 14434 leader_election.cc:304] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:45.329486 14514 raft_consensus.cc:2804] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:45.329938 14514 raft_consensus.cc:493] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:45.330299 14514 raft_consensus.cc:3060] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.331332 14568 raft_consensus.cc:493] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.331938 14568 raft_consensus.cc:515] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.334425 14568 leader_election.cc:290] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:45.335626 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f62e820fa642a1bcd4095a04b01648" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
I20260709 12:28:45.335528 14320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f62e820fa642a1bcd4095a04b01648" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" is_pre_election: true
I20260709 12:28:45.336817 14395 raft_consensus.cc:2468] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 0.
I20260709 12:28:45.336833 14320 raft_consensus.cc:2468] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 0.
I20260709 12:28:45.338335 14435 leader_election.cc:304] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: 
I20260709 12:28:45.339232 14568 raft_consensus.cc:2804] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:45.339252 14514 raft_consensus.cc:515] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.339768 14568 raft_consensus.cc:493] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:45.340297 14568 raft_consensus.cc:3060] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.341795 14514 leader_election.cc:290] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:45.343180 14320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad19be826494e9fa81d8b26a929ff4c" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:45.343252 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad19be826494e9fa81d8b26a929ff4c" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:45.344026 14320 raft_consensus.cc:3060] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.344187 14395 raft_consensus.cc:3060] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.348909 14516 consensus_queue.cc:1048] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:45.349093 14568 raft_consensus.cc:515] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.351788 14568 leader_election.cc:290] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:45.353377 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f62e820fa642a1bcd4095a04b01648" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:45.354038 14320 raft_consensus.cc:2468] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 1.
I20260709 12:28:45.354362 14394 raft_consensus.cc:3060] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.354717 14319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f62e820fa642a1bcd4095a04b01648" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:45.354833 14395 raft_consensus.cc:2468] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 1.
I20260709 12:28:45.355763 14319 raft_consensus.cc:3060] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.358620 14435 leader_election.cc:304] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: 
I20260709 12:28:45.361302 14568 raft_consensus.cc:2804] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:45.363226 14394 raft_consensus.cc:2468] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 1.
I20260709 12:28:45.364763 14435 leader_election.cc:304] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: 
I20260709 12:28:45.365777 14514 raft_consensus.cc:2804] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:45.366339 14319 raft_consensus.cc:2468] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 1.
I20260709 12:28:45.366528 14514 raft_consensus.cc:697] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Becoming Leader. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:45.368415 14568 raft_consensus.cc:697] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Becoming Leader. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:45.367941 14514 consensus_queue.cc:237] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.369688 14568 consensus_queue.cc:237] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.377713 14542 consensus_queue.cc:1048] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:45.385738 14542 raft_consensus.cc:493] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.386399 14542 raft_consensus.cc:515] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.387574 14217 catalog_manager.cc:5697] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 reported cstate change: term changed from 0 to 1, leader changed from <none> to abe2b40322a94e769d80927662234903 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "abe2b40322a94e769d80927662234903" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:45.389194 14542 leader_election.cc:290] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a153a1160e9240439004094da9aa01ab (127.2.4.194:36325), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:45.391395 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903" is_pre_election: true
I20260709 12:28:45.392184 14466 raft_consensus.cc:2468] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in term 0.
I20260709 12:28:45.393325 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
I20260709 12:28:45.393525 14286 leader_election.cc:304] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:45.394178 14395 raft_consensus.cc:2468] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in term 0.
I20260709 12:28:45.394742 14542 raft_consensus.cc:2804] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:45.395216 14542 raft_consensus.cc:493] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:45.396175 14542 raft_consensus.cc:3060] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.403650 14568 raft_consensus.cc:493] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.405613 14568 raft_consensus.cc:515] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.407810 14542 raft_consensus.cc:515] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.409190 14568 leader_election.cc:290] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:45.410199 14319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec14432a4543eea89701c25066887b" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" is_pre_election: true
I20260709 12:28:45.411038 14319 raft_consensus.cc:2468] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 0.
I20260709 12:28:45.411366 14395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:45.412070 14395 raft_consensus.cc:3060] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.412966 14434 leader_election.cc:304] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:45.414705 14568 raft_consensus.cc:2804] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:45.411048 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec14432a4543eea89701c25066887b" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
I20260709 12:28:45.416092 14568 raft_consensus.cc:493] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:45.417124 14568 raft_consensus.cc:3060] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.417807 14542 leader_election.cc:290] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 election: Requested vote from peers a153a1160e9240439004094da9aa01ab (127.2.4.194:36325), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:45.417850 14530 consensus_queue.cc:1048] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:45.416649 14394 raft_consensus.cc:2468] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 0.
I20260709 12:28:45.422035 14530 consensus_queue.cc:1048] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:45.422190 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92" candidate_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903"
I20260709 12:28:45.422947 14466 raft_consensus.cc:3060] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.424666 14215 catalog_manager.cc:5697] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 reported cstate change: term changed from 0 to 1, leader changed from <none> to abe2b40322a94e769d80927662234903 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "abe2b40322a94e769d80927662234903" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:45.433408 14530 consensus_queue.cc:1048] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:45.434396 14466 raft_consensus.cc:2468] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in term 1.
I20260709 12:28:45.438622 14286 leader_election.cc:304] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:45.439620 14530 raft_consensus.cc:2804] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:45.441466 14568 raft_consensus.cc:515] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.447170 14319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec14432a4543eea89701c25066887b" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:45.452734 14319 raft_consensus.cc:3060] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.444998 14568 leader_election.cc:290] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:45.473573 14319 raft_consensus.cc:2468] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 1.
I20260709 12:28:45.477109 14434 leader_election.cc:304] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:45.478626 14568 raft_consensus.cc:2804] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:45.479028 14543 consensus_queue.cc:1048] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:45.479877 14568 raft_consensus.cc:697] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Becoming Leader. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:45.478310 14395 raft_consensus.cc:2468] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e621e3baa9aa42ab963c4f3af3bca8b8 in term 1.
I20260709 12:28:45.479700 14516 consensus_queue.cc:1048] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:45.481117 14568 consensus_queue.cc:237] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:45.490175 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec14432a4543eea89701c25066887b" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:45.490427 14530 raft_consensus.cc:697] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 LEADER]: Becoming Leader. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Running, Role: LEADER
I20260709 12:28:45.499334 14530 consensus_queue.cc:237] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
W20260709 12:28:45.501513 14266 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:45.525637 14543 consensus_queue.cc:1048] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:45.495806 14394 raft_consensus.cc:3060] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.558997 14215 catalog_manager.cc:5697] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 reported cstate change: term changed from 0 to 1, leader changed from <none> to abe2b40322a94e769d80927662234903 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "abe2b40322a94e769d80927662234903" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:45.571012 14394 raft_consensus.cc:2468] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 1.
I20260709 12:28:45.575240 14218 catalog_manager.cc:5697] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:45.710968 14568 consensus_queue.cc:1048] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:45.728044 14570 consensus_queue.cc:1048] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:45.760283 14570 consensus_queue.cc:1048] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:45.763176 14570 consensus_queue.cc:1048] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:45.881961  2067 ts_manager.cc:295] Set tserver state for e621e3baa9aa42ab963c4f3af3bca8b8 to MAINTENANCE_MODE
I20260709 12:28:45.883010  2067 tablet_server.cc:179] TabletServer@127.2.4.193:0 shutting down...
I20260709 12:28:45.911891 14592 consensus_queue.cc:1048] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:45.918169  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:45.919181  2067 tablet_replica.cc:333] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8: stopping tablet replica
I20260709 12:28:45.919960  2067 raft_consensus.cc:2243] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.920454  2067 raft_consensus.cc:2272] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.923452  2067 tablet_replica.cc:333] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8: stopping tablet replica
I20260709 12:28:45.924427  2067 raft_consensus.cc:2243] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.925030  2067 raft_consensus.cc:2272] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.932456  2067 tablet_replica.cc:333] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8: stopping tablet replica
I20260709 12:28:45.933697  2067 raft_consensus.cc:2243] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:45.933537 14568 consensus_queue.cc:1048] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:45.935195  2067 raft_consensus.cc:2272] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.940330  2067 tablet_replica.cc:333] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8: stopping tablet replica
I20260709 12:28:45.941303  2067 raft_consensus.cc:2243] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:45.942653  2067 raft_consensus.cc:2272] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:45.945719 14434 consensus_peers.cc:597] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:45.947058  2067 tablet_replica.cc:333] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8: stopping tablet replica
I20260709 12:28:45.948082  2067 raft_consensus.cc:2243] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:45.949329  2067 raft_consensus.cc:2272] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:45.949484 14434 consensus_peers.cc:597] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:45.953182  2067 tablet_replica.cc:333] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8: stopping tablet replica
I20260709 12:28:45.954519  2067 raft_consensus.cc:2243] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.955310  2067 raft_consensus.cc:2272] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.959784  2067 tablet_replica.cc:333] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8: stopping tablet replica
I20260709 12:28:45.960973  2067 raft_consensus.cc:2243] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:45.962185  2067 pending_rounds.cc:70] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8: Trying to abort 1 pending ops.
I20260709 12:28:45.962539  2067 pending_rounds.cc:77] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305626114061975552 op_type: NO_OP noop_request { }
I20260709 12:28:45.963050  2067 raft_consensus.cc:2889] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:45.963485  2067 raft_consensus.cc:2272] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:45.966411 14360 consensus_peers.cc:597] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:45.966938  2067 tablet_replica.cc:333] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8: stopping tablet replica
I20260709 12:28:45.967707  2067 raft_consensus.cc:2243] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.968235  2067 raft_consensus.cc:2272] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.970455  2067 tablet_replica.cc:333] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8: stopping tablet replica
I20260709 12:28:45.971055  2067 raft_consensus.cc:2243] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:45.971536  2067 raft_consensus.cc:2272] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:45.974378  2067 tablet_replica.cc:333] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8: stopping tablet replica
I20260709 12:28:45.974988  2067 raft_consensus.cc:2243] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:45.975920  2067 raft_consensus.cc:2272] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:46.010684  2067 tablet_server.cc:196] TabletServer@127.2.4.193:0 shutdown complete.
W20260709 12:28:46.159749 14434 consensus_peers.cc:597] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:46.241145 14434 consensus_peers.cc:597] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:46.504755 14266 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:46.879703 14514 raft_consensus.cc:493] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e621e3baa9aa42ab963c4f3af3bca8b8)
I20260709 12:28:46.880343 14514 raft_consensus.cc:515] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } }
I20260709 12:28:46.882349 14514 leader_election.cc:290] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a153a1160e9240439004094da9aa01ab (127.2.4.194:36325), e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175)
I20260709 12:28:46.884192 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6dcc07ba9784ae98d3974c6abb1b53b" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
W20260709 12:28:46.889209 14434 leader_election.cc:336] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111)
I20260709 12:28:46.889804 14434 leader_election.cc:304] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe2b40322a94e769d80927662234903; no voters: a153a1160e9240439004094da9aa01ab, e621e3baa9aa42ab963c4f3af3bca8b8
I20260709 12:28:46.890620 14514 raft_consensus.cc:2749] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:46.964085 14599 raft_consensus.cc:493] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e621e3baa9aa42ab963c4f3af3bca8b8)
I20260709 12:28:46.964756 14599 raft_consensus.cc:515] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:46.967314 14599 leader_election.cc:290] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:46.968850 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "354b6360e1104e169aaee9531ae2c3a6" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903" is_pre_election: true
W20260709 12:28:46.971807 14360 leader_election.cc:336] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111)
I20260709 12:28:46.972250 14360 leader_election.cc:304] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a153a1160e9240439004094da9aa01ab; no voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8
I20260709 12:28:46.973017 14599 raft_consensus.cc:2749] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:46.987279 14602 raft_consensus.cc:493] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.987932 14602 raft_consensus.cc:515] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:46.989984 14602 leader_election.cc:290] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:46.991484 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
I20260709 12:28:46.992398 14394 raft_consensus.cc:2468] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 1.
I20260709 12:28:46.993686 14435 leader_election.cc:304] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: 
I20260709 12:28:46.994534 14602 raft_consensus.cc:2804] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:46.994922 14602 raft_consensus.cc:493] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:46.994853 14434 leader_election.cc:336] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111)
I20260709 12:28:46.995581 14602 raft_consensus.cc:3060] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:47.003423 14602 raft_consensus.cc:515] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:47.005326 14602 leader_election.cc:290] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:47.006963 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:47.007767 14394 raft_consensus.cc:3060] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:47.009064 14599 raft_consensus.cc:493] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e621e3baa9aa42ab963c4f3af3bca8b8)
I20260709 12:28:47.009809 14599 raft_consensus.cc:515] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
W20260709 12:28:47.010066 14434 leader_election.cc:336] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111)
I20260709 12:28:47.013449 14599 leader_election.cc:290] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:47.014866 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "320069d83c9f4143869dd90f6d253134" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903" is_pre_election: true
I20260709 12:28:47.016444 14394 raft_consensus.cc:2468] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 2.
W20260709 12:28:47.017405 14360 leader_election.cc:336] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111)
I20260709 12:28:47.017973 14435 leader_election.cc:304] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: e621e3baa9aa42ab963c4f3af3bca8b8
I20260709 12:28:47.018095 14360 leader_election.cc:304] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a153a1160e9240439004094da9aa01ab; no voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8
I20260709 12:28:47.019059 14602 raft_consensus.cc:2804] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:47.019157 14599 raft_consensus.cc:2749] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:47.019670 14602 raft_consensus.cc:697] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 2 LEADER]: Becoming Leader. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:47.020635 14602 consensus_queue.cc:237] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:47.028954 14215 catalog_manager.cc:5697] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 reported cstate change: term changed from 1 to 2, leader changed from e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193) to abe2b40322a94e769d80927662234903 (127.2.4.195). New cstate: current_term: 2 leader_uuid: "abe2b40322a94e769d80927662234903" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:47.056577 14602 raft_consensus.cc:493] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e621e3baa9aa42ab963c4f3af3bca8b8)
I20260709 12:28:47.057368 14602 raft_consensus.cc:515] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:47.059834 14602 leader_election.cc:290] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:47.061262 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "320069d83c9f4143869dd90f6d253134" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
I20260709 12:28:47.062103 14394 raft_consensus.cc:2468] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 1.
I20260709 12:28:47.063292 14435 leader_election.cc:304] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: 
I20260709 12:28:47.064194 14602 raft_consensus.cc:2804] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:28:47.064219 14434 leader_election.cc:336] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111)
I20260709 12:28:47.064636 14602 raft_consensus.cc:493] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e621e3baa9aa42ab963c4f3af3bca8b8)
I20260709 12:28:47.065089 14602 raft_consensus.cc:3060] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:47.071545 14602 raft_consensus.cc:515] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:47.073551 14602 leader_election.cc:290] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:47.074887 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "320069d83c9f4143869dd90f6d253134" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:47.075537 14394 raft_consensus.cc:3060] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:47.077715 14434 leader_election.cc:336] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111)
I20260709 12:28:47.081705 14394 raft_consensus.cc:2468] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 2.
I20260709 12:28:47.083098 14435 leader_election.cc:304] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: e621e3baa9aa42ab963c4f3af3bca8b8
I20260709 12:28:47.083964 14602 raft_consensus.cc:2804] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:47.084492 14602 raft_consensus.cc:697] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 2 LEADER]: Becoming Leader. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:47.085284 14602 consensus_queue.cc:237] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:47.094005 14215 catalog_manager.cc:5697] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 reported cstate change: term changed from 1 to 2, leader changed from e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193) to abe2b40322a94e769d80927662234903 (127.2.4.195). New cstate: current_term: 2 leader_uuid: "abe2b40322a94e769d80927662234903" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:47.146132 14514 raft_consensus.cc:493] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e621e3baa9aa42ab963c4f3af3bca8b8)
I20260709 12:28:47.146911 14514 raft_consensus.cc:515] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:47.149050 14514 leader_election.cc:290] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
W20260709 12:28:47.150625 14434 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111) [suppressed 36 similar messages]
I20260709 12:28:47.150457 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "527f315df63447e3bc065eb01506adec" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
W20260709 12:28:47.153671 14434 leader_election.cc:336] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111)
I20260709 12:28:47.154171 14434 leader_election.cc:304] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe2b40322a94e769d80927662234903; no voters: a153a1160e9240439004094da9aa01ab, e621e3baa9aa42ab963c4f3af3bca8b8
I20260709 12:28:47.154939 14514 raft_consensus.cc:2749] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:47.283033 14599 raft_consensus.cc:493] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e621e3baa9aa42ab963c4f3af3bca8b8)
I20260709 12:28:47.283645 14599 raft_consensus.cc:515] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:47.285753 14599 leader_election.cc:290] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:47.288694 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "527f315df63447e3bc065eb01506adec" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903" is_pre_election: true
I20260709 12:28:47.289412 14466 raft_consensus.cc:2468] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 1.
I20260709 12:28:47.290737 14361 leader_election.cc:304] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: 
W20260709 12:28:47.291256 14360 leader_election.cc:336] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111)
I20260709 12:28:47.291517 14609 raft_consensus.cc:2804] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:47.292066 14609 raft_consensus.cc:493] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Starting leader election (detected failure of leader e621e3baa9aa42ab963c4f3af3bca8b8)
I20260709 12:28:47.292429 14609 raft_consensus.cc:3060] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:47.298810 14609 raft_consensus.cc:515] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:47.300777 14609 leader_election.cc:290] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:47.302134 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "527f315df63447e3bc065eb01506adec" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903"
I20260709 12:28:47.302836 14466 raft_consensus.cc:3060] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:47.304894 14360 leader_election.cc:336] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111)
I20260709 12:28:47.308835 14466 raft_consensus.cc:2468] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 2.
I20260709 12:28:47.310305 14361 leader_election.cc:304] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: e621e3baa9aa42ab963c4f3af3bca8b8
I20260709 12:28:47.311275 14609 raft_consensus.cc:2804] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:47.311759 14609 raft_consensus.cc:697] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Becoming Leader. State: Replica: a153a1160e9240439004094da9aa01ab, State: Running, Role: LEADER
I20260709 12:28:47.312528 14609 consensus_queue.cc:237] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:47.320936 14215 catalog_manager.cc:5697] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab reported cstate change: term changed from 1 to 2, leader changed from e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193) to a153a1160e9240439004094da9aa01ab (127.2.4.194). New cstate: current_term: 2 leader_uuid: "a153a1160e9240439004094da9aa01ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:47.355099 14609 raft_consensus.cc:493] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e621e3baa9aa42ab963c4f3af3bca8b8)
I20260709 12:28:47.355877 14609 raft_consensus.cc:515] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } }
I20260709 12:28:47.359146 14609 leader_election.cc:290] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers abe2b40322a94e769d80927662234903 (127.2.4.195:37025), e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175)
I20260709 12:28:47.359447 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6dcc07ba9784ae98d3974c6abb1b53b" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903" is_pre_election: true
I20260709 12:28:47.360359 14466 raft_consensus.cc:2468] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 1.
I20260709 12:28:47.361513 14361 leader_election.cc:304] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: 
I20260709 12:28:47.362357 14609 raft_consensus.cc:2804] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:47.362769 14609 raft_consensus.cc:493] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Starting leader election (detected failure of leader e621e3baa9aa42ab963c4f3af3bca8b8)
I20260709 12:28:47.363071 14609 raft_consensus.cc:3060] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:47.363472 14360 leader_election.cc:336] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111)
I20260709 12:28:47.368818 14609 raft_consensus.cc:515] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } }
I20260709 12:28:47.370754 14609 leader_election.cc:290] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 election: Requested vote from peers abe2b40322a94e769d80927662234903 (127.2.4.195:37025), e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175)
I20260709 12:28:47.371811 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6dcc07ba9784ae98d3974c6abb1b53b" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "abe2b40322a94e769d80927662234903"
I20260709 12:28:47.372375 14466 raft_consensus.cc:3060] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:47.374814 14360 leader_election.cc:336] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111)
I20260709 12:28:47.377884 14466 raft_consensus.cc:2468] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 2.
I20260709 12:28:47.379364 14361 leader_election.cc:304] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: e621e3baa9aa42ab963c4f3af3bca8b8
I20260709 12:28:47.381045 14609 raft_consensus.cc:2804] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:47.381800 14609 raft_consensus.cc:697] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Becoming Leader. State: Replica: a153a1160e9240439004094da9aa01ab, State: Running, Role: LEADER
I20260709 12:28:47.383052 14609 consensus_queue.cc:237] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } }
I20260709 12:28:47.392212 14215 catalog_manager.cc:5697] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab reported cstate change: term changed from 1 to 2, leader changed from e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193) to a153a1160e9240439004094da9aa01ab (127.2.4.194). New cstate: current_term: 2 leader_uuid: "a153a1160e9240439004094da9aa01ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:47.416683 14514 raft_consensus.cc:493] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e621e3baa9aa42ab963c4f3af3bca8b8)
I20260709 12:28:47.417342 14514 raft_consensus.cc:515] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:47.419402 14514 leader_election.cc:290] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:47.420579 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "354b6360e1104e169aaee9531ae2c3a6" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab" is_pre_election: true
I20260709 12:28:47.421268 14394 raft_consensus.cc:2468] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 1.
I20260709 12:28:47.422400 14435 leader_election.cc:304] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: 
I20260709 12:28:47.423367 14514 raft_consensus.cc:2804] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:28:47.423727 14434 leader_election.cc:336] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111)
I20260709 12:28:47.423764 14514 raft_consensus.cc:493] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e621e3baa9aa42ab963c4f3af3bca8b8)
I20260709 12:28:47.424456 14514 raft_consensus.cc:3060] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:47.430101 14514 raft_consensus.cc:515] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:47.431910 14514 leader_election.cc:290] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:47.433086 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "354b6360e1104e169aaee9531ae2c3a6" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:47.433729 14394 raft_consensus.cc:3060] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:47.435931 14434 leader_election.cc:336] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111)
I20260709 12:28:47.439469 14394 raft_consensus.cc:2468] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 2.
I20260709 12:28:47.440913 14435 leader_election.cc:304] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: e621e3baa9aa42ab963c4f3af3bca8b8
I20260709 12:28:47.441731 14514 raft_consensus.cc:2804] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:47.442397 14514 raft_consensus.cc:697] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 2 LEADER]: Becoming Leader. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:47.443317 14514 consensus_queue.cc:237] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:47.452064 14215 catalog_manager.cc:5697] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 reported cstate change: term changed from 1 to 2, leader changed from e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193) to abe2b40322a94e769d80927662234903 (127.2.4.195). New cstate: current_term: 2 leader_uuid: "abe2b40322a94e769d80927662234903" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:47.483706 14434 consensus_peers.cc:597] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.506201 14266 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:47.582787 14394 raft_consensus.cc:1275] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Refusing update from remote peer abe2b40322a94e769d80927662234903: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:28:47.584196 14514 consensus_queue.cc:1048] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:28:47.614495 14434 consensus_peers.cc:597] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.700699 14466 raft_consensus.cc:1275] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:47.702358 14609 consensus_queue.cc:1048] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:47.713819 14394 raft_consensus.cc:1275] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Refusing update from remote peer abe2b40322a94e769d80927662234903: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:47.716073 14602 consensus_queue.cc:1048] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:28:47.718184 14360 consensus_peers.cc:597] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.812855 14360 consensus_peers.cc:597] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.819331 14466 raft_consensus.cc:1275] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:47.820791 14599 consensus_queue.cc:1048] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:47.838312 14394 raft_consensus.cc:1275] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Refusing update from remote peer abe2b40322a94e769d80927662234903: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:47.839836 14602 consensus_queue.cc:1048] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:28:47.854910 14434 consensus_peers.cc:597] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:48.372617 14434 consensus_peers.cc:597] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.431236 14360 consensus_peers.cc:597] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.507441 14266 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:28:48.527913 14434 consensus_peers.cc:597] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.634487 14434 consensus_peers.cc:597] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:48.849934 14434 consensus_peers.cc:597] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:49.508843 14266 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:28:49.998533 14434 consensus_peers.cc:597] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:50.152503 14434 consensus_peers.cc:597] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:50.242513 14360 consensus_peers.cc:597] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:50.315989 14434 consensus_peers.cc:597] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:50.428030 14360 consensus_peers.cc:597] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:50.510224 14266 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:28:50.850390 14434 consensus_peers.cc:597] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:50.921048 14360 consensus_peers.cc:597] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:50.933862 14434 consensus_peers.cc:597] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:51.035370  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:51.042984 14625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:51.043625 14626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:51.045466 14628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:51.055080  2067 server_base.cc:1061] running on GCE node
I20260709 12:28:51.056313  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:51.056658  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:51.056883  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600131056862 us; error 0 us; skew 500 ppm
I20260709 12:28:51.057538  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:51.061250  2067 webserver.cc:533] Webserver started at http://127.2.4.193:41745/ using document root <none> and password file <none>
I20260709 12:28:51.061930  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:51.062146  2067 fs_manager.cc:365] Using existing metadata directory in first data directory
W20260709 12:28:51.066478 14434 consensus_peers.cc:597] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:51.067080  2067 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:28:51.071938 14633 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:51.073191  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:28:51.073733  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-0-root
uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
format_stamp: "Formatted at 2026-07-09 12:28:43 on dist-test-slave-4qvp"
I20260709 12:28:51.074069  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599727094521-2067-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:51.114014  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:51.115463  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:51.117100  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:51.120875 14640 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 12:28:51.184132  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:28:51.184527  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.065s	user 0.003s	sys 0.000s
I20260709 12:28:51.184888  2067 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:28:51.191888 14640 tablet_bootstrap.cc:492] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:51.209089 14640 tablet_bootstrap.cc:492] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:28:51.210202 14640 tablet_bootstrap.cc:492] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap complete.
I20260709 12:28:51.210841 14640 ts_tablet_manager.cc:1403] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 12:28:51.213228 14640 raft_consensus.cc:359] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.214067 14640 raft_consensus.cc:740] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.214838 14640 consensus_queue.cc:260] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.217420 14640 ts_tablet_manager.cc:1434] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:28:51.218281 14640 tablet_bootstrap.cc:492] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:51.232188 14640 tablet_bootstrap.cc:492] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:51.232995  2067 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:28:51.233070 14640 tablet_bootstrap.cc:492] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap complete.
I20260709 12:28:51.233438  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.049s	user 0.046s	sys 0.000s
I20260709 12:28:51.234236 14640 ts_tablet_manager.cc:1403] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 12:28:51.236586 14640 raft_consensus.cc:359] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.237212 14640 raft_consensus.cc:740] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.237983 14640 consensus_queue.cc:260] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.239835 14640 ts_tablet_manager.cc:1434] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:28:51.240792 14640 tablet_bootstrap.cc:492] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:51.258612 14640 tablet_bootstrap.cc:492] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:51.259811 14640 tablet_bootstrap.cc:492] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap complete.
I20260709 12:28:51.260432 14640 ts_tablet_manager.cc:1403] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.003s
I20260709 12:28:51.263161 14640 raft_consensus.cc:359] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.263775 14640 raft_consensus.cc:740] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.264510 14640 consensus_queue.cc:260] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.266466 14640 ts_tablet_manager.cc:1434] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:28:51.267167 14640 tablet_bootstrap.cc:492] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:51.278508 14640 tablet_bootstrap.cc:492] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:51.279448 14640 tablet_bootstrap.cc:492] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap complete.
I20260709 12:28:51.280262 14640 ts_tablet_manager.cc:1403] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.013s	user 0.012s	sys 0.000s
W20260709 12:28:51.282451 14434 consensus_peers.cc:597] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:51.282714 14640 raft_consensus.cc:359] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.283432 14640 raft_consensus.cc:740] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.284235 14640 consensus_queue.cc:260] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.285959 14640 ts_tablet_manager.cc:1434] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:28:51.286639 14640 tablet_bootstrap.cc:492] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:51.301021 14640 tablet_bootstrap.cc:492] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:51.302552 14640 tablet_bootstrap.cc:492] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap complete.
I20260709 12:28:51.303651 14640 ts_tablet_manager.cc:1403] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.017s	user 0.016s	sys 0.000s
I20260709 12:28:51.305985  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.193:43175
I20260709 12:28:51.306123 14702 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.193:43175 every 8 connection(s)
I20260709 12:28:51.306990 14640 raft_consensus.cc:359] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.307761 14640 raft_consensus.cc:740] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.308535 14640 consensus_queue.cc:260] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.311322 14640 ts_tablet_manager.cc:1434] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:28:51.312119  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:51.312346 14640 tablet_bootstrap.cc:492] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:51.330786 14640 tablet_bootstrap.cc:492] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:51.332042 14640 tablet_bootstrap.cc:492] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap complete.
I20260709 12:28:51.332885 14640 ts_tablet_manager.cc:1403] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.004s
I20260709 12:28:51.335062 14703 heartbeater.cc:344] Connected to a master server at 127.2.4.254:35633
I20260709 12:28:51.335609 14703 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:51.335860 14640 raft_consensus.cc:359] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.336621 14640 raft_consensus.cc:740] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.336678 14703 heartbeater.cc:507] Master 127.2.4.254:35633 requested a full tablet report, sending...
I20260709 12:28:51.337589 14640 consensus_queue.cc:260] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.340649 14640 ts_tablet_manager.cc:1434] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:51.341657 14640 tablet_bootstrap.cc:492] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:51.343492 14466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f"
dest_uuid: "abe2b40322a94e769d80927662234903"
mode: GRACEFUL
new_leader_uuid: "a153a1160e9240439004094da9aa01ab"
 from {username='slave'} at 127.0.0.1:35678
I20260709 12:28:51.344271 14466 raft_consensus.cc:606] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Received request to transfer leadership to TS a153a1160e9240439004094da9aa01ab
I20260709 12:28:51.347456 14215 ts_manager.cc:194] Re-registered known tserver with Master: e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175)
I20260709 12:28:51.347450 14466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92"
dest_uuid: "abe2b40322a94e769d80927662234903"
mode: GRACEFUL
new_leader_uuid: "a153a1160e9240439004094da9aa01ab"
 from {username='slave'} at 127.0.0.1:35678
I20260709 12:28:51.348192 14466 raft_consensus.cc:606] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 2 LEADER]: Received request to transfer leadership to TS a153a1160e9240439004094da9aa01ab
I20260709 12:28:51.351125 14466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ad19be826494e9fa81d8b26a929ff4c"
dest_uuid: "abe2b40322a94e769d80927662234903"
mode: GRACEFUL
new_leader_uuid: "a153a1160e9240439004094da9aa01ab"
 from {username='slave'} at 127.0.0.1:35678
I20260709 12:28:51.351725 14466 raft_consensus.cc:606] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Received request to transfer leadership to TS a153a1160e9240439004094da9aa01ab
I20260709 12:28:51.352908  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:28:51.353498  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:51.354298  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:51.358276 14640 tablet_bootstrap.cc:492] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:51.359413 14640 tablet_bootstrap.cc:492] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap complete.
I20260709 12:28:51.360232 14640 ts_tablet_manager.cc:1403] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.005s
I20260709 12:28:51.363353 14640 raft_consensus.cc:359] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.364244 14640 raft_consensus.cc:740] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.364363 14215 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41680
I20260709 12:28:51.365085 14640 consensus_queue.cc:260] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.367735 14640 ts_tablet_manager.cc:1434] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:51.368608 14703 heartbeater.cc:499] Master 127.2.4.254:35633 was elected leader, sending a full tablet report...
I20260709 12:28:51.368837 14640 tablet_bootstrap.cc:492] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:51.390734 14712 raft_consensus.cc:993] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903: : Instructing follower a153a1160e9240439004094da9aa01ab to start an election
I20260709 12:28:51.391503 14712 raft_consensus.cc:1081] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 2 LEADER]: Signalling peer a153a1160e9240439004094da9aa01ab to start an election
I20260709 12:28:51.394338 14394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92"
dest_uuid: "a153a1160e9240439004094da9aa01ab"
 from {username='slave'} at 127.0.0.1:42562
I20260709 12:28:51.395005 14394 raft_consensus.cc:493] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:51.395329 14394 raft_consensus.cc:3060] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:51.395857 14640 tablet_bootstrap.cc:492] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:51.396793 14640 tablet_bootstrap.cc:492] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap complete.
I20260709 12:28:51.397428 14640 ts_tablet_manager.cc:1403] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.000s
I20260709 12:28:51.400583 14640 raft_consensus.cc:359] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } }
I20260709 12:28:51.401383 14640 raft_consensus.cc:740] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.403820 14676 raft_consensus.cc:3060] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:51.403913 14394 raft_consensus.cc:515] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.402365 14640 consensus_queue.cc:260] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } } peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } }
I20260709 12:28:51.410256 14640 ts_tablet_manager.cc:1434] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:28:51.411315 14640 tablet_bootstrap.cc:492] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:51.413915 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "abe2b40322a94e769d80927662234903"
I20260709 12:28:51.414619 14466 raft_consensus.cc:3055] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:51.415122 14466 raft_consensus.cc:740] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:51.416054 14466 consensus_queue.cc:260] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.417351 14466 raft_consensus.cc:3060] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:51.425712 14675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:51.426507 14675 raft_consensus.cc:3060] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:28:51.431588 14466 raft_consensus.cc:2468] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 3.
I20260709 12:28:51.414597 14394 leader_election.cc:290] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 3 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:51.435608 14361 leader_election.cc:304] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: 
I20260709 12:28:51.436661 14599 raft_consensus.cc:2804] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:51.437114 14675 raft_consensus.cc:2468] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 3.
I20260709 12:28:51.437937 14640 tablet_bootstrap.cc:492] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:51.439033 14640 tablet_bootstrap.cc:492] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap complete.
I20260709 12:28:51.439741 14640 ts_tablet_manager.cc:1403] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.029s	user 0.025s	sys 0.000s
I20260709 12:28:51.441028 14599 raft_consensus.cc:697] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 3 LEADER]: Becoming Leader. State: Replica: a153a1160e9240439004094da9aa01ab, State: Running, Role: LEADER
I20260709 12:28:51.442374 14640 raft_consensus.cc:359] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.442739 14599 consensus_queue.cc:237] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.443226 14640 raft_consensus.cc:740] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.444145 14640 consensus_queue.cc:260] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.446278 14640 ts_tablet_manager.cc:1434] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:28:51.447103 14640 tablet_bootstrap.cc:492] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap starting.
I20260709 12:28:51.453285 14217 catalog_manager.cc:5697] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab reported cstate change: term changed from 2 to 3, leader changed from abe2b40322a94e769d80927662234903 (127.2.4.195) to a153a1160e9240439004094da9aa01ab (127.2.4.194). New cstate: current_term: 3 leader_uuid: "a153a1160e9240439004094da9aa01ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:51.464715 14640 tablet_bootstrap.cc:492] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:51.466035 14640 tablet_bootstrap.cc:492] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8: Bootstrap complete.
I20260709 12:28:51.466784 14640 ts_tablet_manager.cc:1403] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.007s
I20260709 12:28:51.469887 14640 raft_consensus.cc:359] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.470654 14640 raft_consensus.cc:740] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621e3baa9aa42ab963c4f3af3bca8b8, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.471573 14640 consensus_queue.cc:260] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.474476 14640 ts_tablet_manager.cc:1434] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
W20260709 12:28:51.528218 14266 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:51.539083 14675 raft_consensus.cc:3060] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:51.543751 14716 raft_consensus.cc:993] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903: : Instructing follower a153a1160e9240439004094da9aa01ab to start an election
I20260709 12:28:51.544795 14712 raft_consensus.cc:1081] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Signalling peer a153a1160e9240439004094da9aa01ab to start an election
I20260709 12:28:51.547168 14394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ad19be826494e9fa81d8b26a929ff4c"
dest_uuid: "a153a1160e9240439004094da9aa01ab"
 from {username='slave'} at 127.0.0.1:42562
I20260709 12:28:51.548112 14394 raft_consensus.cc:493] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:51.548624 14394 raft_consensus.cc:3060] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:51.561537 14394 raft_consensus.cc:515] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.567800 14675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad19be826494e9fa81d8b26a929ff4c" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:51.568727 14394 leader_election.cc:290] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:51.569133 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad19be826494e9fa81d8b26a929ff4c" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "abe2b40322a94e769d80927662234903"
I20260709 12:28:51.570328 14466 raft_consensus.cc:3055] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:51.569582 14675 raft_consensus.cc:3060] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:51.570868 14466 raft_consensus.cc:740] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:51.573568 14466 consensus_queue.cc:260] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.576531 14466 raft_consensus.cc:3060] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:51.584688 14675 raft_consensus.cc:2468] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 2.
I20260709 12:28:51.586519 14360 leader_election.cc:304] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:51.587579 14599 raft_consensus.cc:2804] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:51.588040 14599 raft_consensus.cc:697] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Becoming Leader. State: Replica: a153a1160e9240439004094da9aa01ab, State: Running, Role: LEADER
I20260709 12:28:51.589323 14599 consensus_queue.cc:237] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.589620 14466 raft_consensus.cc:2468] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 2.
I20260709 12:28:51.599145 14215 catalog_manager.cc:5697] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab reported cstate change: term changed from 1 to 2, leader changed from abe2b40322a94e769d80927662234903 (127.2.4.195) to a153a1160e9240439004094da9aa01ab (127.2.4.194). New cstate: current_term: 2 leader_uuid: "a153a1160e9240439004094da9aa01ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:51.605289 14675 raft_consensus.cc:3060] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:51.758570 14675 raft_consensus.cc:3060] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:51.835474 14621 raft_consensus.cc:993] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903: : Instructing follower a153a1160e9240439004094da9aa01ab to start an election
I20260709 12:28:51.838028 14716 raft_consensus.cc:1081] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Signalling peer a153a1160e9240439004094da9aa01ab to start an election
I20260709 12:28:51.841244 14394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f"
dest_uuid: "a153a1160e9240439004094da9aa01ab"
 from {username='slave'} at 127.0.0.1:42562
I20260709 12:28:51.842121 14394 raft_consensus.cc:493] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:51.842669 14394 raft_consensus.cc:3060] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:51.851750 14394 raft_consensus.cc:515] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.853986 14394 leader_election.cc:290] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:51.854857 14675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:51.855275 14466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "abe2b40322a94e769d80927662234903"
I20260709 12:28:51.855816 14675 raft_consensus.cc:3060] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:51.856201 14466 raft_consensus.cc:3055] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:51.856781 14466 raft_consensus.cc:740] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:51.858227 14466 consensus_queue.cc:260] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.860116 14466 raft_consensus.cc:3060] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:51.865685 14675 raft_consensus.cc:2468] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 2.
I20260709 12:28:51.867554 14360 leader_election.cc:304] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:51.868700 14599 raft_consensus.cc:2804] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:51.869243 14599 raft_consensus.cc:697] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Becoming Leader. State: Replica: a153a1160e9240439004094da9aa01ab, State: Running, Role: LEADER
I20260709 12:28:51.870309 14466 raft_consensus.cc:2468] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 2.
I20260709 12:28:51.870462 14599 consensus_queue.cc:237] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:51.879446 14215 catalog_manager.cc:5697] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab reported cstate change: term changed from 1 to 2, leader changed from abe2b40322a94e769d80927662234903 (127.2.4.195) to a153a1160e9240439004094da9aa01ab (127.2.4.194). New cstate: current_term: 2 leader_uuid: "a153a1160e9240439004094da9aa01ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:51.898304 14675 raft_consensus.cc:1275] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 3 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:51.900189 14719 consensus_queue.cc:1048] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:51.900969 14719 consensus_queue.cc:1243] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [LEADER]: Peer e621e3baa9aa42ab963c4f3af3bca8b8 log is divergent from this leader: its last log entry 1.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260709 12:28:51.903748 14675 pending_rounds.cc:85] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8: Aborting all ops after (but not including) 0
I20260709 12:28:51.904039 14675 pending_rounds.cc:107] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:28:51.904321 14675 raft_consensus.cc:2889] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:51.916368 14466 raft_consensus.cc:1275] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 3 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:51.918807 14599 consensus_queue.cc:1048] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:52.112772 14675 raft_consensus.cc:1275] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:52.114289 14599 consensus_queue.cc:1048] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:52.131594 14466 raft_consensus.cc:1275] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:52.133579 14599 consensus_queue.cc:1048] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:52.363929  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:52.389024 14394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f"
dest_uuid: "a153a1160e9240439004094da9aa01ab"
mode: GRACEFUL
new_leader_uuid: "abe2b40322a94e769d80927662234903"
 from {username='slave'} at 127.0.0.1:41588
I20260709 12:28:52.389760 14394 raft_consensus.cc:606] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Received request to transfer leadership to TS abe2b40322a94e769d80927662234903
I20260709 12:28:52.392920 14394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92"
dest_uuid: "a153a1160e9240439004094da9aa01ab"
mode: GRACEFUL
new_leader_uuid: "abe2b40322a94e769d80927662234903"
 from {username='slave'} at 127.0.0.1:41588
I20260709 12:28:52.393500 14394 raft_consensus.cc:606] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 3 LEADER]: Received request to transfer leadership to TS abe2b40322a94e769d80927662234903
I20260709 12:28:52.394804  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:28:52.395505  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:52.396406  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:52.423848 14466 raft_consensus.cc:1275] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:52.426519 14599 consensus_queue.cc:1048] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:52.434444 14739 raft_consensus.cc:993] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab: : Instructing follower abe2b40322a94e769d80927662234903 to start an election
I20260709 12:28:52.434975 14739 raft_consensus.cc:1081] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Signalling peer abe2b40322a94e769d80927662234903 to start an election
I20260709 12:28:52.437034 14676 raft_consensus.cc:1275] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:52.439314 14468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f"
dest_uuid: "abe2b40322a94e769d80927662234903"
 from {username='slave'} at 127.0.0.1:36130
I20260709 12:28:52.439713 14739 consensus_queue.cc:1048] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:52.442803 14468 raft_consensus.cc:493] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:52.443286 14468 raft_consensus.cc:3060] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:52.456365 14468 raft_consensus.cc:515] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:52.461454 14468 leader_election.cc:290] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 3 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:52.461894 14676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:52.462278 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:52.462769 14676 raft_consensus.cc:3060] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:52.463042 14394 raft_consensus.cc:3055] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:52.463470 14394 raft_consensus.cc:740] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a153a1160e9240439004094da9aa01ab, State: Running, Role: LEADER
I20260709 12:28:52.464287 14394 consensus_queue.cc:260] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:52.465344 14394 raft_consensus.cc:3060] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:52.473709 14394 raft_consensus.cc:2468] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 3.
I20260709 12:28:52.474360 14676 raft_consensus.cc:2468] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 3.
I20260709 12:28:52.475641 14435 leader_election.cc:304] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: 
I20260709 12:28:52.476619 14621 raft_consensus.cc:2804] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:52.477371 14621 raft_consensus.cc:697] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 3 LEADER]: Becoming Leader. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:52.478507 14621 consensus_queue.cc:237] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:52.488644 14719 raft_consensus.cc:993] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab: : Instructing follower abe2b40322a94e769d80927662234903 to start an election
I20260709 12:28:52.487779 14217 catalog_manager.cc:5697] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 reported cstate change: term changed from 2 to 3, leader changed from a153a1160e9240439004094da9aa01ab (127.2.4.194) to abe2b40322a94e769d80927662234903 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "abe2b40322a94e769d80927662234903" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:52.489420 14599 raft_consensus.cc:1081] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 3 LEADER]: Signalling peer abe2b40322a94e769d80927662234903 to start an election
I20260709 12:28:52.491567 14468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92"
dest_uuid: "abe2b40322a94e769d80927662234903"
 from {username='slave'} at 127.0.0.1:36130
I20260709 12:28:52.492484 14468 raft_consensus.cc:493] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:52.492920 14468 raft_consensus.cc:3060] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:52.502588 14468 raft_consensus.cc:515] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:52.504880 14468 leader_election.cc:290] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 4 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:52.505913 14676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:52.506651 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:52.506744 14676 raft_consensus.cc:3060] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:52.507334 14394 raft_consensus.cc:3055] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:28:52.507725 14394 raft_consensus.cc:740] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a153a1160e9240439004094da9aa01ab, State: Running, Role: LEADER
I20260709 12:28:52.508518 14394 consensus_queue.cc:260] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:52.509527 14394 raft_consensus.cc:3060] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:52.516062 14394 raft_consensus.cc:2468] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 4.
I20260709 12:28:52.516571 14676 raft_consensus.cc:2468] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 4.
I20260709 12:28:52.517206 14435 leader_election.cc:304] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, abe2b40322a94e769d80927662234903; no voters: 
I20260709 12:28:52.518289 14621 raft_consensus.cc:2804] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:28:52.518898 14621 raft_consensus.cc:697] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 4 LEADER]: Becoming Leader. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:52.520015 14621 consensus_queue.cc:237] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:52.527465 14217 catalog_manager.cc:5697] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 reported cstate change: term changed from 3 to 4, leader changed from a153a1160e9240439004094da9aa01ab (127.2.4.194) to abe2b40322a94e769d80927662234903 (127.2.4.195). New cstate: current_term: 4 leader_uuid: "abe2b40322a94e769d80927662234903" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:52.530153 14266 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:52.907315 14394 raft_consensus.cc:1275] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 3 FOLLOWER]: Refusing update from remote peer abe2b40322a94e769d80927662234903: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:52.909152 14621 consensus_queue.cc:1048] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:52.921108 14676 raft_consensus.cc:1275] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 3 FOLLOWER]: Refusing update from remote peer abe2b40322a94e769d80927662234903: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:52.923108 14749 consensus_queue.cc:1048] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:52.936826 14676 raft_consensus.cc:1275] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 4 FOLLOWER]: Refusing update from remote peer abe2b40322a94e769d80927662234903: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:28:52.938330 14749 consensus_queue.cc:1048] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:52.951013 14394 raft_consensus.cc:1275] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 4 FOLLOWER]: Refusing update from remote peer abe2b40322a94e769d80927662234903: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:28:52.952747 14621 consensus_queue.cc:1048] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:53.403648  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:53.412271 14468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "320069d83c9f4143869dd90f6d253134"
dest_uuid: "abe2b40322a94e769d80927662234903"
mode: GRACEFUL
new_leader_uuid: "a153a1160e9240439004094da9aa01ab"
 from {username='slave'} at 127.0.0.1:35678
I20260709 12:28:53.412832 14468 raft_consensus.cc:606] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 2 LEADER]: Received request to transfer leadership to TS a153a1160e9240439004094da9aa01ab
I20260709 12:28:53.415750 14468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f"
dest_uuid: "abe2b40322a94e769d80927662234903"
mode: GRACEFUL
new_leader_uuid: "a153a1160e9240439004094da9aa01ab"
 from {username='slave'} at 127.0.0.1:35678
I20260709 12:28:53.416230 14468 raft_consensus.cc:606] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 3 LEADER]: Received request to transfer leadership to TS a153a1160e9240439004094da9aa01ab
I20260709 12:28:53.417331  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:28:53.417896  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:53.418452  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:28:53.531296 14266 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:53.647809 14752 raft_consensus.cc:993] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903: : Instructing follower a153a1160e9240439004094da9aa01ab to start an election
I20260709 12:28:53.648552 14753 raft_consensus.cc:1081] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 2 LEADER]: Signalling peer a153a1160e9240439004094da9aa01ab to start an election
I20260709 12:28:53.650465 14394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "320069d83c9f4143869dd90f6d253134"
dest_uuid: "a153a1160e9240439004094da9aa01ab"
 from {username='slave'} at 127.0.0.1:42562
I20260709 12:28:53.651055 14394 raft_consensus.cc:493] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:53.651408 14394 raft_consensus.cc:3060] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:53.659582 14394 raft_consensus.cc:515] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:53.661203 14394 leader_election.cc:290] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 3 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:53.662323 14468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "320069d83c9f4143869dd90f6d253134" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "abe2b40322a94e769d80927662234903"
I20260709 12:28:53.662220 14676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "320069d83c9f4143869dd90f6d253134" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:53.663156 14468 raft_consensus.cc:3055] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:53.663156 14676 raft_consensus.cc:3060] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:53.663712 14468 raft_consensus.cc:740] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:53.664755 14468 consensus_queue.cc:260] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:53.666155 14468 raft_consensus.cc:3060] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:53.672097 14676 raft_consensus.cc:2468] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 3.
I20260709 12:28:53.673760 14360 leader_election.cc:304] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:53.674525 14468 raft_consensus.cc:2468] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 3.
I20260709 12:28:53.674772 14739 raft_consensus.cc:2804] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:53.676520 14739 raft_consensus.cc:697] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 3 LEADER]: Becoming Leader. State: Replica: a153a1160e9240439004094da9aa01ab, State: Running, Role: LEADER
I20260709 12:28:53.677743 14739 consensus_queue.cc:237] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:53.686331 14217 catalog_manager.cc:5697] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab reported cstate change: term changed from 2 to 3, leader changed from abe2b40322a94e769d80927662234903 (127.2.4.195) to a153a1160e9240439004094da9aa01ab (127.2.4.194). New cstate: current_term: 3 leader_uuid: "a153a1160e9240439004094da9aa01ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:54.012728 14752 raft_consensus.cc:993] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903: : Instructing follower a153a1160e9240439004094da9aa01ab to start an election
I20260709 12:28:54.013250 14753 raft_consensus.cc:1081] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 3 LEADER]: Signalling peer a153a1160e9240439004094da9aa01ab to start an election
I20260709 12:28:54.015568 14394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f"
dest_uuid: "a153a1160e9240439004094da9aa01ab"
 from {username='slave'} at 127.0.0.1:42562
I20260709 12:28:54.016120 14394 raft_consensus.cc:493] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:54.016397 14394 raft_consensus.cc:3060] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:54.021965 14394 raft_consensus.cc:515] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:54.023801 14394 leader_election.cc:290] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 4 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:54.024968 14676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:54.025099 14468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625d6ab82ccb4bfca8333bbf41e4ce0f" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "abe2b40322a94e769d80927662234903"
I20260709 12:28:54.025732 14676 raft_consensus.cc:3060] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:54.025878 14468 raft_consensus.cc:3055] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:28:54.026325 14468 raft_consensus.cc:740] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:54.027150 14468 consensus_queue.cc:260] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:54.028257 14468 raft_consensus.cc:3060] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:54.031889 14676 raft_consensus.cc:2468] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 4.
I20260709 12:28:54.033246 14360 leader_election.cc:304] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:54.034125 14739 raft_consensus.cc:2804] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:28:54.034690 14739 raft_consensus.cc:697] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 4 LEADER]: Becoming Leader. State: Replica: a153a1160e9240439004094da9aa01ab, State: Running, Role: LEADER
I20260709 12:28:54.035293 14468 raft_consensus.cc:2468] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 4.
I20260709 12:28:54.035691 14739 consensus_queue.cc:237] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:54.046418 14217 catalog_manager.cc:5697] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab reported cstate change: term changed from 3 to 4, leader changed from abe2b40322a94e769d80927662234903 (127.2.4.195) to a153a1160e9240439004094da9aa01ab (127.2.4.194). New cstate: current_term: 4 leader_uuid: "a153a1160e9240439004094da9aa01ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:54.148554 14676 raft_consensus.cc:1275] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 3 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:54.150511 14754 consensus_queue.cc:1048] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:54.154222 14468 raft_consensus.cc:1275] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 3 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:54.155994 14754 consensus_queue.cc:1048] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:28:54.426040  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:54.433822 14394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "320069d83c9f4143869dd90f6d253134"
dest_uuid: "a153a1160e9240439004094da9aa01ab"
mode: GRACEFUL
new_leader_uuid: "abe2b40322a94e769d80927662234903"
 from {username='slave'} at 127.0.0.1:41588
I20260709 12:28:54.434459 14394 raft_consensus.cc:606] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 3 LEADER]: Received request to transfer leadership to TS abe2b40322a94e769d80927662234903
I20260709 12:28:54.437696 14394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ad19be826494e9fa81d8b26a929ff4c"
dest_uuid: "a153a1160e9240439004094da9aa01ab"
mode: GRACEFUL
new_leader_uuid: "abe2b40322a94e769d80927662234903"
 from {username='slave'} at 127.0.0.1:41588
I20260709 12:28:54.438230 14394 raft_consensus.cc:606] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Received request to transfer leadership to TS abe2b40322a94e769d80927662234903
I20260709 12:28:54.439540  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:28:54.440232  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:54.441005  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:54.494267 14676 raft_consensus.cc:1275] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 4 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:28:54.496168 14739 consensus_queue.cc:1048] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:28:54.508237 14468 raft_consensus.cc:1275] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 4 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:28:54.510957 14768 consensus_queue.cc:1048] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:28:54.532397 14266 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:54.702229 14739 raft_consensus.cc:993] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab: : Instructing follower abe2b40322a94e769d80927662234903 to start an election
I20260709 12:28:54.702821 14739 raft_consensus.cc:1081] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Signalling peer abe2b40322a94e769d80927662234903 to start an election
I20260709 12:28:54.704499 14468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ad19be826494e9fa81d8b26a929ff4c"
dest_uuid: "abe2b40322a94e769d80927662234903"
 from {username='slave'} at 127.0.0.1:36130
I20260709 12:28:54.705240 14468 raft_consensus.cc:493] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:54.705610 14468 raft_consensus.cc:3060] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:54.712185 14468 raft_consensus.cc:515] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:54.714064 14468 leader_election.cc:290] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 3 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:54.715309 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad19be826494e9fa81d8b26a929ff4c" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:54.715313 14676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad19be826494e9fa81d8b26a929ff4c" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:54.716296 14394 raft_consensus.cc:3055] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:54.716331 14676 raft_consensus.cc:3060] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:54.716825 14394 raft_consensus.cc:740] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a153a1160e9240439004094da9aa01ab, State: Running, Role: LEADER
I20260709 12:28:54.717777 14394 consensus_queue.cc:260] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:54.719117 14394 raft_consensus.cc:3060] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:54.725145 14676 raft_consensus.cc:2468] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 3.
I20260709 12:28:54.726868 14434 leader_election.cc:304] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:54.727090 14394 raft_consensus.cc:2468] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 3.
I20260709 12:28:54.727805 14752 raft_consensus.cc:2804] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:54.728406 14752 raft_consensus.cc:697] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 3 LEADER]: Becoming Leader. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:54.729306 14752 consensus_queue.cc:237] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:54.738252 14218 catalog_manager.cc:5697] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 reported cstate change: term changed from 2 to 3, leader changed from a153a1160e9240439004094da9aa01ab (127.2.4.194) to abe2b40322a94e769d80927662234903 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "abe2b40322a94e769d80927662234903" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:54.764640 14768 raft_consensus.cc:993] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab: : Instructing follower abe2b40322a94e769d80927662234903 to start an election
I20260709 12:28:54.765261 14739 raft_consensus.cc:1081] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 3 LEADER]: Signalling peer abe2b40322a94e769d80927662234903 to start an election
I20260709 12:28:54.767025 14468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "320069d83c9f4143869dd90f6d253134"
dest_uuid: "abe2b40322a94e769d80927662234903"
 from {username='slave'} at 127.0.0.1:36130
I20260709 12:28:54.767584 14468 raft_consensus.cc:493] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:54.767874 14468 raft_consensus.cc:3060] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:54.774396 14468 raft_consensus.cc:515] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:54.776391 14468 leader_election.cc:290] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 4 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), a153a1160e9240439004094da9aa01ab (127.2.4.194:36325)
I20260709 12:28:54.777457 14676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "320069d83c9f4143869dd90f6d253134" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:54.777721 14394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "320069d83c9f4143869dd90f6d253134" candidate_uuid: "abe2b40322a94e769d80927662234903" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a153a1160e9240439004094da9aa01ab"
I20260709 12:28:54.778211 14676 raft_consensus.cc:3060] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:54.778414 14394 raft_consensus.cc:3055] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:28:54.778843 14394 raft_consensus.cc:740] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a153a1160e9240439004094da9aa01ab, State: Running, Role: LEADER
I20260709 12:28:54.779575 14394 consensus_queue.cc:260] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:54.780689 14394 raft_consensus.cc:3060] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:54.784377 14676 raft_consensus.cc:2468] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 4.
I20260709 12:28:54.785727 14434 leader_election.cc:304] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe2b40322a94e769d80927662234903, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:54.786441 14752 raft_consensus.cc:2804] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:28:54.786715 14394 raft_consensus.cc:2468] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe2b40322a94e769d80927662234903 in term 4.
I20260709 12:28:54.786896 14752 raft_consensus.cc:697] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 4 LEADER]: Becoming Leader. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:54.787961 14752 consensus_queue.cc:237] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:54.795009 14218 catalog_manager.cc:5697] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 reported cstate change: term changed from 3 to 4, leader changed from a153a1160e9240439004094da9aa01ab (127.2.4.194) to abe2b40322a94e769d80927662234903 (127.2.4.195). New cstate: current_term: 4 leader_uuid: "abe2b40322a94e769d80927662234903" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:55.159411 14676 raft_consensus.cc:1275] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 3 FOLLOWER]: Refusing update from remote peer abe2b40322a94e769d80927662234903: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:55.161062 14752 consensus_queue.cc:1048] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:28:55.178399 14394 raft_consensus.cc:1275] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 3 FOLLOWER]: Refusing update from remote peer abe2b40322a94e769d80927662234903: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:55.180315 14777 consensus_queue.cc:1048] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:28:55.398734 14394 raft_consensus.cc:1275] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 4 FOLLOWER]: Refusing update from remote peer abe2b40322a94e769d80927662234903: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:28:55.400460 14777 consensus_queue.cc:1048] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:28:55.411051 14676 raft_consensus.cc:1275] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 4 FOLLOWER]: Refusing update from remote peer abe2b40322a94e769d80927662234903: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:28:55.413187 14752 consensus_queue.cc:1048] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:28:55.448158  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:55.456714 14468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "36f62e820fa642a1bcd4095a04b01648"
dest_uuid: "abe2b40322a94e769d80927662234903"
mode: GRACEFUL
new_leader_uuid: "a153a1160e9240439004094da9aa01ab"
 from {username='slave'} at 127.0.0.1:35678
I20260709 12:28:55.457417 14468 raft_consensus.cc:606] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Received request to transfer leadership to TS a153a1160e9240439004094da9aa01ab
I20260709 12:28:55.460983 14468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92"
dest_uuid: "abe2b40322a94e769d80927662234903"
mode: GRACEFUL
new_leader_uuid: "a153a1160e9240439004094da9aa01ab"
 from {username='slave'} at 127.0.0.1:35678
I20260709 12:28:55.461613 14468 raft_consensus.cc:606] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 4 LEADER]: Received request to transfer leadership to TS a153a1160e9240439004094da9aa01ab
I20260709 12:28:55.463748  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:28:55.464531  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:55.465339  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:28:55.533440 14266 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:55.562266 14777 raft_consensus.cc:993] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903: : Instructing follower a153a1160e9240439004094da9aa01ab to start an election
I20260709 12:28:55.562929 14752 raft_consensus.cc:1081] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 4 LEADER]: Signalling peer a153a1160e9240439004094da9aa01ab to start an election
I20260709 12:28:55.564666 14394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92"
dest_uuid: "a153a1160e9240439004094da9aa01ab"
 from {username='slave'} at 127.0.0.1:42562
I20260709 12:28:55.565308 14394 raft_consensus.cc:493] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:55.565613 14394 raft_consensus.cc:3060] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:28:55.571815 14394 raft_consensus.cc:515] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:55.573733 14394 leader_election.cc:290] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 5 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:55.575098 14675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:55.575093 14468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aeb6f75770b4fd5bf7692ba6e2ffa92" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "abe2b40322a94e769d80927662234903"
I20260709 12:28:55.575909 14675 raft_consensus.cc:3060] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:28:55.576217 14468 raft_consensus.cc:3055] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:28:55.576648 14468 raft_consensus.cc:740] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:55.577487 14468 consensus_queue.cc:260] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:55.578631 14468 raft_consensus.cc:3060] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:28:55.583086 14675 raft_consensus.cc:2468] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 5.
I20260709 12:28:55.584673 14468 raft_consensus.cc:2468] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 5.
I20260709 12:28:55.584697 14360 leader_election.cc:304] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:55.585976 14739 raft_consensus.cc:2804] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:28:55.586985 14739 raft_consensus.cc:697] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 5 LEADER]: Becoming Leader. State: Replica: a153a1160e9240439004094da9aa01ab, State: Running, Role: LEADER
I20260709 12:28:55.587932 14739 consensus_queue.cc:237] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:55.595927 14218 catalog_manager.cc:5697] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab reported cstate change: term changed from 4 to 5, leader changed from abe2b40322a94e769d80927662234903 (127.2.4.195) to a153a1160e9240439004094da9aa01ab (127.2.4.194). New cstate: current_term: 5 leader_uuid: "a153a1160e9240439004094da9aa01ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:55.598320 14777 raft_consensus.cc:993] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903: : Instructing follower a153a1160e9240439004094da9aa01ab to start an election
I20260709 12:28:55.599001 14752 raft_consensus.cc:1081] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Signalling peer a153a1160e9240439004094da9aa01ab to start an election
I20260709 12:28:55.600929 14394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "36f62e820fa642a1bcd4095a04b01648"
dest_uuid: "a153a1160e9240439004094da9aa01ab"
 from {username='slave'} at 127.0.0.1:42562
I20260709 12:28:55.601477 14394 raft_consensus.cc:493] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:55.601908 14394 raft_consensus.cc:3060] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:55.610066 14394 raft_consensus.cc:515] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:55.611748 14394 leader_election.cc:290] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 election: Requested vote from peers e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175), abe2b40322a94e769d80927662234903 (127.2.4.195:37025)
I20260709 12:28:55.612912 14675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f62e820fa642a1bcd4095a04b01648" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8"
I20260709 12:28:55.613010 14468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f62e820fa642a1bcd4095a04b01648" candidate_uuid: "a153a1160e9240439004094da9aa01ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "abe2b40322a94e769d80927662234903"
I20260709 12:28:55.613830 14675 raft_consensus.cc:3060] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:55.614045 14468 raft_consensus.cc:3055] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:55.614547 14468 raft_consensus.cc:740] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: abe2b40322a94e769d80927662234903, State: Running, Role: LEADER
I20260709 12:28:55.615599 14468 consensus_queue.cc:260] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:55.616886 14468 raft_consensus.cc:3060] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:55.621142 14675 raft_consensus.cc:2468] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 2.
I20260709 12:28:55.622502 14468 raft_consensus.cc:2468] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a153a1160e9240439004094da9aa01ab in term 2.
I20260709 12:28:55.622776 14360 leader_election.cc:304] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a153a1160e9240439004094da9aa01ab, e621e3baa9aa42ab963c4f3af3bca8b8; no voters: 
I20260709 12:28:55.624027 14739 raft_consensus.cc:2804] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:55.624559 14739 raft_consensus.cc:697] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Becoming Leader. State: Replica: a153a1160e9240439004094da9aa01ab, State: Running, Role: LEADER
I20260709 12:28:55.625447 14739 consensus_queue.cc:237] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } }
I20260709 12:28:55.635164 14218 catalog_manager.cc:5697] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab reported cstate change: term changed from 1 to 2, leader changed from abe2b40322a94e769d80927662234903 (127.2.4.195) to a153a1160e9240439004094da9aa01ab (127.2.4.194). New cstate: current_term: 2 leader_uuid: "a153a1160e9240439004094da9aa01ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a153a1160e9240439004094da9aa01ab" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 36325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.059087 14468 raft_consensus.cc:1275] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:56.060640 14783 consensus_queue.cc:1048] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:56.077587 14675 raft_consensus.cc:1275] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 5 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:28:56.081159 14675 raft_consensus.cc:1275] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:56.082031 14739 consensus_queue.cc:1048] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:28:56.084102 14791 consensus_queue.cc:1048] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621e3baa9aa42ab963c4f3af3bca8b8" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:56.099457 14468 raft_consensus.cc:1275] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 5 FOLLOWER]: Refusing update from remote peer a153a1160e9240439004094da9aa01ab: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:28:56.102581 14791 consensus_queue.cc:1048] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe2b40322a94e769d80927662234903" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 37025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:28:56.473112  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:56.509752  2067 tablet_server.cc:179] TabletServer@127.2.4.193:43175 shutting down...
W20260709 12:28:56.534746 14266 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:56.544821  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:56.545918  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:56.546849  2067 raft_consensus.cc:2243] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.547413  2067 raft_consensus.cc:2272] T d6dcc07ba9784ae98d3974c6abb1b53b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.550065  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:56.550701  2067 raft_consensus.cc:2243] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.551203  2067 raft_consensus.cc:2272] T 9eec14432a4543eea89701c25066887b P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.553902  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:56.554652  2067 raft_consensus.cc:2243] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.555243  2067 raft_consensus.cc:2272] T 320069d83c9f4143869dd90f6d253134 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.558741  2067 tablet_replica.cc:333] stopping tablet replica
W20260709 12:28:56.558786 14360 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 87 similar messages]
I20260709 12:28:56.560081  2067 raft_consensus.cc:2243] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.560756  2067 raft_consensus.cc:2272] T 8ad19be826494e9fa81d8b26a929ff4c P e621e3baa9aa42ab963c4f3af3bca8b8 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:56.562508 14360 consensus_peers.cc:597] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:56.563928  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:56.564574  2067 raft_consensus.cc:2243] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.565025  2067 raft_consensus.cc:2272] T 354b6360e1104e169aaee9531ae2c3a6 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.567417  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:56.567983  2067 raft_consensus.cc:2243] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.568404  2067 raft_consensus.cc:2272] T 527f315df63447e3bc065eb01506adec P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.570711  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:56.571267  2067 raft_consensus.cc:2243] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.571708  2067 raft_consensus.cc:2272] T 625d6ab82ccb4bfca8333bbf41e4ce0f P e621e3baa9aa42ab963c4f3af3bca8b8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.573910  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:56.574754  2067 raft_consensus.cc:2243] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.575161  2067 raft_consensus.cc:2272] T 5e6555f9875c4743a1680538e4c03821 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.577613  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:56.578253  2067 raft_consensus.cc:2243] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.578732  2067 raft_consensus.cc:2272] T 36f62e820fa642a1bcd4095a04b01648 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.582260  2067 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:56.582882  2067 raft_consensus.cc:2243] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.583449  2067 raft_consensus.cc:2272] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P e621e3baa9aa42ab963c4f3af3bca8b8 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:56.587347 14434 consensus_peers.cc:597] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:56.619344  2067 tablet_server.cc:196] TabletServer@127.2.4.193:43175 shutdown complete.
W20260709 12:28:56.624555 14360 consensus_peers.cc:597] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:56.652025  2067 tablet_server.cc:179] TabletServer@127.2.4.194:0 shutting down...
W20260709 12:28:56.662060 14360 consensus_peers.cc:597] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:56.674825 14434 consensus_peers.cc:597] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:56.684984 14434 consensus_peers.cc:597] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:56.698773 14360 consensus_peers.cc:597] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab -> Peer e621e3baa9aa42ab963c4f3af3bca8b8 (127.2.4.193:43175): Couldn't send request to peer e621e3baa9aa42ab963c4f3af3bca8b8. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:43175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:56.702140  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:56.703259  2067 tablet_replica.cc:333] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab: stopping tablet replica
I20260709 12:28:56.704113  2067 raft_consensus.cc:2243] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:56.705183  2067 raft_consensus.cc:2272] T 36f62e820fa642a1bcd4095a04b01648 P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.709050  2067 tablet_replica.cc:333] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab: stopping tablet replica
I20260709 12:28:56.710114  2067 raft_consensus.cc:2243] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:28:56.711316  2067 raft_consensus.cc:2272] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P a153a1160e9240439004094da9aa01ab [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.715035  2067 tablet_replica.cc:333] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab: stopping tablet replica
I20260709 12:28:56.715798  2067 raft_consensus.cc:2243] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.716252  2067 raft_consensus.cc:2272] T 320069d83c9f4143869dd90f6d253134 P a153a1160e9240439004094da9aa01ab [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.718737  2067 tablet_replica.cc:333] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab: stopping tablet replica
I20260709 12:28:56.719290  2067 raft_consensus.cc:2243] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:56.720208  2067 raft_consensus.cc:2272] T d6dcc07ba9784ae98d3974c6abb1b53b P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.722755  2067 tablet_replica.cc:333] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab: stopping tablet replica
I20260709 12:28:56.723496  2067 raft_consensus.cc:2243] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:56.724344  2067 raft_consensus.cc:2272] T 527f315df63447e3bc065eb01506adec P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.726867  2067 tablet_replica.cc:333] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab: stopping tablet replica
I20260709 12:28:56.727517  2067 raft_consensus.cc:2243] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:28:56.728346  2067 raft_consensus.cc:2272] T 625d6ab82ccb4bfca8333bbf41e4ce0f P a153a1160e9240439004094da9aa01ab [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.730822  2067 tablet_replica.cc:333] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab: stopping tablet replica
I20260709 12:28:56.731415  2067 raft_consensus.cc:2243] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:56.732342  2067 raft_consensus.cc:2272] T 5e6555f9875c4743a1680538e4c03821 P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.735097  2067 tablet_replica.cc:333] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab: stopping tablet replica
I20260709 12:28:56.735666  2067 raft_consensus.cc:2243] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.736128  2067 raft_consensus.cc:2272] T 9eec14432a4543eea89701c25066887b P a153a1160e9240439004094da9aa01ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.738672  2067 tablet_replica.cc:333] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab: stopping tablet replica
I20260709 12:28:56.739317  2067 raft_consensus.cc:2243] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.739928  2067 raft_consensus.cc:2272] T 354b6360e1104e169aaee9531ae2c3a6 P a153a1160e9240439004094da9aa01ab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.742625  2067 tablet_replica.cc:333] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab: stopping tablet replica
I20260709 12:28:56.743235  2067 raft_consensus.cc:2243] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.743770  2067 raft_consensus.cc:2272] T 8ad19be826494e9fa81d8b26a929ff4c P a153a1160e9240439004094da9aa01ab [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.781282  2067 tablet_server.cc:196] TabletServer@127.2.4.194:0 shutdown complete.
I20260709 12:28:56.807421  2067 tablet_server.cc:179] TabletServer@127.2.4.195:0 shutting down...
W20260709 12:28:56.822369 14435 consensus_peers.cc:597] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 -> Peer a153a1160e9240439004094da9aa01ab (127.2.4.194:36325): Couldn't send request to peer a153a1160e9240439004094da9aa01ab. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:36325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:56.843801 14435 consensus_peers.cc:597] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 -> Peer a153a1160e9240439004094da9aa01ab (127.2.4.194:36325): Couldn't send request to peer a153a1160e9240439004094da9aa01ab. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:36325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:56.852913  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:56.854075  2067 tablet_replica.cc:333] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903: stopping tablet replica
I20260709 12:28:56.855008  2067 raft_consensus.cc:2243] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:56.856124  2067 raft_consensus.cc:2272] T 9eec14432a4543eea89701c25066887b P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.859026  2067 tablet_replica.cc:333] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903: stopping tablet replica
I20260709 12:28:56.859850  2067 raft_consensus.cc:2243] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.860423  2067 raft_consensus.cc:2272] T 5e6555f9875c4743a1680538e4c03821 P abe2b40322a94e769d80927662234903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.862996  2067 tablet_replica.cc:333] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903: stopping tablet replica
I20260709 12:28:56.863595  2067 raft_consensus.cc:2243] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:56.864445  2067 raft_consensus.cc:2272] T 8ad19be826494e9fa81d8b26a929ff4c P abe2b40322a94e769d80927662234903 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.866848  2067 tablet_replica.cc:333] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903: stopping tablet replica
I20260709 12:28:56.867486  2067 raft_consensus.cc:2243] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:56.868343  2067 raft_consensus.cc:2272] T 354b6360e1104e169aaee9531ae2c3a6 P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.870792  2067 tablet_replica.cc:333] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903: stopping tablet replica
I20260709 12:28:56.871546  2067 raft_consensus.cc:2243] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.871968  2067 raft_consensus.cc:2272] T 7aeb6f75770b4fd5bf7692ba6e2ffa92 P abe2b40322a94e769d80927662234903 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.875288  2067 tablet_replica.cc:333] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903: stopping tablet replica
I20260709 12:28:56.875860  2067 raft_consensus.cc:2243] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.876284  2067 raft_consensus.cc:2272] T 36f62e820fa642a1bcd4095a04b01648 P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.879259  2067 tablet_replica.cc:333] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903: stopping tablet replica
I20260709 12:28:56.879814  2067 raft_consensus.cc:2243] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.880273  2067 raft_consensus.cc:2272] T d6dcc07ba9784ae98d3974c6abb1b53b P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.882644  2067 tablet_replica.cc:333] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903: stopping tablet replica
I20260709 12:28:56.883165  2067 raft_consensus.cc:2243] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.883569  2067 raft_consensus.cc:2272] T 625d6ab82ccb4bfca8333bbf41e4ce0f P abe2b40322a94e769d80927662234903 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.886096  2067 tablet_replica.cc:333] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903: stopping tablet replica
I20260709 12:28:56.886659  2067 raft_consensus.cc:2243] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:56.887145  2067 raft_consensus.cc:2272] T 527f315df63447e3bc065eb01506adec P abe2b40322a94e769d80927662234903 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.889743  2067 tablet_replica.cc:333] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903: stopping tablet replica
I20260709 12:28:56.890249  2067 raft_consensus.cc:2243] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:28:56.891124  2067 raft_consensus.cc:2272] T 320069d83c9f4143869dd90f6d253134 P abe2b40322a94e769d80927662234903 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.928041  2067 tablet_server.cc:196] TabletServer@127.2.4.195:0 shutdown complete.
I20260709 12:28:56.961160  2067 master.cc:562] Master@127.2.4.254:35633 shutting down...
I20260709 12:28:56.994508  2067 raft_consensus.cc:2243] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:56.995266  2067 raft_consensus.cc:2272] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:56.995594  2067 tablet_replica.cc:333] T 00000000000000000000000000000000 P bee45cca01884435ac7cf61afe80bed3: stopping tablet replica
I20260709 12:28:57.019053  2067 master.cc:584] Master@127.2.4.254:35633 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13789 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:28:57.069151  2067 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.4.254:46203
I20260709 12:28:57.070627  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:57.076978 14809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:57.078564 14810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:57.080214 14812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:57.080828  2067 server_base.cc:1061] running on GCE node
I20260709 12:28:57.082072  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:57.082276  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:57.082394  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600137082383 us; error 0 us; skew 500 ppm
I20260709 12:28:57.082978  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:57.086073  2067 webserver.cc:533] Webserver started at http://127.2.4.254:42073/ using document root <none> and password file <none>
I20260709 12:28:57.086639  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:57.086839  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:57.087100  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:57.088465  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/master-0-root/instance:
uuid: "0c44f86e99a54a12b701a67b16bf881b"
format_stamp: "Formatted at 2026-07-09 12:28:57 on dist-test-slave-4qvp"
I20260709 12:28:57.093889  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.006s
I20260709 12:28:57.098325 14817 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:57.099334  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:28:57.099709  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/master-0-root
uuid: "0c44f86e99a54a12b701a67b16bf881b"
format_stamp: "Formatted at 2026-07-09 12:28:57 on dist-test-slave-4qvp"
I20260709 12:28:57.100119  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:57.117143  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:57.118727  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:57.170068  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.254:46203
I20260709 12:28:57.170202 14868 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.254:46203 every 8 connection(s)
I20260709 12:28:57.174719 14869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:57.188743 14869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b: Bootstrap starting.
I20260709 12:28:57.194476 14869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:57.200349 14869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b: No bootstrap required, opened a new log
I20260709 12:28:57.203109 14869 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c44f86e99a54a12b701a67b16bf881b" member_type: VOTER }
I20260709 12:28:57.203526 14869 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:57.203714 14869 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c44f86e99a54a12b701a67b16bf881b, State: Initialized, Role: FOLLOWER
I20260709 12:28:57.204332 14869 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c44f86e99a54a12b701a67b16bf881b" member_type: VOTER }
I20260709 12:28:57.204756 14869 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:28:57.204953 14869 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:28:57.205173 14869 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.210567 14869 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c44f86e99a54a12b701a67b16bf881b" member_type: VOTER }
I20260709 12:28:57.211110 14869 leader_election.cc:304] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c44f86e99a54a12b701a67b16bf881b; no voters: 
I20260709 12:28:57.212360 14869 leader_election.cc:290] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:28:57.212926 14872 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:57.214888 14872 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [term 1 LEADER]: Becoming Leader. State: Replica: 0c44f86e99a54a12b701a67b16bf881b, State: Running, Role: LEADER
I20260709 12:28:57.215765 14872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c44f86e99a54a12b701a67b16bf881b" member_type: VOTER }
I20260709 12:28:57.215868 14869 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:28:57.222157 14874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c44f86e99a54a12b701a67b16bf881b. Latest consensus state: current_term: 1 leader_uuid: "0c44f86e99a54a12b701a67b16bf881b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c44f86e99a54a12b701a67b16bf881b" member_type: VOTER } }
I20260709 12:28:57.223207 14874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:57.222877 14873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c44f86e99a54a12b701a67b16bf881b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c44f86e99a54a12b701a67b16bf881b" member_type: VOTER } }
I20260709 12:28:57.224102 14873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:57.225534 14881 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:28:57.230959 14881 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:28:57.233305  2067 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:28:57.241591 14881 catalog_manager.cc:1383] Generated new cluster ID: 4877f946e05a42518a0ea2c4f1a1d710
I20260709 12:28:57.242061 14881 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:28:57.263608 14881 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:28:57.265223 14881 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:28:57.286253 14881 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b: Generated new TSK 0
I20260709 12:28:57.287057 14881 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:28:57.300628  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:57.308210 14890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:57.308967 14891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:57.311033 14893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:57.311759  2067 server_base.cc:1061] running on GCE node
I20260709 12:28:57.312983  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:57.313184  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:57.313400  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600137313382 us; error 0 us; skew 500 ppm
I20260709 12:28:57.314070  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:57.321182  2067 webserver.cc:533] Webserver started at http://127.2.4.193:42883/ using document root <none> and password file <none>
I20260709 12:28:57.321831  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:57.322068  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:57.322400  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:57.323629  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/ts-0-root/instance:
uuid: "946d0404898c4d6cbb83c33bdf4b1920"
format_stamp: "Formatted at 2026-07-09 12:28:57 on dist-test-slave-4qvp"
I20260709 12:28:57.329555  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 12:28:57.334301 14898 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:57.335435  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:28:57.336051  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/ts-0-root
uuid: "946d0404898c4d6cbb83c33bdf4b1920"
format_stamp: "Formatted at 2026-07-09 12:28:57 on dist-test-slave-4qvp"
I20260709 12:28:57.336485  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:57.359349  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:57.360756  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:57.362465  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:57.365005  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:57.365273  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:57.365525  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:57.365748  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:28:57.418772  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.193:46073
I20260709 12:28:57.418929 14960 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.193:46073 every 8 connection(s)
I20260709 12:28:57.424639  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:57.434068 14965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:57.436586 14966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:57.438733 14968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:57.443456 14961 heartbeater.cc:344] Connected to a master server at 127.2.4.254:46203
I20260709 12:28:57.444092 14961 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:57.444376  2067 server_base.cc:1061] running on GCE node
I20260709 12:28:57.445142 14961 heartbeater.cc:507] Master 127.2.4.254:46203 requested a full tablet report, sending...
I20260709 12:28:57.445740  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:57.446035  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:57.446239  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600137446214 us; error 0 us; skew 500 ppm
I20260709 12:28:57.446856  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:57.447873 14834 ts_manager.cc:194] Registered new tserver with Master: 946d0404898c4d6cbb83c33bdf4b1920 (127.2.4.193:46073)
I20260709 12:28:57.449973  2067 webserver.cc:533] Webserver started at http://127.2.4.194:37027/ using document root <none> and password file <none>
I20260709 12:28:57.450635  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:57.450538 14834 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59240
I20260709 12:28:57.450979  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:57.451411  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:57.452714  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/ts-1-root/instance:
uuid: "5cd7bd327ce34244aabf4f9be1142e6e"
format_stamp: "Formatted at 2026-07-09 12:28:57 on dist-test-slave-4qvp"
I20260709 12:28:57.457860  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 12:28:57.462366 14973 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:57.463456  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:28:57.463773  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/ts-1-root
uuid: "5cd7bd327ce34244aabf4f9be1142e6e"
format_stamp: "Formatted at 2026-07-09 12:28:57 on dist-test-slave-4qvp"
I20260709 12:28:57.464066  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:57.476867  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:57.478528  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:57.480369  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:57.483217  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:57.483431  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:57.483748  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:57.483901  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:57.536691  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.194:39797
I20260709 12:28:57.536815 15035 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.194:39797 every 8 connection(s)
I20260709 12:28:57.543282  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:57.552424 15039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:57.552533 15040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:57.558033 15042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:57.558674 15036 heartbeater.cc:344] Connected to a master server at 127.2.4.254:46203
I20260709 12:28:57.559278 15036 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:57.559664  2067 server_base.cc:1061] running on GCE node
I20260709 12:28:57.560421 15036 heartbeater.cc:507] Master 127.2.4.254:46203 requested a full tablet report, sending...
I20260709 12:28:57.560957  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:57.561275  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:57.561517  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600137561491 us; error 0 us; skew 500 ppm
I20260709 12:28:57.562388  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:57.563431 14834 ts_manager.cc:194] Registered new tserver with Master: 5cd7bd327ce34244aabf4f9be1142e6e (127.2.4.194:39797)
I20260709 12:28:57.565629  2067 webserver.cc:533] Webserver started at http://127.2.4.195:42219/ using document root <none> and password file <none>
I20260709 12:28:57.565678 14834 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59252
I20260709 12:28:57.566454  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:57.566771  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:57.567101  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:57.568609  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/ts-2-root/instance:
uuid: "d784628a1b6f4db387b149570005cf2e"
format_stamp: "Formatted at 2026-07-09 12:28:57 on dist-test-slave-4qvp"
I20260709 12:28:57.574364  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.004s
I20260709 12:28:57.578598 15047 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:57.579602  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:28:57.580016  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/ts-2-root
uuid: "d784628a1b6f4db387b149570005cf2e"
format_stamp: "Formatted at 2026-07-09 12:28:57 on dist-test-slave-4qvp"
I20260709 12:28:57.580303  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599727094521-2067-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:57.605183  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:57.606669  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:57.608255  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:57.610988  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:57.611186  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:57.611516  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:57.611723  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:57.665688  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.195:44693
I20260709 12:28:57.666016 15109 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.195:44693 every 8 connection(s)
I20260709 12:28:57.691870 15110 heartbeater.cc:344] Connected to a master server at 127.2.4.254:46203
I20260709 12:28:57.692703 15110 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:57.694018 15110 heartbeater.cc:507] Master 127.2.4.254:46203 requested a full tablet report, sending...
I20260709 12:28:57.697194 14834 ts_manager.cc:194] Registered new tserver with Master: d784628a1b6f4db387b149570005cf2e (127.2.4.195:44693)
I20260709 12:28:57.698107  2067 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.028172401s
I20260709 12:28:57.699345 14834 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59256
I20260709 12:28:57.738107 14834 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59272:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:57.740792 14834 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:57.816435 14926 tablet_service.cc:1511] Processing CreateTablet for tablet 859e9eaf687f4ccaaf30f3532b5c0baa (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=368695a2fc9c454daa4009cb5d10495e]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:57.818233 14926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 859e9eaf687f4ccaaf30f3532b5c0baa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:57.822956 14925 tablet_service.cc:1511] Processing CreateTablet for tablet 87fc96feabe64019ad8f09667ce28500 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=368695a2fc9c454daa4009cb5d10495e]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:57.824658 14925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87fc96feabe64019ad8f09667ce28500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:57.826647 15001 tablet_service.cc:1511] Processing CreateTablet for tablet 859e9eaf687f4ccaaf30f3532b5c0baa (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=368695a2fc9c454daa4009cb5d10495e]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:57.828320 15001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 859e9eaf687f4ccaaf30f3532b5c0baa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:57.830875 15000 tablet_service.cc:1511] Processing CreateTablet for tablet 87fc96feabe64019ad8f09667ce28500 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=368695a2fc9c454daa4009cb5d10495e]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:57.832434 15000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87fc96feabe64019ad8f09667ce28500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:57.839875 15075 tablet_service.cc:1511] Processing CreateTablet for tablet 859e9eaf687f4ccaaf30f3532b5c0baa (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=368695a2fc9c454daa4009cb5d10495e]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:57.841418 15075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 859e9eaf687f4ccaaf30f3532b5c0baa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:57.856503 15074 tablet_service.cc:1511] Processing CreateTablet for tablet 87fc96feabe64019ad8f09667ce28500 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=368695a2fc9c454daa4009cb5d10495e]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:57.864252 15074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87fc96feabe64019ad8f09667ce28500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:57.868405 15125 tablet_bootstrap.cc:492] T 87fc96feabe64019ad8f09667ce28500 P 946d0404898c4d6cbb83c33bdf4b1920: Bootstrap starting.
I20260709 12:28:57.869220 15126 tablet_bootstrap.cc:492] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e: Bootstrap starting.
I20260709 12:28:57.876400 15127 tablet_bootstrap.cc:492] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e: Bootstrap starting.
I20260709 12:28:57.878093 15126 tablet_bootstrap.cc:654] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:57.883400 15125 tablet_bootstrap.cc:654] T 87fc96feabe64019ad8f09667ce28500 P 946d0404898c4d6cbb83c33bdf4b1920: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:57.884099 15127 tablet_bootstrap.cc:654] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:57.891348 15126 tablet_bootstrap.cc:492] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e: No bootstrap required, opened a new log
I20260709 12:28:57.891893 15126 ts_tablet_manager.cc:1403] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e: Time spent bootstrapping tablet: real 0.023s	user 0.007s	sys 0.012s
I20260709 12:28:57.892271 15127 tablet_bootstrap.cc:492] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e: No bootstrap required, opened a new log
I20260709 12:28:57.892304 15125 tablet_bootstrap.cc:492] T 87fc96feabe64019ad8f09667ce28500 P 946d0404898c4d6cbb83c33bdf4b1920: No bootstrap required, opened a new log
I20260709 12:28:57.893077 15127 ts_tablet_manager.cc:1403] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.005s
I20260709 12:28:57.893132 15125 ts_tablet_manager.cc:1403] T 87fc96feabe64019ad8f09667ce28500 P 946d0404898c4d6cbb83c33bdf4b1920: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.009s
I20260709 12:28:57.894618 15126 raft_consensus.cc:359] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:57.895358 15126 raft_consensus.cc:385] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:57.895625 15126 raft_consensus.cc:740] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd7bd327ce34244aabf4f9be1142e6e, State: Initialized, Role: FOLLOWER
I20260709 12:28:57.896379 15126 consensus_queue.cc:260] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:57.896960 15125 raft_consensus.cc:359] T 87fc96feabe64019ad8f09667ce28500 P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:57.897900 15125 raft_consensus.cc:385] T 87fc96feabe64019ad8f09667ce28500 P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:57.898202 15125 raft_consensus.cc:740] T 87fc96feabe64019ad8f09667ce28500 P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946d0404898c4d6cbb83c33bdf4b1920, State: Initialized, Role: FOLLOWER
I20260709 12:28:57.898124 15127 raft_consensus.cc:359] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:57.899029 15127 raft_consensus.cc:385] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:57.899329 15127 raft_consensus.cc:740] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d784628a1b6f4db387b149570005cf2e, State: Initialized, Role: FOLLOWER
I20260709 12:28:57.901010 15127 consensus_queue.cc:260] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:57.899273 15125 consensus_queue.cc:260] T 87fc96feabe64019ad8f09667ce28500 P 946d0404898c4d6cbb83c33bdf4b1920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:57.901593 15036 heartbeater.cc:499] Master 127.2.4.254:46203 was elected leader, sending a full tablet report...
I20260709 12:28:57.902232 15126 ts_tablet_manager.cc:1434] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.005s
I20260709 12:28:57.903328 15126 tablet_bootstrap.cc:492] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e: Bootstrap starting.
I20260709 12:28:57.910400 15126 tablet_bootstrap.cc:654] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:57.923758 15127 ts_tablet_manager.cc:1434] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e: Time spent starting tablet: real 0.030s	user 0.018s	sys 0.011s
I20260709 12:28:57.923759 15125 ts_tablet_manager.cc:1434] T 87fc96feabe64019ad8f09667ce28500 P 946d0404898c4d6cbb83c33bdf4b1920: Time spent starting tablet: real 0.030s	user 0.018s	sys 0.008s
I20260709 12:28:57.924036 15110 heartbeater.cc:499] Master 127.2.4.254:46203 was elected leader, sending a full tablet report...
I20260709 12:28:57.925333 14961 heartbeater.cc:499] Master 127.2.4.254:46203 was elected leader, sending a full tablet report...
I20260709 12:28:57.925531 15127 tablet_bootstrap.cc:492] T 87fc96feabe64019ad8f09667ce28500 P d784628a1b6f4db387b149570005cf2e: Bootstrap starting.
I20260709 12:28:57.925778 15126 tablet_bootstrap.cc:492] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e: No bootstrap required, opened a new log
I20260709 12:28:57.926369 15126 ts_tablet_manager.cc:1403] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.010s
I20260709 12:28:57.928354 15125 tablet_bootstrap.cc:492] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920: Bootstrap starting.
I20260709 12:28:57.929759 15126 raft_consensus.cc:359] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:57.930680 15126 raft_consensus.cc:385] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:57.930992 15126 raft_consensus.cc:740] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd7bd327ce34244aabf4f9be1142e6e, State: Initialized, Role: FOLLOWER
I20260709 12:28:57.931819 15126 consensus_queue.cc:260] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:57.933576 15127 tablet_bootstrap.cc:654] T 87fc96feabe64019ad8f09667ce28500 P d784628a1b6f4db387b149570005cf2e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:57.934144 15126 ts_tablet_manager.cc:1434] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.005s
I20260709 12:28:57.937458 15125 tablet_bootstrap.cc:654] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:57.947634 15127 tablet_bootstrap.cc:492] T 87fc96feabe64019ad8f09667ce28500 P d784628a1b6f4db387b149570005cf2e: No bootstrap required, opened a new log
I20260709 12:28:57.948311 15127 ts_tablet_manager.cc:1403] T 87fc96feabe64019ad8f09667ce28500 P d784628a1b6f4db387b149570005cf2e: Time spent bootstrapping tablet: real 0.023s	user 0.007s	sys 0.011s
I20260709 12:28:57.951818 15127 raft_consensus.cc:359] T 87fc96feabe64019ad8f09667ce28500 P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:57.952661 15127 raft_consensus.cc:385] T 87fc96feabe64019ad8f09667ce28500 P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:57.953043 15127 raft_consensus.cc:740] T 87fc96feabe64019ad8f09667ce28500 P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d784628a1b6f4db387b149570005cf2e, State: Initialized, Role: FOLLOWER
I20260709 12:28:57.953857 15125 tablet_bootstrap.cc:492] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920: No bootstrap required, opened a new log
I20260709 12:28:57.953948 15127 consensus_queue.cc:260] T 87fc96feabe64019ad8f09667ce28500 P d784628a1b6f4db387b149570005cf2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:57.954756 15125 ts_tablet_manager.cc:1403] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.011s
I20260709 12:28:57.956925 15127 ts_tablet_manager.cc:1434] T 87fc96feabe64019ad8f09667ce28500 P d784628a1b6f4db387b149570005cf2e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.004s
I20260709 12:28:57.958459 15125 raft_consensus.cc:359] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:57.959088 15125 raft_consensus.cc:385] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:57.959383 15125 raft_consensus.cc:740] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946d0404898c4d6cbb83c33bdf4b1920, State: Initialized, Role: FOLLOWER
I20260709 12:28:57.960175 15125 consensus_queue.cc:260] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:57.962381 15125 ts_tablet_manager.cc:1434] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:58.070592 15132 raft_consensus.cc:493] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.071197 15132 raft_consensus.cc:515] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.073376 15132 leader_election.cc:290] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946d0404898c4d6cbb83c33bdf4b1920 (127.2.4.193:46073), 5cd7bd327ce34244aabf4f9be1142e6e (127.2.4.194:39797)
I20260709 12:28:58.085515 14936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859e9eaf687f4ccaaf30f3532b5c0baa" candidate_uuid: "d784628a1b6f4db387b149570005cf2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946d0404898c4d6cbb83c33bdf4b1920" is_pre_election: true
I20260709 12:28:58.086549 14936 raft_consensus.cc:2468] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d784628a1b6f4db387b149570005cf2e in term 0.
I20260709 12:28:58.087358 15011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859e9eaf687f4ccaaf30f3532b5c0baa" candidate_uuid: "d784628a1b6f4db387b149570005cf2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" is_pre_election: true
I20260709 12:28:58.088009 15051 leader_election.cc:304] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946d0404898c4d6cbb83c33bdf4b1920, d784628a1b6f4db387b149570005cf2e; no voters: 
I20260709 12:28:58.088235 15011 raft_consensus.cc:2468] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d784628a1b6f4db387b149570005cf2e in term 0.
I20260709 12:28:58.089107 15132 raft_consensus.cc:2804] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:58.089661 15132 raft_consensus.cc:493] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:58.090013 15132 raft_consensus.cc:3060] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:58.095749 15132 raft_consensus.cc:515] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.097539 15132 leader_election.cc:290] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [CANDIDATE]: Term 1 election: Requested vote from peers 946d0404898c4d6cbb83c33bdf4b1920 (127.2.4.193:46073), 5cd7bd327ce34244aabf4f9be1142e6e (127.2.4.194:39797)
I20260709 12:28:58.098862 14936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859e9eaf687f4ccaaf30f3532b5c0baa" candidate_uuid: "d784628a1b6f4db387b149570005cf2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946d0404898c4d6cbb83c33bdf4b1920"
I20260709 12:28:58.098855 15011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859e9eaf687f4ccaaf30f3532b5c0baa" candidate_uuid: "d784628a1b6f4db387b149570005cf2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd7bd327ce34244aabf4f9be1142e6e"
I20260709 12:28:58.099789 14936 raft_consensus.cc:3060] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:58.099825 15011 raft_consensus.cc:3060] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:58.106379 15011 raft_consensus.cc:2468] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d784628a1b6f4db387b149570005cf2e in term 1.
I20260709 12:28:58.107687 14936 raft_consensus.cc:2468] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d784628a1b6f4db387b149570005cf2e in term 1.
I20260709 12:28:58.107875 15050 leader_election.cc:304] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd7bd327ce34244aabf4f9be1142e6e, d784628a1b6f4db387b149570005cf2e; no voters: 
I20260709 12:28:58.109469 15132 raft_consensus.cc:2804] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:58.110836 15132 raft_consensus.cc:697] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 1 LEADER]: Becoming Leader. State: Replica: d784628a1b6f4db387b149570005cf2e, State: Running, Role: LEADER
I20260709 12:28:58.111735 15132 consensus_queue.cc:237] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.120272 14832 catalog_manager.cc:5697] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e reported cstate change: term changed from 0 to 1, leader changed from <none> to d784628a1b6f4db387b149570005cf2e (127.2.4.195). New cstate: current_term: 1 leader_uuid: "d784628a1b6f4db387b149570005cf2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:58.127496 15131 raft_consensus.cc:493] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.128159 15131 raft_consensus.cc:515] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.133034 15131 leader_election.cc:290] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946d0404898c4d6cbb83c33bdf4b1920 (127.2.4.193:46073), d784628a1b6f4db387b149570005cf2e (127.2.4.195:44693)
I20260709 12:28:58.143842 14936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fc96feabe64019ad8f09667ce28500" candidate_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946d0404898c4d6cbb83c33bdf4b1920" is_pre_election: true
I20260709 12:28:58.144151 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fc96feabe64019ad8f09667ce28500" candidate_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d784628a1b6f4db387b149570005cf2e" is_pre_election: true
I20260709 12:28:58.144698 14936 raft_consensus.cc:2468] T 87fc96feabe64019ad8f09667ce28500 P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd7bd327ce34244aabf4f9be1142e6e in term 0.
I20260709 12:28:58.145076 15085 raft_consensus.cc:2468] T 87fc96feabe64019ad8f09667ce28500 P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd7bd327ce34244aabf4f9be1142e6e in term 0.
I20260709 12:28:58.146083 14977 leader_election.cc:304] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd7bd327ce34244aabf4f9be1142e6e, 946d0404898c4d6cbb83c33bdf4b1920; no voters: 
I20260709 12:28:58.147212 15131 raft_consensus.cc:2804] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:58.147699 15131 raft_consensus.cc:493] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:58.147977 15131 raft_consensus.cc:3060] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:58.155412 15131 raft_consensus.cc:515] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.157572 15131 leader_election.cc:290] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [CANDIDATE]: Term 1 election: Requested vote from peers 946d0404898c4d6cbb83c33bdf4b1920 (127.2.4.193:46073), d784628a1b6f4db387b149570005cf2e (127.2.4.195:44693)
I20260709 12:28:58.158720 14936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fc96feabe64019ad8f09667ce28500" candidate_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946d0404898c4d6cbb83c33bdf4b1920"
I20260709 12:28:58.159428 14936 raft_consensus.cc:3060] T 87fc96feabe64019ad8f09667ce28500 P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:58.159317 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87fc96feabe64019ad8f09667ce28500" candidate_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d784628a1b6f4db387b149570005cf2e"
I20260709 12:28:58.160063 15085 raft_consensus.cc:3060] T 87fc96feabe64019ad8f09667ce28500 P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:58.166399 14936 raft_consensus.cc:2468] T 87fc96feabe64019ad8f09667ce28500 P 946d0404898c4d6cbb83c33bdf4b1920 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd7bd327ce34244aabf4f9be1142e6e in term 1.
I20260709 12:28:58.167106 15085 raft_consensus.cc:2468] T 87fc96feabe64019ad8f09667ce28500 P d784628a1b6f4db387b149570005cf2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd7bd327ce34244aabf4f9be1142e6e in term 1.
I20260709 12:28:58.168134 14977 leader_election.cc:304] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd7bd327ce34244aabf4f9be1142e6e, 946d0404898c4d6cbb83c33bdf4b1920; no voters: 
I20260709 12:28:58.169234 15131 raft_consensus.cc:2804] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:58.170713 15131 raft_consensus.cc:697] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 LEADER]: Becoming Leader. State: Replica: 5cd7bd327ce34244aabf4f9be1142e6e, State: Running, Role: LEADER
I20260709 12:28:58.171733 15131 consensus_queue.cc:237] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.179909 14832 catalog_manager.cc:5697] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd7bd327ce34244aabf4f9be1142e6e (127.2.4.194). New cstate: current_term: 1 leader_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:58.230790 14832 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59286:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:58.233584 14832 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:58.261463 14926 tablet_service.cc:1511] Processing CreateTablet for tablet a583c39b20c2407994f7a88b6f84f21b (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=6e6ea3e9eb524494b270d76c82d32ab6]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:58.262969 14926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a583c39b20c2407994f7a88b6f84f21b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:58.263082 15001 tablet_service.cc:1511] Processing CreateTablet for tablet a583c39b20c2407994f7a88b6f84f21b (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=6e6ea3e9eb524494b270d76c82d32ab6]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:58.264456 15001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a583c39b20c2407994f7a88b6f84f21b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:58.267192 15074 tablet_service.cc:1511] Processing CreateTablet for tablet a583c39b20c2407994f7a88b6f84f21b (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=6e6ea3e9eb524494b270d76c82d32ab6]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:58.268636 15074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a583c39b20c2407994f7a88b6f84f21b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:58.268471 14925 tablet_service.cc:1511] Processing CreateTablet for tablet 2ff417c4ce7047a28bc432b279873e3a (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=6e6ea3e9eb524494b270d76c82d32ab6]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:58.270430 14925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ff417c4ce7047a28bc432b279873e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:58.270300 15000 tablet_service.cc:1511] Processing CreateTablet for tablet 2ff417c4ce7047a28bc432b279873e3a (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=6e6ea3e9eb524494b270d76c82d32ab6]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:58.272028 15000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ff417c4ce7047a28bc432b279873e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:58.283003 15126 tablet_bootstrap.cc:492] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e: Bootstrap starting.
I20260709 12:28:58.288229 15075 tablet_service.cc:1511] Processing CreateTablet for tablet 2ff417c4ce7047a28bc432b279873e3a (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=6e6ea3e9eb524494b270d76c82d32ab6]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:58.290040 15075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ff417c4ce7047a28bc432b279873e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:58.290072 15126 tablet_bootstrap.cc:654] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:58.292623 15127 tablet_bootstrap.cc:492] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e: Bootstrap starting.
I20260709 12:28:58.304517 15125 tablet_bootstrap.cc:492] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920: Bootstrap starting.
I20260709 12:28:58.309027 15126 tablet_bootstrap.cc:492] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e: No bootstrap required, opened a new log
I20260709 12:28:58.309581 15127 tablet_bootstrap.cc:654] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:58.312197 15125 tablet_bootstrap.cc:654] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:58.309751 15126 ts_tablet_manager.cc:1403] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e: Time spent bootstrapping tablet: real 0.027s	user 0.005s	sys 0.018s
I20260709 12:28:58.316677 15126 raft_consensus.cc:359] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.317795 15126 raft_consensus.cc:385] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:58.318248 15126 raft_consensus.cc:740] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd7bd327ce34244aabf4f9be1142e6e, State: Initialized, Role: FOLLOWER
I20260709 12:28:58.319568 15126 consensus_queue.cc:260] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.323266 15126 ts_tablet_manager.cc:1434] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:28:58.325552 15126 tablet_bootstrap.cc:492] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e: Bootstrap starting.
I20260709 12:28:58.328455 15125 tablet_bootstrap.cc:492] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920: No bootstrap required, opened a new log
I20260709 12:28:58.329135 15125 ts_tablet_manager.cc:1403] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.005s
I20260709 12:28:58.329295 15127 tablet_bootstrap.cc:492] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e: No bootstrap required, opened a new log
I20260709 12:28:58.329883 15127 ts_tablet_manager.cc:1403] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.010s
I20260709 12:28:58.332523 15125 raft_consensus.cc:359] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.332939 15127 raft_consensus.cc:359] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.333495 15125 raft_consensus.cc:385] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:58.333767 15127 raft_consensus.cc:385] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:58.333882 15125 raft_consensus.cc:740] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946d0404898c4d6cbb83c33bdf4b1920, State: Initialized, Role: FOLLOWER
I20260709 12:28:58.334146 15127 raft_consensus.cc:740] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d784628a1b6f4db387b149570005cf2e, State: Initialized, Role: FOLLOWER
I20260709 12:28:58.333854 15126 tablet_bootstrap.cc:654] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:58.335533 15125 consensus_queue.cc:260] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.336508 15127 consensus_queue.cc:260] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.339217 15127 ts_tablet_manager.cc:1434] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:28:58.340391 15127 tablet_bootstrap.cc:492] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e: Bootstrap starting.
I20260709 12:28:58.346146 15127 tablet_bootstrap.cc:654] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:58.354959 15125 ts_tablet_manager.cc:1434] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 12:28:58.358029 15125 tablet_bootstrap.cc:492] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920: Bootstrap starting.
I20260709 12:28:58.369053 15125 tablet_bootstrap.cc:654] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:58.370355 15126 tablet_bootstrap.cc:492] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e: No bootstrap required, opened a new log
I20260709 12:28:58.370925 15126 ts_tablet_manager.cc:1403] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e: Time spent bootstrapping tablet: real 0.047s	user 0.012s	sys 0.022s
I20260709 12:28:58.371413 15131 raft_consensus.cc:493] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.372046 15131 raft_consensus.cc:515] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.374296 15131 leader_election.cc:290] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946d0404898c4d6cbb83c33bdf4b1920 (127.2.4.193:46073), d784628a1b6f4db387b149570005cf2e (127.2.4.195:44693)
I20260709 12:28:58.375213 15125 tablet_bootstrap.cc:492] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920: No bootstrap required, opened a new log
I20260709 12:28:58.375384 14936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a583c39b20c2407994f7a88b6f84f21b" candidate_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946d0404898c4d6cbb83c33bdf4b1920" is_pre_election: true
I20260709 12:28:58.375818 15125 ts_tablet_manager.cc:1403] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.003s
I20260709 12:28:58.375754 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a583c39b20c2407994f7a88b6f84f21b" candidate_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d784628a1b6f4db387b149570005cf2e" is_pre_election: true
I20260709 12:28:58.376266 14936 raft_consensus.cc:2468] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd7bd327ce34244aabf4f9be1142e6e in term 0.
I20260709 12:28:58.376731 15127 tablet_bootstrap.cc:492] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e: No bootstrap required, opened a new log
I20260709 12:28:58.376753 15085 raft_consensus.cc:2468] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd7bd327ce34244aabf4f9be1142e6e in term 0.
I20260709 12:28:58.377979 14977 leader_election.cc:304] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd7bd327ce34244aabf4f9be1142e6e, 946d0404898c4d6cbb83c33bdf4b1920; no voters: 
I20260709 12:28:58.378432 15127 ts_tablet_manager.cc:1403] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.012s
I20260709 12:28:58.379122 15131 raft_consensus.cc:2804] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:58.379573 15131 raft_consensus.cc:493] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:58.379402 15125 raft_consensus.cc:359] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.380093 15131 raft_consensus.cc:3060] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:58.380235 15125 raft_consensus.cc:385] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:58.380769 15125 raft_consensus.cc:740] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946d0404898c4d6cbb83c33bdf4b1920, State: Initialized, Role: FOLLOWER
I20260709 12:28:58.381548 15125 consensus_queue.cc:260] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.382071 15127 raft_consensus.cc:359] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.382915 15127 raft_consensus.cc:385] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:58.383189 15127 raft_consensus.cc:740] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d784628a1b6f4db387b149570005cf2e, State: Initialized, Role: FOLLOWER
I20260709 12:28:58.384259 15125 ts_tablet_manager.cc:1434] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:28:58.384065 15127 consensus_queue.cc:260] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.374202 15126 raft_consensus.cc:359] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.386516 15126 raft_consensus.cc:385] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:58.386808 15126 raft_consensus.cc:740] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd7bd327ce34244aabf4f9be1142e6e, State: Initialized, Role: FOLLOWER
I20260709 12:28:58.387617 15126 consensus_queue.cc:260] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.389706 15131 raft_consensus.cc:515] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.393255 14936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a583c39b20c2407994f7a88b6f84f21b" candidate_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946d0404898c4d6cbb83c33bdf4b1920"
I20260709 12:28:58.393555 15127 ts_tablet_manager.cc:1434] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:28:58.393956 14936 raft_consensus.cc:3060] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:58.397516 15131 leader_election.cc:290] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [CANDIDATE]: Term 1 election: Requested vote from peers 946d0404898c4d6cbb83c33bdf4b1920 (127.2.4.193:46073), d784628a1b6f4db387b149570005cf2e (127.2.4.195:44693)
I20260709 12:28:58.390141 15126 ts_tablet_manager.cc:1434] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:28:58.399215 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a583c39b20c2407994f7a88b6f84f21b" candidate_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d784628a1b6f4db387b149570005cf2e"
I20260709 12:28:58.400012 15085 raft_consensus.cc:3060] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:58.403201 14936 raft_consensus.cc:2468] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd7bd327ce34244aabf4f9be1142e6e in term 1.
I20260709 12:28:58.404742 14977 leader_election.cc:304] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd7bd327ce34244aabf4f9be1142e6e, 946d0404898c4d6cbb83c33bdf4b1920; no voters: 
I20260709 12:28:58.405802 15131 raft_consensus.cc:2804] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:58.406366 15131 raft_consensus.cc:697] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 LEADER]: Becoming Leader. State: Replica: 5cd7bd327ce34244aabf4f9be1142e6e, State: Running, Role: LEADER
I20260709 12:28:58.407290 15131 consensus_queue.cc:237] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.417423 15085 raft_consensus.cc:2468] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd7bd327ce34244aabf4f9be1142e6e in term 1.
I20260709 12:28:58.416778 14833 catalog_manager.cc:5697] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd7bd327ce34244aabf4f9be1142e6e (127.2.4.194). New cstate: current_term: 1 leader_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:58.512792 15131 raft_consensus.cc:493] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.513530 15131 raft_consensus.cc:515] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.515611 15131 leader_election.cc:290] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946d0404898c4d6cbb83c33bdf4b1920 (127.2.4.193:46073), d784628a1b6f4db387b149570005cf2e (127.2.4.195:44693)
I20260709 12:28:58.516862 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff417c4ce7047a28bc432b279873e3a" candidate_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d784628a1b6f4db387b149570005cf2e" is_pre_election: true
I20260709 12:28:58.516742 14936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff417c4ce7047a28bc432b279873e3a" candidate_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946d0404898c4d6cbb83c33bdf4b1920" is_pre_election: true
I20260709 12:28:58.517786 15085 raft_consensus.cc:2468] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd7bd327ce34244aabf4f9be1142e6e in term 0.
I20260709 12:28:58.517997 14936 raft_consensus.cc:2468] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd7bd327ce34244aabf4f9be1142e6e in term 0.
I20260709 12:28:58.519060 14975 leader_election.cc:304] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd7bd327ce34244aabf4f9be1142e6e, d784628a1b6f4db387b149570005cf2e; no voters: 
I20260709 12:28:58.520035 15131 raft_consensus.cc:2804] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:58.520493 15131 raft_consensus.cc:493] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:58.520845 15131 raft_consensus.cc:3060] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:58.522992 15132 consensus_queue.cc:1048] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:58.530155 15131 raft_consensus.cc:515] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.537695 15138 consensus_queue.cc:1048] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:58.539075 14935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff417c4ce7047a28bc432b279873e3a" candidate_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946d0404898c4d6cbb83c33bdf4b1920"
I20260709 12:28:58.539952 14935 raft_consensus.cc:3060] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:58.550410 15131 leader_election.cc:290] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [CANDIDATE]: Term 1 election: Requested vote from peers 946d0404898c4d6cbb83c33bdf4b1920 (127.2.4.193:46073), d784628a1b6f4db387b149570005cf2e (127.2.4.195:44693)
I20260709 12:28:58.552073 15085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff417c4ce7047a28bc432b279873e3a" candidate_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d784628a1b6f4db387b149570005cf2e"
I20260709 12:28:58.552901 15085 raft_consensus.cc:3060] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:58.553834 14935 raft_consensus.cc:2468] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd7bd327ce34244aabf4f9be1142e6e in term 1.
I20260709 12:28:58.555325 14977 leader_election.cc:304] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd7bd327ce34244aabf4f9be1142e6e, 946d0404898c4d6cbb83c33bdf4b1920; no voters: 
I20260709 12:28:58.556600 15131 raft_consensus.cc:2804] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:58.557314 15131 raft_consensus.cc:697] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 LEADER]: Becoming Leader. State: Replica: 5cd7bd327ce34244aabf4f9be1142e6e, State: Running, Role: LEADER
I20260709 12:28:58.558609 15131 consensus_queue.cc:237] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.565847 15142 consensus_queue.cc:1048] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:58.578758 15085 raft_consensus.cc:2468] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd7bd327ce34244aabf4f9be1142e6e in term 1.
I20260709 12:28:58.584113 14832 catalog_manager.cc:5697] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd7bd327ce34244aabf4f9be1142e6e (127.2.4.194). New cstate: current_term: 1 leader_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:58.592363 15161 consensus_queue.cc:1048] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:58.636261 15084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "859e9eaf687f4ccaaf30f3532b5c0baa"
dest_uuid: "d784628a1b6f4db387b149570005cf2e"
mode: GRACEFUL
new_leader_uuid: "946d0404898c4d6cbb83c33bdf4b1920"
 from {username='slave'} at 127.0.0.1:58470
I20260709 12:28:58.636958 15084 raft_consensus.cc:606] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 1 LEADER]: Received request to transfer leadership to TS 946d0404898c4d6cbb83c33bdf4b1920
I20260709 12:28:58.659675 15011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a583c39b20c2407994f7a88b6f84f21b"
dest_uuid: "5cd7bd327ce34244aabf4f9be1142e6e"
mode: GRACEFUL
new_leader_uuid: "946d0404898c4d6cbb83c33bdf4b1920"
 from {username='slave'} at 127.0.0.1:60734
I20260709 12:28:58.660383 15011 raft_consensus.cc:606] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 LEADER]: Received request to transfer leadership to TS 946d0404898c4d6cbb83c33bdf4b1920
I20260709 12:28:58.667263 15011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ff417c4ce7047a28bc432b279873e3a"
dest_uuid: "5cd7bd327ce34244aabf4f9be1142e6e"
mode: GRACEFUL
new_leader_uuid: "946d0404898c4d6cbb83c33bdf4b1920"
 from {username='slave'} at 127.0.0.1:60734
I20260709 12:28:58.668054 15011 raft_consensus.cc:606] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 LEADER]: Received request to transfer leadership to TS 946d0404898c4d6cbb83c33bdf4b1920
I20260709 12:28:58.817408 15164 consensus_queue.cc:1048] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:58.834272 15142 raft_consensus.cc:993] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e: : Instructing follower 946d0404898c4d6cbb83c33bdf4b1920 to start an election
I20260709 12:28:58.834791 15142 raft_consensus.cc:1081] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 LEADER]: Signalling peer 946d0404898c4d6cbb83c33bdf4b1920 to start an election
I20260709 12:28:58.837603 14936 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a583c39b20c2407994f7a88b6f84f21b"
dest_uuid: "946d0404898c4d6cbb83c33bdf4b1920"
 from {username='slave'} at 127.0.0.1:60052
I20260709 12:28:58.842173 14936 raft_consensus.cc:493] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:58.842653 14936 raft_consensus.cc:3060] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:58.845894 15164 consensus_queue.cc:1048] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:58.854912 14936 raft_consensus.cc:515] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.867681 14936 leader_election.cc:290] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [CANDIDATE]: Term 2 election: Requested vote from peers 5cd7bd327ce34244aabf4f9be1142e6e (127.2.4.194:39797), d784628a1b6f4db387b149570005cf2e (127.2.4.195:44693)
I20260709 12:28:58.873287 15011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a583c39b20c2407994f7a88b6f84f21b" candidate_uuid: "946d0404898c4d6cbb83c33bdf4b1920" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5cd7bd327ce34244aabf4f9be1142e6e"
I20260709 12:28:58.874179 15011 raft_consensus.cc:3055] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:58.874482 15011 raft_consensus.cc:740] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5cd7bd327ce34244aabf4f9be1142e6e, State: Running, Role: LEADER
I20260709 12:28:58.875728 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a583c39b20c2407994f7a88b6f84f21b" candidate_uuid: "946d0404898c4d6cbb83c33bdf4b1920" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d784628a1b6f4db387b149570005cf2e"
I20260709 12:28:58.875476 15011 consensus_queue.cc:260] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.876562 15084 raft_consensus.cc:3060] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:58.877214 15011 raft_consensus.cc:3060] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:58.886166 15084 raft_consensus.cc:2468] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946d0404898c4d6cbb83c33bdf4b1920 in term 2.
I20260709 12:28:58.886657 15011 raft_consensus.cc:2468] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946d0404898c4d6cbb83c33bdf4b1920 in term 2.
I20260709 12:28:58.888136 14900 leader_election.cc:304] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5cd7bd327ce34244aabf4f9be1142e6e, 946d0404898c4d6cbb83c33bdf4b1920, d784628a1b6f4db387b149570005cf2e; no voters: 
I20260709 12:28:58.889292 15133 raft_consensus.cc:2804] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:58.890463 15133 raft_consensus.cc:697] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 LEADER]: Becoming Leader. State: Replica: 946d0404898c4d6cbb83c33bdf4b1920, State: Running, Role: LEADER
I20260709 12:28:58.891415 15133 consensus_queue.cc:237] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.898226 14832 catalog_manager.cc:5697] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 reported cstate change: term changed from 1 to 2, leader changed from 5cd7bd327ce34244aabf4f9be1142e6e (127.2.4.194) to 946d0404898c4d6cbb83c33bdf4b1920 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "946d0404898c4d6cbb83c33bdf4b1920" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:58.974836 15132 raft_consensus.cc:993] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e: : Instructing follower 946d0404898c4d6cbb83c33bdf4b1920 to start an election
I20260709 12:28:58.975404 15132 raft_consensus.cc:1081] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 1 LEADER]: Signalling peer 946d0404898c4d6cbb83c33bdf4b1920 to start an election
I20260709 12:28:58.977274 14936 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "859e9eaf687f4ccaaf30f3532b5c0baa"
dest_uuid: "946d0404898c4d6cbb83c33bdf4b1920"
 from {username='slave'} at 127.0.0.1:60046
I20260709 12:28:58.977932 14936 raft_consensus.cc:493] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:58.978215 14936 raft_consensus.cc:3060] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:58.984323 14936 raft_consensus.cc:515] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.986147 14936 leader_election.cc:290] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [CANDIDATE]: Term 2 election: Requested vote from peers 5cd7bd327ce34244aabf4f9be1142e6e (127.2.4.194:39797), d784628a1b6f4db387b149570005cf2e (127.2.4.195:44693)
I20260709 12:28:58.987502 15011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859e9eaf687f4ccaaf30f3532b5c0baa" candidate_uuid: "946d0404898c4d6cbb83c33bdf4b1920" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5cd7bd327ce34244aabf4f9be1142e6e"
I20260709 12:28:58.987540 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859e9eaf687f4ccaaf30f3532b5c0baa" candidate_uuid: "946d0404898c4d6cbb83c33bdf4b1920" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d784628a1b6f4db387b149570005cf2e"
I20260709 12:28:58.988165 15011 raft_consensus.cc:3060] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:58.988519 15084 raft_consensus.cc:3055] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:58.988919 15084 raft_consensus.cc:740] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d784628a1b6f4db387b149570005cf2e, State: Running, Role: LEADER
I20260709 12:28:58.989923 15084 consensus_queue.cc:260] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:58.991464 15084 raft_consensus.cc:3060] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:58.995388 15011 raft_consensus.cc:2468] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946d0404898c4d6cbb83c33bdf4b1920 in term 2.
I20260709 12:28:58.997045 14901 leader_election.cc:304] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd7bd327ce34244aabf4f9be1142e6e, 946d0404898c4d6cbb83c33bdf4b1920; no voters: 
I20260709 12:28:58.998090 15133 raft_consensus.cc:2804] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:58.998620 15133 raft_consensus.cc:697] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 LEADER]: Becoming Leader. State: Replica: 946d0404898c4d6cbb83c33bdf4b1920, State: Running, Role: LEADER
I20260709 12:28:58.999507 15133 consensus_queue.cc:237] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:59.000160 15084 raft_consensus.cc:2468] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946d0404898c4d6cbb83c33bdf4b1920 in term 2.
I20260709 12:28:59.007694 14832 catalog_manager.cc:5697] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 reported cstate change: term changed from 1 to 2, leader changed from d784628a1b6f4db387b149570005cf2e (127.2.4.195) to 946d0404898c4d6cbb83c33bdf4b1920 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "946d0404898c4d6cbb83c33bdf4b1920" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:59.017256 15164 consensus_queue.cc:1048] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:59.026788 15161 raft_consensus.cc:993] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e: : Instructing follower 946d0404898c4d6cbb83c33bdf4b1920 to start an election
I20260709 12:28:59.027401 15161 raft_consensus.cc:1081] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 LEADER]: Signalling peer 946d0404898c4d6cbb83c33bdf4b1920 to start an election
I20260709 12:28:59.028959 15161 consensus_queue.cc:1048] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:59.029908 14935 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2ff417c4ce7047a28bc432b279873e3a"
dest_uuid: "946d0404898c4d6cbb83c33bdf4b1920"
 from {username='slave'} at 127.0.0.1:60052
I20260709 12:28:59.031204 14935 raft_consensus.cc:493] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:59.031675 14935 raft_consensus.cc:3060] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:59.042279 14935 raft_consensus.cc:515] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:59.044852 14935 leader_election.cc:290] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [CANDIDATE]: Term 2 election: Requested vote from peers 5cd7bd327ce34244aabf4f9be1142e6e (127.2.4.194:39797), d784628a1b6f4db387b149570005cf2e (127.2.4.195:44693)
I20260709 12:28:59.045761 15011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff417c4ce7047a28bc432b279873e3a" candidate_uuid: "946d0404898c4d6cbb83c33bdf4b1920" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5cd7bd327ce34244aabf4f9be1142e6e"
I20260709 12:28:59.046538 15011 raft_consensus.cc:3055] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:59.046842 15011 raft_consensus.cc:740] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5cd7bd327ce34244aabf4f9be1142e6e, State: Running, Role: LEADER
I20260709 12:28:59.047591 15084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff417c4ce7047a28bc432b279873e3a" candidate_uuid: "946d0404898c4d6cbb83c33bdf4b1920" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d784628a1b6f4db387b149570005cf2e"
I20260709 12:28:59.047787 15011 consensus_queue.cc:260] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:59.048377 15084 raft_consensus.cc:3060] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:59.048964 15011 raft_consensus.cc:3060] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:59.057233 15084 raft_consensus.cc:2468] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946d0404898c4d6cbb83c33bdf4b1920 in term 2.
I20260709 12:28:59.057715 15011 raft_consensus.cc:2468] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 946d0404898c4d6cbb83c33bdf4b1920 in term 2.
I20260709 12:28:59.059005 14900 leader_election.cc:304] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946d0404898c4d6cbb83c33bdf4b1920, d784628a1b6f4db387b149570005cf2e; no voters: 
I20260709 12:28:59.060060 15133 raft_consensus.cc:2804] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:59.060632 15133 raft_consensus.cc:697] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 LEADER]: Becoming Leader. State: Replica: 946d0404898c4d6cbb83c33bdf4b1920, State: Running, Role: LEADER
I20260709 12:28:59.061534 15133 consensus_queue.cc:237] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:59.068768 14834 catalog_manager.cc:5697] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 reported cstate change: term changed from 1 to 2, leader changed from 5cd7bd327ce34244aabf4f9be1142e6e (127.2.4.194) to 946d0404898c4d6cbb83c33bdf4b1920 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "946d0404898c4d6cbb83c33bdf4b1920" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:59.179064  2067 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:28:59.181095  2067 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:28:59.181711  2067 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:28:59.196771 14935 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ff417c4ce7047a28bc432b279873e3a"
dest_uuid: "946d0404898c4d6cbb83c33bdf4b1920"
mode: GRACEFUL
new_leader_uuid: "d784628a1b6f4db387b149570005cf2e"
 from {username='slave'} at 127.0.0.1:60058
I20260709 12:28:59.197417 14935 raft_consensus.cc:606] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 LEADER]: Received request to transfer leadership to TS d784628a1b6f4db387b149570005cf2e
I20260709 12:28:59.198899  2067 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:28:59.296314 15011 raft_consensus.cc:1275] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 2 FOLLOWER]: Refusing update from remote peer 946d0404898c4d6cbb83c33bdf4b1920: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:59.297972 15133 consensus_queue.cc:1048] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:59.309769 15084 raft_consensus.cc:1275] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e [term 2 FOLLOWER]: Refusing update from remote peer 946d0404898c4d6cbb83c33bdf4b1920: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:59.313481 15133 consensus_queue.cc:1048] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:59.504928 15084 raft_consensus.cc:1275] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 2 FOLLOWER]: Refusing update from remote peer 946d0404898c4d6cbb83c33bdf4b1920: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:59.506968 15177 consensus_queue.cc:1048] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:59.518151 15011 raft_consensus.cc:1275] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e [term 2 FOLLOWER]: Refusing update from remote peer 946d0404898c4d6cbb83c33bdf4b1920: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:59.520613 15133 consensus_queue.cc:1048] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:59.590936 15011 raft_consensus.cc:1275] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 2 FOLLOWER]: Refusing update from remote peer 946d0404898c4d6cbb83c33bdf4b1920: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:59.592720 15133 consensus_queue.cc:1048] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:59.603952 15084 raft_consensus.cc:1275] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 2 FOLLOWER]: Refusing update from remote peer 946d0404898c4d6cbb83c33bdf4b1920: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:59.607194 15178 consensus_queue.cc:1048] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:59.616290 15133 raft_consensus.cc:993] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920: : Instructing follower d784628a1b6f4db387b149570005cf2e to start an election
I20260709 12:28:59.616886 15178 raft_consensus.cc:1081] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 LEADER]: Signalling peer d784628a1b6f4db387b149570005cf2e to start an election
I20260709 12:28:59.618822 15084 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2ff417c4ce7047a28bc432b279873e3a"
dest_uuid: "d784628a1b6f4db387b149570005cf2e"
 from {username='slave'} at 127.0.0.1:58474
I20260709 12:28:59.621169 15084 raft_consensus.cc:493] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:59.621608 15084 raft_consensus.cc:3060] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.628562 15084 raft_consensus.cc:515] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:59.630753 15084 leader_election.cc:290] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [CANDIDATE]: Term 3 election: Requested vote from peers 946d0404898c4d6cbb83c33bdf4b1920 (127.2.4.193:46073), 5cd7bd327ce34244aabf4f9be1142e6e (127.2.4.194:39797)
I20260709 12:28:59.631809 14935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff417c4ce7047a28bc432b279873e3a" candidate_uuid: "d784628a1b6f4db387b149570005cf2e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "946d0404898c4d6cbb83c33bdf4b1920"
I20260709 12:28:59.632200 15011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff417c4ce7047a28bc432b279873e3a" candidate_uuid: "d784628a1b6f4db387b149570005cf2e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5cd7bd327ce34244aabf4f9be1142e6e"
I20260709 12:28:59.632560 14935 raft_consensus.cc:3055] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:59.632966 14935 raft_consensus.cc:740] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 946d0404898c4d6cbb83c33bdf4b1920, State: Running, Role: LEADER
I20260709 12:28:59.632988 15011 raft_consensus.cc:3060] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.633881 14935 consensus_queue.cc:260] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:59.634977 14935 raft_consensus.cc:3060] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.639797 15011 raft_consensus.cc:2468] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d784628a1b6f4db387b149570005cf2e in term 3.
I20260709 12:28:59.641009 14935 raft_consensus.cc:2468] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d784628a1b6f4db387b149570005cf2e in term 3.
I20260709 12:28:59.641366 15050 leader_election.cc:304] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd7bd327ce34244aabf4f9be1142e6e, d784628a1b6f4db387b149570005cf2e; no voters: 
I20260709 12:28:59.642300 15132 raft_consensus.cc:2804] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:59.643375 15132 raft_consensus.cc:697] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 3 LEADER]: Becoming Leader. State: Replica: d784628a1b6f4db387b149570005cf2e, State: Running, Role: LEADER
I20260709 12:28:59.644236 15132 consensus_queue.cc:237] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } }
I20260709 12:28:59.650981 14833 catalog_manager.cc:5697] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e reported cstate change: term changed from 2 to 3, leader changed from 946d0404898c4d6cbb83c33bdf4b1920 (127.2.4.193) to d784628a1b6f4db387b149570005cf2e (127.2.4.195). New cstate: current_term: 3 leader_uuid: "d784628a1b6f4db387b149570005cf2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946d0404898c4d6cbb83c33bdf4b1920" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 46073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd7bd327ce34244aabf4f9be1142e6e" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 39797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d784628a1b6f4db387b149570005cf2e" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 44693 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.852437  2067 tablet_server.cc:179] TabletServer@127.2.4.193:0 shutting down...
I20260709 12:28:59.875715  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:59.876485  2067 tablet_replica.cc:333] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920: stopping tablet replica
I20260709 12:28:59.877099  2067 raft_consensus.cc:2243] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.877544  2067 raft_consensus.cc:2272] T 2ff417c4ce7047a28bc432b279873e3a P 946d0404898c4d6cbb83c33bdf4b1920 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.880375  2067 tablet_replica.cc:333] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920: stopping tablet replica
I20260709 12:28:59.880924  2067 raft_consensus.cc:2243] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.881865  2067 raft_consensus.cc:2272] T a583c39b20c2407994f7a88b6f84f21b P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.884570  2067 tablet_replica.cc:333] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920: stopping tablet replica
I20260709 12:28:59.885073  2067 raft_consensus.cc:2243] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.886027  2067 raft_consensus.cc:2272] T 859e9eaf687f4ccaaf30f3532b5c0baa P 946d0404898c4d6cbb83c33bdf4b1920 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.888470  2067 tablet_replica.cc:333] T 87fc96feabe64019ad8f09667ce28500 P 946d0404898c4d6cbb83c33bdf4b1920: stopping tablet replica
I20260709 12:28:59.888985  2067 raft_consensus.cc:2243] T 87fc96feabe64019ad8f09667ce28500 P 946d0404898c4d6cbb83c33bdf4b1920 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.889381  2067 raft_consensus.cc:2272] T 87fc96feabe64019ad8f09667ce28500 P 946d0404898c4d6cbb83c33bdf4b1920 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.914875  2067 tablet_server.cc:196] TabletServer@127.2.4.193:0 shutdown complete.
I20260709 12:28:59.929409  2067 tablet_server.cc:179] TabletServer@127.2.4.194:0 shutting down...
I20260709 12:28:59.952744  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:59.953446  2067 tablet_replica.cc:333] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e: stopping tablet replica
I20260709 12:28:59.954129  2067 raft_consensus.cc:2243] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.954583  2067 raft_consensus.cc:2272] T 2ff417c4ce7047a28bc432b279873e3a P 5cd7bd327ce34244aabf4f9be1142e6e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.956972  2067 tablet_replica.cc:333] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e: stopping tablet replica
I20260709 12:28:59.957518  2067 raft_consensus.cc:2243] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.958060  2067 raft_consensus.cc:2272] T a583c39b20c2407994f7a88b6f84f21b P 5cd7bd327ce34244aabf4f9be1142e6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.960726  2067 tablet_replica.cc:333] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e: stopping tablet replica
I20260709 12:28:59.961309  2067 raft_consensus.cc:2243] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.961850  2067 raft_consensus.cc:2272] T 859e9eaf687f4ccaaf30f3532b5c0baa P 5cd7bd327ce34244aabf4f9be1142e6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.964690  2067 tablet_replica.cc:333] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e: stopping tablet replica
I20260709 12:28:59.965305  2067 raft_consensus.cc:2243] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.966346  2067 raft_consensus.cc:2272] T 87fc96feabe64019ad8f09667ce28500 P 5cd7bd327ce34244aabf4f9be1142e6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.991992  2067 tablet_server.cc:196] TabletServer@127.2.4.194:0 shutdown complete.
I20260709 12:29:00.010284  2067 tablet_server.cc:179] TabletServer@127.2.4.195:0 shutting down...
I20260709 12:29:00.035136  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:00.035938  2067 tablet_replica.cc:333] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e: stopping tablet replica
I20260709 12:29:00.036576  2067 raft_consensus.cc:2243] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:00.037227  2067 pending_rounds.cc:70] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e: Trying to abort 1 pending ops.
I20260709 12:29:00.037408  2067 pending_rounds.cc:77] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305626171990786048 op_type: NO_OP noop_request { }
I20260709 12:29:00.037773  2067 raft_consensus.cc:2889] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:00.038038  2067 raft_consensus.cc:2272] T 2ff417c4ce7047a28bc432b279873e3a P d784628a1b6f4db387b149570005cf2e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.040832  2067 tablet_replica.cc:333] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e: stopping tablet replica
I20260709 12:29:00.041390  2067 raft_consensus.cc:2243] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.041929  2067 raft_consensus.cc:2272] T a583c39b20c2407994f7a88b6f84f21b P d784628a1b6f4db387b149570005cf2e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.044508  2067 tablet_replica.cc:333] T 87fc96feabe64019ad8f09667ce28500 P d784628a1b6f4db387b149570005cf2e: stopping tablet replica
I20260709 12:29:00.045020  2067 raft_consensus.cc:2243] T 87fc96feabe64019ad8f09667ce28500 P d784628a1b6f4db387b149570005cf2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.045415  2067 raft_consensus.cc:2272] T 87fc96feabe64019ad8f09667ce28500 P d784628a1b6f4db387b149570005cf2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.047470  2067 tablet_replica.cc:333] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e: stopping tablet replica
I20260709 12:29:00.047938  2067 raft_consensus.cc:2243] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.048349  2067 raft_consensus.cc:2272] T 859e9eaf687f4ccaaf30f3532b5c0baa P d784628a1b6f4db387b149570005cf2e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.063294  2067 tablet_server.cc:196] TabletServer@127.2.4.195:0 shutdown complete.
I20260709 12:29:00.080360  2067 master.cc:562] Master@127.2.4.254:46203 shutting down...
I20260709 12:29:00.102208  2067 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:00.103075  2067 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.103447  2067 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0c44f86e99a54a12b701a67b16bf881b: stopping tablet replica
I20260709 12:29:00.125058  2067 master.cc:584] Master@127.2.4.254:46203 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (3090 ms)
[----------] 6 tests from LeaderRebalancerTest (412860 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:29:00.160233  2067 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.4.254:42861
I20260709 12:29:00.161581  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:00.168073 15183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:00.168254 15184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:00.171928 15186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:00.172019  2067 server_base.cc:1061] running on GCE node
I20260709 12:29:00.173447  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:00.173753  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:00.173949  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600140173925 us; error 0 us; skew 500 ppm
I20260709 12:29:00.174624  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:00.177588  2067 webserver.cc:533] Webserver started at http://127.2.4.254:33087/ using document root <none> and password file <none>
I20260709 12:29:00.178247  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:00.178475  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:00.178781  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:00.180094  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/master-0-root/instance:
uuid: "244a03776aa247b2b722430191ed6aca"
format_stamp: "Formatted at 2026-07-09 12:29:00 on dist-test-slave-4qvp"
I20260709 12:29:00.185514  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:29:00.190114 15191 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:00.191187  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:29:00.191573  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/master-0-root
uuid: "244a03776aa247b2b722430191ed6aca"
format_stamp: "Formatted at 2026-07-09 12:29:00 on dist-test-slave-4qvp"
I20260709 12:29:00.191928  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:00.225059  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:00.226384  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:00.270849  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.254:42861
I20260709 12:29:00.270956 15242 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.254:42861 every 8 connection(s)
I20260709 12:29:00.275739 15243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.287633 15243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca: Bootstrap starting.
I20260709 12:29:00.292483 15243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:00.297564 15243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca: No bootstrap required, opened a new log
I20260709 12:29:00.299916 15243 raft_consensus.cc:359] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244a03776aa247b2b722430191ed6aca" member_type: VOTER }
I20260709 12:29:00.300362 15243 raft_consensus.cc:385] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:00.300557 15243 raft_consensus.cc:740] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 244a03776aa247b2b722430191ed6aca, State: Initialized, Role: FOLLOWER
I20260709 12:29:00.301092 15243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244a03776aa247b2b722430191ed6aca" member_type: VOTER }
I20260709 12:29:00.301563 15243 raft_consensus.cc:399] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:29:00.301894 15243 raft_consensus.cc:493] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:29:00.302119 15243 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:00.308527 15243 raft_consensus.cc:515] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244a03776aa247b2b722430191ed6aca" member_type: VOTER }
I20260709 12:29:00.309196 15243 leader_election.cc:304] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 244a03776aa247b2b722430191ed6aca; no voters: 
I20260709 12:29:00.310623 15243 leader_election.cc:290] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:29:00.311164 15246 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:00.313143 15246 raft_consensus.cc:697] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [term 1 LEADER]: Becoming Leader. State: Replica: 244a03776aa247b2b722430191ed6aca, State: Running, Role: LEADER
I20260709 12:29:00.314026 15246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244a03776aa247b2b722430191ed6aca" member_type: VOTER }
I20260709 12:29:00.314308 15243 sys_catalog.cc:565] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:29:00.317679 15248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 244a03776aa247b2b722430191ed6aca. Latest consensus state: current_term: 1 leader_uuid: "244a03776aa247b2b722430191ed6aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244a03776aa247b2b722430191ed6aca" member_type: VOTER } }
I20260709 12:29:00.318523 15248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:00.317679 15247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "244a03776aa247b2b722430191ed6aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244a03776aa247b2b722430191ed6aca" member_type: VOTER } }
I20260709 12:29:00.319264 15247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:00.323048 15252 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:29:00.328648 15252 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:29:00.332934  2067 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:29:00.339555 15252 catalog_manager.cc:1383] Generated new cluster ID: 69aa79f93ecc489b953af3a483c05355
I20260709 12:29:00.339898 15252 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:29:00.360328 15252 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:29:00.362111 15252 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:29:00.381191 15252 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca: Generated new TSK 0
I20260709 12:29:00.382001 15252 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:29:00.400601  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:00.407582 15264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:00.408792 15265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:00.410858 15267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:00.411252  2067 server_base.cc:1061] running on GCE node
I20260709 12:29:00.412458  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:00.412659  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:00.412851  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600140412834 us; error 0 us; skew 500 ppm
I20260709 12:29:00.413393  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:00.416415  2067 webserver.cc:533] Webserver started at http://127.2.4.193:37453/ using document root <none> and password file <none>
I20260709 12:29:00.416913  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:00.417083  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:00.417318  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:00.418646  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/ts-0-root/instance:
uuid: "fcf7b677a0b646948a96f66b61cc90bb"
format_stamp: "Formatted at 2026-07-09 12:29:00 on dist-test-slave-4qvp"
I20260709 12:29:00.424518  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:29:00.428915 15272 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:00.430076  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:29:00.430408  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/ts-0-root
uuid: "fcf7b677a0b646948a96f66b61cc90bb"
format_stamp: "Formatted at 2026-07-09 12:29:00 on dist-test-slave-4qvp"
I20260709 12:29:00.430747  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:00.446240  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:00.447614  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:00.449283  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:00.451869  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:00.452067  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:00.452395  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:00.452548  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:00.505297  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.193:34773
I20260709 12:29:00.505450 15334 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.193:34773 every 8 connection(s)
I20260709 12:29:00.511044  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:00.519023 15339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:00.521049 15340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:00.524389 15342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:00.525126  2067 server_base.cc:1061] running on GCE node
I20260709 12:29:00.525884 15335 heartbeater.cc:344] Connected to a master server at 127.2.4.254:42861
I20260709 12:29:00.526305 15335 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:00.527094  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:00.527377  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:00.527313 15335 heartbeater.cc:507] Master 127.2.4.254:42861 requested a full tablet report, sending...
I20260709 12:29:00.527750  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600140527736 us; error 0 us; skew 500 ppm
I20260709 12:29:00.528530  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:00.530558 15208 ts_manager.cc:194] Registered new tserver with Master: fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773)
I20260709 12:29:00.531865  2067 webserver.cc:533] Webserver started at http://127.2.4.194:45479/ using document root <none> and password file <none>
I20260709 12:29:00.532682  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:00.532918  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:00.533243  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:00.533247 15208 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56016
I20260709 12:29:00.534960  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/ts-1-root/instance:
uuid: "6b0dbefdcaa7411a8877659d441f6413"
format_stamp: "Formatted at 2026-07-09 12:29:00 on dist-test-slave-4qvp"
I20260709 12:29:00.540059  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.005s
I20260709 12:29:00.544621 15347 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:00.545854  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:29:00.546316  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/ts-1-root
uuid: "6b0dbefdcaa7411a8877659d441f6413"
format_stamp: "Formatted at 2026-07-09 12:29:00 on dist-test-slave-4qvp"
I20260709 12:29:00.546657  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:00.577319  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:00.578606  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:00.580291  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:00.582901  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:00.583108  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:00.583446  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:00.583593  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:00.629303  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.194:34701
I20260709 12:29:00.629498 15409 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.194:34701 every 8 connection(s)
I20260709 12:29:00.635046  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:00.643453 15413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:00.644006 15414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:00.645946 15416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:00.647923  2067 server_base.cc:1061] running on GCE node
I20260709 12:29:00.649816  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:29:00.650074 15410 heartbeater.cc:344] Connected to a master server at 127.2.4.254:42861
W20260709 12:29:00.650175  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:00.650815  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600140650790 us; error 0 us; skew 500 ppm
I20260709 12:29:00.650889 15410 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:00.651857  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:00.652328 15410 heartbeater.cc:507] Master 127.2.4.254:42861 requested a full tablet report, sending...
I20260709 12:29:00.655985  2067 webserver.cc:533] Webserver started at http://127.2.4.195:37019/ using document root <none> and password file <none>
I20260709 12:29:00.657021 15208 ts_manager.cc:194] Registered new tserver with Master: 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:00.657025  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:00.657852  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:00.658341  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:00.658874 15208 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56030
I20260709 12:29:00.660389  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/ts-2-root/instance:
uuid: "98426102f6f642258ad743626d97034a"
format_stamp: "Formatted at 2026-07-09 12:29:00 on dist-test-slave-4qvp"
I20260709 12:29:00.667089  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.003s	sys 0.004s
I20260709 12:29:00.671379 15421 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:00.672422  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:29:00.672869  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/ts-2-root
uuid: "98426102f6f642258ad743626d97034a"
format_stamp: "Formatted at 2026-07-09 12:29:00 on dist-test-slave-4qvp"
I20260709 12:29:00.673223  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599727094521-2067-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:00.693120  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:00.694499  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:00.695999  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:00.698627  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:00.698930  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:00.699210  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:00.699393  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:00.746973  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.195:46775
I20260709 12:29:00.747119 15483 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.195:46775 every 8 connection(s)
I20260709 12:29:00.764272 15484 heartbeater.cc:344] Connected to a master server at 127.2.4.254:42861
I20260709 12:29:00.764726 15484 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:00.765559 15484 heartbeater.cc:507] Master 127.2.4.254:42861 requested a full tablet report, sending...
I20260709 12:29:00.768106 15208 ts_manager.cc:194] Registered new tserver with Master: 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:00.768965  2067 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017755372s
I20260709 12:29:00.770033 15208 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56034
I20260709 12:29:00.800005 15208 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56036:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:00.802699 15208 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:00.889336 15449 tablet_service.cc:1511] Processing CreateTablet for tablet 4e4e50d080484b2bbf21b074105618f6 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:00.890084 15448 tablet_service.cc:1511] Processing CreateTablet for tablet 14354702f32a45e2b59f53620e8f40da (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:00.890959 15449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4e50d080484b2bbf21b074105618f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.891999 15448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14354702f32a45e2b59f53620e8f40da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.899875 15447 tablet_service.cc:1511] Processing CreateTablet for tablet 8db3593e4616490a977852f03cfb8752 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:00.902963 15300 tablet_service.cc:1511] Processing CreateTablet for tablet 4e4e50d080484b2bbf21b074105618f6 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:00.904565 15300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4e50d080484b2bbf21b074105618f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.905465 15299 tablet_service.cc:1511] Processing CreateTablet for tablet 14354702f32a45e2b59f53620e8f40da (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:00.906190 15297 tablet_service.cc:1511] Processing CreateTablet for tablet 15904f9d56b744f9bf6f12d1fdd2da13 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:00.906872 15299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14354702f32a45e2b59f53620e8f40da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.907974 15297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15904f9d56b744f9bf6f12d1fdd2da13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.909664 15296 tablet_service.cc:1511] Processing CreateTablet for tablet 3919d8a786804398a997ab35ffd8d848 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:00.911104 15296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3919d8a786804398a997ab35ffd8d848. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.911469 15295 tablet_service.cc:1511] Processing CreateTablet for tablet f4c8ccf018614a79bb5fe9f2896231ea (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:00.913216 15295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4c8ccf018614a79bb5fe9f2896231ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.913259 15371 tablet_service.cc:1511] Processing CreateTablet for tablet 3919d8a786804398a997ab35ffd8d848 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:00.915014 15371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3919d8a786804398a997ab35ffd8d848. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.914923 15374 tablet_service.cc:1511] Processing CreateTablet for tablet 14354702f32a45e2b59f53620e8f40da (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:00.916708 15374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14354702f32a45e2b59f53620e8f40da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.918960 15373 tablet_service.cc:1511] Processing CreateTablet for tablet 8db3593e4616490a977852f03cfb8752 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:00.920574 15373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8db3593e4616490a977852f03cfb8752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.922920 15370 tablet_service.cc:1511] Processing CreateTablet for tablet f4c8ccf018614a79bb5fe9f2896231ea (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:00.924597 15370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4c8ccf018614a79bb5fe9f2896231ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.924922 15294 tablet_service.cc:1511] Processing CreateTablet for tablet 81e70726b51a4dabb3e47f6074f33bb4 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:00.926589 15294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81e70726b51a4dabb3e47f6074f33bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.926936 15372 tablet_service.cc:1511] Processing CreateTablet for tablet 15904f9d56b744f9bf6f12d1fdd2da13 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:00.928550 15372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15904f9d56b744f9bf6f12d1fdd2da13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.937943 15375 tablet_service.cc:1511] Processing CreateTablet for tablet 4e4e50d080484b2bbf21b074105618f6 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:00.942936 15369 tablet_service.cc:1511] Processing CreateTablet for tablet 81e70726b51a4dabb3e47f6074f33bb4 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:00.904232 15298 tablet_service.cc:1511] Processing CreateTablet for tablet 8db3593e4616490a977852f03cfb8752 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:00.945504 15375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4e50d080484b2bbf21b074105618f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.945834 15298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8db3593e4616490a977852f03cfb8752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.946718 15445 tablet_service.cc:1511] Processing CreateTablet for tablet 3919d8a786804398a997ab35ffd8d848 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:00.902968 15446 tablet_service.cc:1511] Processing CreateTablet for tablet 15904f9d56b744f9bf6f12d1fdd2da13 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:00.949292 15369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81e70726b51a4dabb3e47f6074f33bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.950083 15447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8db3593e4616490a977852f03cfb8752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.949797 15444 tablet_service.cc:1511] Processing CreateTablet for tablet f4c8ccf018614a79bb5fe9f2896231ea (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:00.951445 15444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4c8ccf018614a79bb5fe9f2896231ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.953475 15446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15904f9d56b744f9bf6f12d1fdd2da13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.954449 15445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3919d8a786804398a997ab35ffd8d848. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.988508 15443 tablet_service.cc:1511] Processing CreateTablet for tablet 81e70726b51a4dabb3e47f6074f33bb4 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:00.990351 15443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81e70726b51a4dabb3e47f6074f33bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:00.998111 15299 tablet_service.cc:1511] Processing CreateTablet for tablet 3540108fcddc4ca1b11dc45a9d2ea775 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:00.999773 15299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3540108fcddc4ca1b11dc45a9d2ea775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:01.006737 15368 tablet_service.cc:1511] Processing CreateTablet for tablet 3540108fcddc4ca1b11dc45a9d2ea775 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:01.008584 15368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3540108fcddc4ca1b11dc45a9d2ea775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:01.018211 15499 tablet_bootstrap.cc:492] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:01.033643 15499 tablet_bootstrap.cc:654] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.057439 15447 tablet_service.cc:1511] Processing CreateTablet for tablet 3540108fcddc4ca1b11dc45a9d2ea775 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:01.061295 15500 tablet_bootstrap.cc:492] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:01.061987 15447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3540108fcddc4ca1b11dc45a9d2ea775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:01.062131 15298 tablet_service.cc:1511] Processing CreateTablet for tablet 02a7fd763e6d4686aba882cfae13b560 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:01.063145 15442 tablet_service.cc:1511] Processing CreateTablet for tablet 02a7fd763e6d4686aba882cfae13b560 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:01.063736 15298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02a7fd763e6d4686aba882cfae13b560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:01.064471 15442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02a7fd763e6d4686aba882cfae13b560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:01.070762 15501 tablet_bootstrap.cc:492] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:01.070580 15371 tablet_service.cc:1511] Processing CreateTablet for tablet 02a7fd763e6d4686aba882cfae13b560 (DEFAULT_TABLE table=test-workload [id=4475876d5a804b77bc874aede82bb4de]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:01.072168 15371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02a7fd763e6d4686aba882cfae13b560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:01.079370 15500 tablet_bootstrap.cc:654] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.093228 15501 tablet_bootstrap.cc:654] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.127135 15499 tablet_bootstrap.cc:492] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:01.127727 15499 ts_tablet_manager.cc:1403] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.110s	user 0.012s	sys 0.030s
I20260709 12:29:01.131132 15501 tablet_bootstrap.cc:492] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:01.131824 15501 ts_tablet_manager.cc:1403] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.061s	user 0.015s	sys 0.007s
I20260709 12:29:01.131299 15499 raft_consensus.cc:359] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.133318 15499 raft_consensus.cc:385] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.133648 15499 raft_consensus.cc:740] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.134522 15499 consensus_queue.cc:260] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.145093 15500 tablet_bootstrap.cc:492] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:01.146049 15500 ts_tablet_manager.cc:1403] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.085s	user 0.013s	sys 0.006s
I20260709 12:29:01.137106 15501 raft_consensus.cc:359] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } } peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } }
I20260709 12:29:01.148583 15501 raft_consensus.cc:385] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.149176 15501 raft_consensus.cc:740] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.150271 15501 consensus_queue.cc:260] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } } peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } }
I20260709 12:29:01.151008 15500 raft_consensus.cc:359] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.152210 15500 raft_consensus.cc:385] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.153915 15500 raft_consensus.cc:740] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.154186 15335 heartbeater.cc:499] Master 127.2.4.254:42861 was elected leader, sending a full tablet report...
I20260709 12:29:01.154958 15500 consensus_queue.cc:260] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.158259 15410 heartbeater.cc:499] Master 127.2.4.254:42861 was elected leader, sending a full tablet report...
I20260709 12:29:01.159300 15500 ts_tablet_manager.cc:1434] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 12:29:01.161054 15500 tablet_bootstrap.cc:492] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:01.168265 15499 ts_tablet_manager.cc:1434] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.040s	user 0.016s	sys 0.004s
I20260709 12:29:01.169888 15499 tablet_bootstrap.cc:492] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:01.170194 15500 tablet_bootstrap.cc:654] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.177408 15499 tablet_bootstrap.cc:654] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.179148 15484 heartbeater.cc:499] Master 127.2.4.254:42861 was elected leader, sending a full tablet report...
I20260709 12:29:01.187502 15501 ts_tablet_manager.cc:1434] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.055s	user 0.023s	sys 0.007s
I20260709 12:29:01.189603 15501 tablet_bootstrap.cc:492] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:01.198328 15501 tablet_bootstrap.cc:654] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.198716 15500 tablet_bootstrap.cc:492] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:01.199393 15500 ts_tablet_manager.cc:1403] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.039s	user 0.021s	sys 0.016s
I20260709 12:29:01.202935 15500 raft_consensus.cc:359] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.204021 15500 raft_consensus.cc:385] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.204419 15500 raft_consensus.cc:740] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.205274 15500 consensus_queue.cc:260] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.207333 15501 tablet_bootstrap.cc:492] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:01.207690 15500 ts_tablet_manager.cc:1434] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:29:01.208004 15499 tablet_bootstrap.cc:492] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:01.208050 15501 ts_tablet_manager.cc:1403] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.006s
I20260709 12:29:01.208756 15499 ts_tablet_manager.cc:1403] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.039s	user 0.010s	sys 0.008s
I20260709 12:29:01.208840 15500 tablet_bootstrap.cc:492] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:01.211580 15501 raft_consensus.cc:359] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.212345 15501 raft_consensus.cc:385] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.212414 15499 raft_consensus.cc:359] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.213428 15499 raft_consensus.cc:385] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.215945 15500 tablet_bootstrap.cc:654] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.216295 15499 raft_consensus.cc:740] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.217486 15499 consensus_queue.cc:260] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.220003 15499 ts_tablet_manager.cc:1434] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:29:01.212704 15501 raft_consensus.cc:740] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.223803 15501 consensus_queue.cc:260] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.221273 15499 tablet_bootstrap.cc:492] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:01.229931 15501 ts_tablet_manager.cc:1434] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.002s
I20260709 12:29:01.232347 15501 tablet_bootstrap.cc:492] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:01.235633 15500 tablet_bootstrap.cc:492] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:01.236356 15500 ts_tablet_manager.cc:1403] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.028s	user 0.022s	sys 0.004s
I20260709 12:29:01.239639 15500 raft_consensus.cc:359] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.240312 15501 tablet_bootstrap.cc:654] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.240522 15500 raft_consensus.cc:385] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.240875 15500 raft_consensus.cc:740] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.241760 15500 consensus_queue.cc:260] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.232888 15499 tablet_bootstrap.cc:654] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.244457 15500 ts_tablet_manager.cc:1434] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:29:01.246035 15500 tablet_bootstrap.cc:492] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:01.254462 15500 tablet_bootstrap.cc:654] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.265340 15506 raft_consensus.cc:493] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.265954 15506 raft_consensus.cc:515] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.267406 15500 tablet_bootstrap.cc:492] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:01.267556 15501 tablet_bootstrap.cc:492] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:01.267721 15499 tablet_bootstrap.cc:492] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:01.268024 15500 ts_tablet_manager.cc:1403] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.008s
I20260709 12:29:01.268419 15501 ts_tablet_manager.cc:1403] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.010s
I20260709 12:29:01.268505 15499 ts_tablet_manager.cc:1403] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.048s	user 0.019s	sys 0.011s
I20260709 12:29:01.272351 15500 raft_consensus.cc:359] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.273528 15500 raft_consensus.cc:385] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.274068 15500 raft_consensus.cc:740] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.273838 15499 raft_consensus.cc:359] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } } peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } }
I20260709 12:29:01.275105 15499 raft_consensus.cc:385] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.275501 15499 raft_consensus.cc:740] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.275703 15500 consensus_queue.cc:260] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.276396 15499 consensus_queue.cc:260] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } } peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } }
I20260709 12:29:01.276305 15506 leader_election.cc:290] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:01.279032 15499 ts_tablet_manager.cc:1434] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:01.280222 15499 tablet_bootstrap.cc:492] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:01.288154 15499 tablet_bootstrap.cc:654] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.291253 15500 ts_tablet_manager.cc:1434] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.023s	user 0.010s	sys 0.001s
I20260709 12:29:01.292415 15505 raft_consensus.cc:493] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.292949 15505 raft_consensus.cc:515] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.298853 15507 raft_consensus.cc:493] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.299659 15507 raft_consensus.cc:515] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.302513 15499 tablet_bootstrap.cc:492] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:01.303476 15499 ts_tablet_manager.cc:1403] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.011s
I20260709 12:29:01.309423 15505 leader_election.cc:290] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:01.311197 15507 leader_election.cc:290] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:01.276415 15501 raft_consensus.cc:359] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.318056 15501 raft_consensus.cc:385] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.318501 15501 raft_consensus.cc:740] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.319754 15501 consensus_queue.cc:260] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.323509 15500 tablet_bootstrap.cc:492] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:01.327353 15501 ts_tablet_manager.cc:1434] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.001s
W20260709 12:29:01.327786 15256 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:01.323140 15499 raft_consensus.cc:359] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.329818 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db3593e4616490a977852f03cfb8752" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
I20260709 12:29:01.336090 15499 raft_consensus.cc:385] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.337668 15499 raft_consensus.cc:740] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.336073 15501 tablet_bootstrap.cc:492] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:01.338976 15507 raft_consensus.cc:493] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.338637 15499 consensus_queue.cc:260] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.339584 15507 raft_consensus.cc:515] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.341452 15499 ts_tablet_manager.cc:1434] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.000s
W20260709 12:29:01.337006 15422 leader_election.cc:343] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:01.341936 15507 leader_election.cc:290] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:01.348826 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14354702f32a45e2b59f53620e8f40da" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
I20260709 12:29:01.349529 15500 tablet_bootstrap.cc:654] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.349788 15385 raft_consensus.cc:2468] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 0.
I20260709 12:29:01.351699 15273 leader_election.cc:304] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:01.352800 15505 raft_consensus.cc:2804] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:01.353235 15505 raft_consensus.cc:493] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.353597 15505 raft_consensus.cc:3060] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.356755 15310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db3593e4616490a977852f03cfb8752" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:01.357424 15499 tablet_bootstrap.cc:492] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
W20260709 12:29:01.362360 15423 leader_election.cc:343] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:01.363051 15423 leader_election.cc:304] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98426102f6f642258ad743626d97034a; no voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb
I20260709 12:29:01.364586 15506 raft_consensus.cc:2749] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:01.370651 15507 raft_consensus.cc:493] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.371456 15507 raft_consensus.cc:515] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.372786 15501 tablet_bootstrap.cc:654] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.378965 15459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14354702f32a45e2b59f53620e8f40da" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:01.376989 15499 tablet_bootstrap.cc:654] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.381091 15507 leader_election.cc:290] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
W20260709 12:29:01.384902 15275 leader_election.cc:343] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98426102f6f642258ad743626d97034a (127.2.4.195:46775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:01.391957 15505 raft_consensus.cc:515] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.394246 15310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15904f9d56b744f9bf6f12d1fdd2da13" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:01.401682 15499 tablet_bootstrap.cc:492] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:01.401909 15310 raft_consensus.cc:2468] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 0.
I20260709 12:29:01.402279 15499 ts_tablet_manager.cc:1403] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.045s	user 0.009s	sys 0.011s
I20260709 12:29:01.395473 15309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14354702f32a45e2b59f53620e8f40da" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:01.396688 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3919d8a786804398a997ab35ffd8d848" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:01.398106 15459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15904f9d56b744f9bf6f12d1fdd2da13" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:01.398856 15501 tablet_bootstrap.cc:492] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:01.399525 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14354702f32a45e2b59f53620e8f40da" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:01.400035 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3919d8a786804398a997ab35ffd8d848" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:01.403432 15349 leader_election.cc:304] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:01.403892 15501 ts_tablet_manager.cc:1403] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.068s	user 0.016s	sys 0.005s
I20260709 12:29:01.404621 15507 raft_consensus.cc:2804] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:29:01.404868 15350 leader_election.cc:343] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98426102f6f642258ad743626d97034a (127.2.4.195:46775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:01.400508 15500 tablet_bootstrap.cc:492] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:01.400693 15506 raft_consensus.cc:493] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.406155 15500 ts_tablet_manager.cc:1403] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.083s	user 0.022s	sys 0.034s
I20260709 12:29:01.405117 15507 raft_consensus.cc:493] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.406594 15507 raft_consensus.cc:3060] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.406281 15506 raft_consensus.cc:515] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } } peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } }
I20260709 12:29:01.403496 15308 raft_consensus.cc:2468] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 0.
I20260709 12:29:01.407378 15501 raft_consensus.cc:359] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.408120 15501 raft_consensus.cc:385] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.408437 15501 raft_consensus.cc:740] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.409446 15501 consensus_queue.cc:260] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.409565 15500 raft_consensus.cc:359] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } } peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } }
I20260709 12:29:01.410239 15506 leader_election.cc:290] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:01.410455 15500 raft_consensus.cc:385] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.410861 15500 raft_consensus.cc:740] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.411690 15500 consensus_queue.cc:260] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } } peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } }
I20260709 12:29:01.416220 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4e50d080484b2bbf21b074105618f6" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:01.417256 15308 raft_consensus.cc:2468] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 0.
I20260709 12:29:01.419093 15501 ts_tablet_manager.cc:1434] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 12:29:01.419040 15423 leader_election.cc:304] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:01.421020 15528 raft_consensus.cc:2804] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:01.420475 15507 raft_consensus.cc:515] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.420650 15501 tablet_bootstrap.cc:492] T 14354702f32a45e2b59f53620e8f40da P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:01.422904 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4e50d080484b2bbf21b074105618f6" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
I20260709 12:29:01.423684 15385 raft_consensus.cc:2468] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 0.
I20260709 12:29:01.418715 15499 raft_consensus.cc:359] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.424569 15499 raft_consensus.cc:385] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.425066 15499 raft_consensus.cc:740] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.426061 15507 leader_election.cc:290] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:01.420598 15349 leader_election.cc:304] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:01.427757 15529 raft_consensus.cc:2804] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:01.428220 15529 raft_consensus.cc:493] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.428442 15500 ts_tablet_manager.cc:1434] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.017s
I20260709 12:29:01.428582 15529 raft_consensus.cc:3060] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:01.428691 15350 leader_election.cc:343] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98426102f6f642258ad743626d97034a (127.2.4.195:46775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:01.429836 15500 tablet_bootstrap.cc:492] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:01.429491 15499 consensus_queue.cc:260] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.433368 15501 tablet_bootstrap.cc:654] T 14354702f32a45e2b59f53620e8f40da P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.421512 15528 raft_consensus.cc:493] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.437963 15528 raft_consensus.cc:3060] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.438758 15500 tablet_bootstrap.cc:654] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:01.436535 15350 leader_election.cc:343] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98426102f6f642258ad743626d97034a (127.2.4.195:46775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:01.448240 15528 raft_consensus.cc:515] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } } peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } }
I20260709 12:29:01.449783 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15904f9d56b744f9bf6f12d1fdd2da13" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a"
W20260709 12:29:01.453311 15350 leader_election.cc:343] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98426102f6f642258ad743626d97034a (127.2.4.195:46775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:01.460124 15309 raft_consensus.cc:2393] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b0dbefdcaa7411a8877659d441f6413 in current term 1: Already voted for candidate fcf7b677a0b646948a96f66b61cc90bb in this term.
I20260709 12:29:01.460227 15499 ts_tablet_manager.cc:1434] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.058s	user 0.012s	sys 0.019s
I20260709 12:29:01.462072 15499 tablet_bootstrap.cc:492] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:01.466019 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14354702f32a45e2b59f53620e8f40da" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:01.466854 15385 raft_consensus.cc:3060] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.472225 15499 tablet_bootstrap.cc:654] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.477744 15528 leader_election.cc:290] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:01.478770 15309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4e50d080484b2bbf21b074105618f6" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:01.479560 15309 raft_consensus.cc:3060] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.480477 15529 raft_consensus.cc:515] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.483000 15529 leader_election.cc:290] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:01.485170 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4e50d080484b2bbf21b074105618f6" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:01.486011 15384 raft_consensus.cc:3060] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.486135 15501 tablet_bootstrap.cc:492] T 14354702f32a45e2b59f53620e8f40da P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:01.486771 15501 ts_tablet_manager.cc:1403] T 14354702f32a45e2b59f53620e8f40da P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.066s	user 0.009s	sys 0.005s
I20260709 12:29:01.486857 15499 tablet_bootstrap.cc:492] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:01.487864 15499 ts_tablet_manager.cc:1403] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.010s
I20260709 12:29:01.491370 15501 raft_consensus.cc:359] T 14354702f32a45e2b59f53620e8f40da P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.492427 15501 raft_consensus.cc:385] T 14354702f32a45e2b59f53620e8f40da P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.492806 15501 raft_consensus.cc:740] T 14354702f32a45e2b59f53620e8f40da P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.493307 15499 raft_consensus.cc:359] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.493860 15501 consensus_queue.cc:260] T 14354702f32a45e2b59f53620e8f40da P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.494480 15499 raft_consensus.cc:385] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.495312 15499 raft_consensus.cc:740] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.496218 15499 consensus_queue.cc:260] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.499001 15501 ts_tablet_manager.cc:1434] T 14354702f32a45e2b59f53620e8f40da P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:29:01.500232 15501 tablet_bootstrap.cc:492] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:01.501328 15384 raft_consensus.cc:2468] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 1.
I20260709 12:29:01.502944 15422 leader_election.cc:304] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, 98426102f6f642258ad743626d97034a; no voters: 
I20260709 12:29:01.506255 15505 leader_election.cc:290] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Requested vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:01.504149 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3919d8a786804398a997ab35ffd8d848" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:01.507795 15501 tablet_bootstrap.cc:654] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.507889 15458 raft_consensus.cc:3060] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.504851 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14354702f32a45e2b59f53620e8f40da" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:01.510054 15457 raft_consensus.cc:3060] T 14354702f32a45e2b59f53620e8f40da P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.506969 15528 raft_consensus.cc:2804] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:01.514104 15528 raft_consensus.cc:697] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Becoming Leader. State: Replica: 98426102f6f642258ad743626d97034a, State: Running, Role: LEADER
I20260709 12:29:01.521715 15528 consensus_queue.cc:237] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } } peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } }
I20260709 12:29:01.513441 15499 ts_tablet_manager.cc:1434] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.004s
I20260709 12:29:01.526197 15501 tablet_bootstrap.cc:492] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:01.525986 15457 raft_consensus.cc:2468] T 14354702f32a45e2b59f53620e8f40da P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 1.
I20260709 12:29:01.527268 15499 tablet_bootstrap.cc:492] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:01.532332 15501 ts_tablet_manager.cc:1403] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.032s	user 0.023s	sys 0.000s
I20260709 12:29:01.527305 15458 raft_consensus.cc:2468] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 1.
I20260709 12:29:01.533092 15275 leader_election.cc:304] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:01.534495 15350 leader_election.cc:304] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, 98426102f6f642258ad743626d97034a; no voters: 
I20260709 12:29:01.534858 15505 raft_consensus.cc:2804] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:01.535653 15505 raft_consensus.cc:697] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Becoming Leader. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:01.536195 15501 raft_consensus.cc:359] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.537180 15501 raft_consensus.cc:385] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.536610 15505 consensus_queue.cc:237] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.537532 15530 raft_consensus.cc:2804] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:01.538450 15530 raft_consensus.cc:697] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Becoming Leader. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:01.539443 15530 consensus_queue.cc:237] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.537554 15501 raft_consensus.cc:740] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.542544 15501 consensus_queue.cc:260] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
W20260709 12:29:01.512351 15349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:34773, user_credentials={real_user=slave}} blocked reactor thread for 91855us
I20260709 12:29:01.546342 15349 leader_election.cc:304] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413; no voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb
I20260709 12:29:01.546898 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15904f9d56b744f9bf6f12d1fdd2da13" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:01.547699 15308 raft_consensus.cc:3060] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.548804 15499 tablet_bootstrap.cc:654] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.550541 15310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3919d8a786804398a997ab35ffd8d848" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:01.551627 15310 raft_consensus.cc:3060] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.560139 15309 raft_consensus.cc:2468] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 1.
I20260709 12:29:01.577553 15501 ts_tablet_manager.cc:1434] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.045s	user 0.009s	sys 0.001s
I20260709 12:29:01.579320 15501 tablet_bootstrap.cc:492] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:01.580724 15528 raft_consensus.cc:493] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.583738 15528 raft_consensus.cc:515] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.590263 15528 leader_election.cc:290] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:01.584847 15385 raft_consensus.cc:2468] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 1.
I20260709 12:29:01.632948 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4c8ccf018614a79bb5fe9f2896231ea" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
W20260709 12:29:01.636490 15422 leader_election.cc:343] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:01.637169 15500 tablet_bootstrap.cc:492] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:01.639205 15500 ts_tablet_manager.cc:1403] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.210s	user 0.044s	sys 0.074s
I20260709 12:29:01.642581 15501 tablet_bootstrap.cc:654] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.643321 15309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4c8ccf018614a79bb5fe9f2896231ea" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:01.644884 15309 raft_consensus.cc:2468] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 0.
I20260709 12:29:01.648685 15529 raft_consensus.cc:2749] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:01.659332 15423 leader_election.cc:304] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 6b0dbefdcaa7411a8877659d441f6413
I20260709 12:29:01.662575 15528 raft_consensus.cc:2804] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:01.663439 15528 raft_consensus.cc:493] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.663969 15528 raft_consensus.cc:3060] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.666394 15499 tablet_bootstrap.cc:492] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:01.668367 15499 ts_tablet_manager.cc:1403] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.141s	user 0.021s	sys 0.001s
I20260709 12:29:01.668946 15500 raft_consensus.cc:359] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.670398 15500 raft_consensus.cc:385] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.670873 15500 raft_consensus.cc:740] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.672397 15500 consensus_queue.cc:260] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.676676 15499 raft_consensus.cc:359] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.678280 15499 raft_consensus.cc:385] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.678772 15499 raft_consensus.cc:740] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.680836 15308 raft_consensus.cc:2468] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 1.
I20260709 12:29:01.682070 15310 raft_consensus.cc:2468] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 1.
I20260709 12:29:01.684311 15349 leader_election.cc:304] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 98426102f6f642258ad743626d97034a
I20260709 12:29:01.681102 15206 catalog_manager.cc:5697] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "6b0dbefdcaa7411a8877659d441f6413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:01.687081 15529 raft_consensus.cc:2804] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:01.687934 15529 raft_consensus.cc:697] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Becoming Leader. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:01.684363 15208 catalog_manager.cc:5697] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a reported cstate change: term changed from 0 to 1, leader changed from <none> to 98426102f6f642258ad743626d97034a (127.2.4.195). New cstate: current_term: 1 leader_uuid: "98426102f6f642258ad743626d97034a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:01.700899 15499 consensus_queue.cc:260] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.707631 15499 ts_tablet_manager.cc:1434] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.039s	user 0.019s	sys 0.016s
I20260709 12:29:01.708361 15529 consensus_queue.cc:237] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.711880 15499 tablet_bootstrap.cc:492] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:01.712689 15500 ts_tablet_manager.cc:1434] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.073s	user 0.005s	sys 0.012s
I20260709 12:29:01.713892 15528 raft_consensus.cc:515] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.715866 15500 tablet_bootstrap.cc:492] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:01.721319 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4c8ccf018614a79bb5fe9f2896231ea" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
W20260709 12:29:01.723250 15422 leader_election.cc:343] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:01.724853 15499 tablet_bootstrap.cc:654] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.719169 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4c8ccf018614a79bb5fe9f2896231ea" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:01.726347 15308 raft_consensus.cc:3060] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.725752 15528 leader_election.cc:290] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:01.734261 15501 tablet_bootstrap.cc:492] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:01.739593 15501 ts_tablet_manager.cc:1403] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.161s	user 0.023s	sys 0.027s
I20260709 12:29:01.735642 15505 raft_consensus.cc:493] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.742558 15505 raft_consensus.cc:515] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.743499 15500 tablet_bootstrap.cc:654] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.747129 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db3593e4616490a977852f03cfb8752" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
I20260709 12:29:01.748363 15385 raft_consensus.cc:2468] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 0.
I20260709 12:29:01.748730 15529 raft_consensus.cc:493] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.749693 15529 raft_consensus.cc:515] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.742578 15308 raft_consensus.cc:2468] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 1.
I20260709 12:29:01.752228 15273 leader_election.cc:304] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:01.753213 15423 leader_election.cc:304] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 6b0dbefdcaa7411a8877659d441f6413
I20260709 12:29:01.753808 15531 raft_consensus.cc:2804] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:01.754312 15499 tablet_bootstrap.cc:492] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:01.754328 15531 raft_consensus.cc:493] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.754390 15310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e70726b51a4dabb3e47f6074f33bb4" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:01.754966 15499 ts_tablet_manager.cc:1403] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.044s	user 0.019s	sys 0.019s
I20260709 12:29:01.754892 15506 raft_consensus.cc:2804] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:01.755692 15310 raft_consensus.cc:2468] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 0.
I20260709 12:29:01.756069 15531 raft_consensus.cc:3060] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.757858 15349 leader_election.cc:304] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:01.759394 15530 raft_consensus.cc:2804] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:01.760010 15530 raft_consensus.cc:493] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.759486 15499 raft_consensus.cc:359] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.760520 15530 raft_consensus.cc:3060] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.760648 15499 raft_consensus.cc:385] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.761250 15499 raft_consensus.cc:740] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.752316 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db3593e4616490a977852f03cfb8752" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:01.764151 15458 raft_consensus.cc:2468] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 0.
I20260709 12:29:01.747282 15505 leader_election.cc:290] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:01.762562 15499 consensus_queue.cc:260] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.786899 15530 raft_consensus.cc:515] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.788203 15529 leader_election.cc:290] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:01.745792 15528 raft_consensus.cc:493] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.789577 15528 raft_consensus.cc:515] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.750761 15501 raft_consensus.cc:359] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.796358 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e70726b51a4dabb3e47f6074f33bb4" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:01.794325 15506 raft_consensus.cc:697] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Becoming Leader. State: Replica: 98426102f6f642258ad743626d97034a, State: Running, Role: LEADER
I20260709 12:29:01.798084 15501 raft_consensus.cc:385] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.799098 15501 raft_consensus.cc:740] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.800192 15501 consensus_queue.cc:260] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
W20260709 12:29:01.801854 15350 leader_election.cc:343] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98426102f6f642258ad743626d97034a (127.2.4.195:46775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:01.801111 15506 consensus_queue.cc:237] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.803682 15501 ts_tablet_manager.cc:1434] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.006s
I20260709 12:29:01.803491 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e70726b51a4dabb3e47f6074f33bb4" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:01.804715 15308 raft_consensus.cc:3060] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.805668 15501 tablet_bootstrap.cc:492] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:01.810742 15499 ts_tablet_manager.cc:1434] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.055s	user 0.018s	sys 0.011s
I20260709 12:29:01.813088 15499 tablet_bootstrap.cc:492] T 02a7fd763e6d4686aba882cfae13b560 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:01.819108 15531 raft_consensus.cc:515] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.820876 15499 tablet_bootstrap.cc:654] T 02a7fd763e6d4686aba882cfae13b560 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.821659 15528 leader_election.cc:290] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:01.823274 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15904f9d56b744f9bf6f12d1fdd2da13" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
I20260709 12:29:01.823485 15310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15904f9d56b744f9bf6f12d1fdd2da13" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:01.826786 15500 tablet_bootstrap.cc:492] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:01.828634 15530 leader_election.cc:290] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:01.831248 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e70726b51a4dabb3e47f6074f33bb4" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:01.833897 15500 ts_tablet_manager.cc:1403] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.119s	user 0.011s	sys 0.023s
I20260709 12:29:01.832551 15531 leader_election.cc:290] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Requested vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:01.843645 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db3593e4616490a977852f03cfb8752" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:01.843719 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db3593e4616490a977852f03cfb8752" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:01.846247 15205 catalog_manager.cc:5697] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193). New cstate: current_term: 1 leader_uuid: "fcf7b677a0b646948a96f66b61cc90bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:01.850879 15310 raft_consensus.cc:2393] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98426102f6f642258ad743626d97034a in current term 1: Already voted for candidate 6b0dbefdcaa7411a8877659d441f6413 in this term.
I20260709 12:29:01.854185 15457 raft_consensus.cc:3060] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.856676 15384 raft_consensus.cc:3060] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:01.866487 15350 leader_election.cc:343] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98426102f6f642258ad743626d97034a (127.2.4.195:46775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:01.866888 15423 leader_election.cc:304] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98426102f6f642258ad743626d97034a; no voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb
I20260709 12:29:01.870061 15506 raft_consensus.cc:3060] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:01.882704 15308 raft_consensus.cc:2468] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 1.
I20260709 12:29:01.884609 15349 leader_election.cc:304] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 98426102f6f642258ad743626d97034a
I20260709 12:29:01.886102 15530 raft_consensus.cc:2804] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:01.886900 15530 raft_consensus.cc:697] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Becoming Leader. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:01.888137 15530 consensus_queue.cc:237] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.888696 15500 raft_consensus.cc:359] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.889739 15500 raft_consensus.cc:385] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.890062 15500 raft_consensus.cc:740] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.868216 15208 catalog_manager.cc:5697] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "6b0dbefdcaa7411a8877659d441f6413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:01.890982 15500 consensus_queue.cc:260] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.894935 15457 raft_consensus.cc:2468] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 1.
I20260709 12:29:01.897457 15384 raft_consensus.cc:2468] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 1.
I20260709 12:29:01.900146 15500 ts_tablet_manager.cc:1434] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.048s	user 0.011s	sys 0.000s
I20260709 12:29:01.900943 15275 leader_election.cc:304] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:01.901940 15531 raft_consensus.cc:2804] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:01.902484 15531 raft_consensus.cc:697] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Becoming Leader. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:01.903529 15531 consensus_queue.cc:237] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.908041 15500 tablet_bootstrap.cc:492] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:01.926708 15528 consensus_queue.cc:1048] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:01.927380 15506 raft_consensus.cc:2749] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:01.939960 15499 tablet_bootstrap.cc:492] T 02a7fd763e6d4686aba882cfae13b560 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:01.940637 15499 ts_tablet_manager.cc:1403] T 02a7fd763e6d4686aba882cfae13b560 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.128s	user 0.026s	sys 0.009s
I20260709 12:29:01.943886 15499 raft_consensus.cc:359] T 02a7fd763e6d4686aba882cfae13b560 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.942196 15206 catalog_manager.cc:5697] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a reported cstate change: term changed from 0 to 1, leader changed from <none> to 98426102f6f642258ad743626d97034a (127.2.4.195). New cstate: current_term: 1 leader_uuid: "98426102f6f642258ad743626d97034a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:01.944569 15499 raft_consensus.cc:385] T 02a7fd763e6d4686aba882cfae13b560 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.944866 15499 raft_consensus.cc:740] T 02a7fd763e6d4686aba882cfae13b560 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.945771 15499 consensus_queue.cc:260] T 02a7fd763e6d4686aba882cfae13b560 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.947286 15500 tablet_bootstrap.cc:654] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.948083 15499 ts_tablet_manager.cc:1434] T 02a7fd763e6d4686aba882cfae13b560 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:01.948917 15205 catalog_manager.cc:5697] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193). New cstate: current_term: 1 leader_uuid: "fcf7b677a0b646948a96f66b61cc90bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:01.957291 15501 tablet_bootstrap.cc:654] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.966069 15501 tablet_bootstrap.cc:492] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:01.966835 15501 ts_tablet_manager.cc:1403] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.162s	user 0.015s	sys 0.003s
I20260709 12:29:01.968830 15500 tablet_bootstrap.cc:492] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:01.969877 15500 ts_tablet_manager.cc:1403] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.068s	user 0.015s	sys 0.005s
I20260709 12:29:01.970937 15501 raft_consensus.cc:359] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.971940 15501 raft_consensus.cc:385] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.971657 15506 consensus_queue.cc:1048] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:01.972458 15501 raft_consensus.cc:740] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.973376 15500 raft_consensus.cc:359] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.973680 15501 consensus_queue.cc:260] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.974357 15500 raft_consensus.cc:385] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:01.974681 15500 raft_consensus.cc:740] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.975677 15500 consensus_queue.cc:260] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:01.977166 15501 ts_tablet_manager.cc:1434] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:29:01.978260 15500 ts_tablet_manager.cc:1434] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:29:01.978833 15501 tablet_bootstrap.cc:492] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:01.979664 15500 tablet_bootstrap.cc:492] T 02a7fd763e6d4686aba882cfae13b560 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:01.987146 15501 tablet_bootstrap.cc:654] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:01.987146 15500 tablet_bootstrap.cc:654] T 02a7fd763e6d4686aba882cfae13b560 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:02.000679 15531 consensus_queue.cc:1048] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:01.992698 15207 catalog_manager.cc:5697] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "6b0dbefdcaa7411a8877659d441f6413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:02.056960 15500 tablet_bootstrap.cc:492] T 02a7fd763e6d4686aba882cfae13b560 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:02.057924 15500 ts_tablet_manager.cc:1403] T 02a7fd763e6d4686aba882cfae13b560 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.079s	user 0.047s	sys 0.026s
I20260709 12:29:02.057596 15529 consensus_queue.cc:1048] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:02.056960 15501 tablet_bootstrap.cc:492] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:02.060413 15501 ts_tablet_manager.cc:1403] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.082s	user 0.027s	sys 0.047s
I20260709 12:29:02.060184 15505 consensus_queue.cc:1048] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:02.061724 15500 raft_consensus.cc:359] T 02a7fd763e6d4686aba882cfae13b560 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.062605 15500 raft_consensus.cc:385] T 02a7fd763e6d4686aba882cfae13b560 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:02.062947 15500 raft_consensus.cc:740] T 02a7fd763e6d4686aba882cfae13b560 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:02.063961 15500 consensus_queue.cc:260] T 02a7fd763e6d4686aba882cfae13b560 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.066918 15500 ts_tablet_manager.cc:1434] T 02a7fd763e6d4686aba882cfae13b560 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:02.075182 15501 raft_consensus.cc:359] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.076264 15501 raft_consensus.cc:385] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:02.076571 15501 raft_consensus.cc:740] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:02.077426 15501 consensus_queue.cc:260] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.079981 15501 ts_tablet_manager.cc:1434] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.002s
I20260709 12:29:02.083446 15530 consensus_queue.cc:1048] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:02.106948 15506 raft_consensus.cc:493] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.107858 15506 raft_consensus.cc:515] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.110877 15506 leader_election.cc:290] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:02.111788 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e70726b51a4dabb3e47f6074f33bb4" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:02.112157 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81e70726b51a4dabb3e47f6074f33bb4" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
I20260709 12:29:02.112902 15308 raft_consensus.cc:2393] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 98426102f6f642258ad743626d97034a in current term 1: Already voted for candidate 6b0dbefdcaa7411a8877659d441f6413 in this term.
I20260709 12:29:02.114305 15423 leader_election.cc:304] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98426102f6f642258ad743626d97034a; no voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb
I20260709 12:29:02.115742 15506 raft_consensus.cc:3060] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:02.122370 15547 raft_consensus.cc:493] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.123006 15547 raft_consensus.cc:515] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.124874 15506 raft_consensus.cc:2749] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:02.125205 15547 leader_election.cc:290] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:02.126235 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02a7fd763e6d4686aba882cfae13b560" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:02.126477 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02a7fd763e6d4686aba882cfae13b560" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
I20260709 12:29:02.127117 15308 raft_consensus.cc:2468] T 02a7fd763e6d4686aba882cfae13b560 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 0.
I20260709 12:29:02.127282 15384 raft_consensus.cc:2468] T 02a7fd763e6d4686aba882cfae13b560 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 0.
I20260709 12:29:02.128418 15423 leader_election.cc:304] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:02.129390 15506 raft_consensus.cc:2804] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:02.129849 15506 raft_consensus.cc:493] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:02.130119 15506 raft_consensus.cc:3060] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:02.137292 15506 raft_consensus.cc:515] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.139675 15506 leader_election.cc:290] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:02.140733 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02a7fd763e6d4686aba882cfae13b560" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:02.141026 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02a7fd763e6d4686aba882cfae13b560" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:02.141424 15308 raft_consensus.cc:3060] T 02a7fd763e6d4686aba882cfae13b560 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:02.141803 15384 raft_consensus.cc:3060] T 02a7fd763e6d4686aba882cfae13b560 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:02.145689 15531 raft_consensus.cc:493] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.146324 15531 raft_consensus.cc:515] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.148602 15384 raft_consensus.cc:2468] T 02a7fd763e6d4686aba882cfae13b560 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 1.
I20260709 12:29:02.149071 15308 raft_consensus.cc:2468] T 02a7fd763e6d4686aba882cfae13b560 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 1.
I20260709 12:29:02.148692 15531 leader_election.cc:290] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:02.150188 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3540108fcddc4ca1b11dc45a9d2ea775" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:02.150029 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3540108fcddc4ca1b11dc45a9d2ea775" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
I20260709 12:29:02.150688 15423 leader_election.cc:304] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:02.150976 15457 raft_consensus.cc:2468] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 0.
I20260709 12:29:02.151412 15385 raft_consensus.cc:2468] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 0.
I20260709 12:29:02.151839 15506 raft_consensus.cc:2804] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:02.152515 15506 raft_consensus.cc:697] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Becoming Leader. State: Replica: 98426102f6f642258ad743626d97034a, State: Running, Role: LEADER
I20260709 12:29:02.152573 15275 leader_election.cc:304] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:02.153982 15531 raft_consensus.cc:2804] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:02.153527 15506 consensus_queue.cc:237] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.154433 15531 raft_consensus.cc:493] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:02.154824 15531 raft_consensus.cc:3060] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:02.163707 15531 raft_consensus.cc:515] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.165561 15531 leader_election.cc:290] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Requested vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:02.164922 15206 catalog_manager.cc:5697] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a reported cstate change: term changed from 0 to 1, leader changed from <none> to 98426102f6f642258ad743626d97034a (127.2.4.195). New cstate: current_term: 1 leader_uuid: "98426102f6f642258ad743626d97034a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:02.166734 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3540108fcddc4ca1b11dc45a9d2ea775" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:02.166867 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3540108fcddc4ca1b11dc45a9d2ea775" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:02.167523 15385 raft_consensus.cc:3060] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:02.167596 15457 raft_consensus.cc:3060] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:02.173097 15567 consensus_queue.cc:1048] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:02.176875 15457 raft_consensus.cc:2468] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 1.
I20260709 12:29:02.178462 15275 leader_election.cc:304] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:02.179562 15385 raft_consensus.cc:2468] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 1.
I20260709 12:29:02.179723 15531 raft_consensus.cc:2804] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:02.180684 15531 raft_consensus.cc:697] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Becoming Leader. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:02.181843 15531 consensus_queue.cc:237] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.194630 15567 consensus_queue.cc:1048] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:02.208729 15205 catalog_manager.cc:5697] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193). New cstate: current_term: 1 leader_uuid: "fcf7b677a0b646948a96f66b61cc90bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:02.231438 15506 consensus_queue.cc:1048] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:02.243628 15385 raft_consensus.cc:3060] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:02.253077 15506 consensus_queue.cc:1048] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:02.290644 15567 consensus_queue.cc:1048] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:02.304489 15529 consensus_queue.cc:1048] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:02.322190 15531 consensus_queue.cc:1048] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:02.339577 15531 consensus_queue.cc:1048] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:02.409415 15458 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e4e50d080484b2bbf21b074105618f6"
dest_uuid: "98426102f6f642258ad743626d97034a"
mode: GRACEFUL
new_leader_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
 from {username='slave'} at 127.0.0.1:42898
I20260709 12:29:02.410332 15458 raft_consensus.cc:606] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Received request to transfer leadership to TS fcf7b677a0b646948a96f66b61cc90bb
I20260709 12:29:02.435317 15385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15904f9d56b744f9bf6f12d1fdd2da13"
dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
mode: GRACEFUL
new_leader_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
 from {username='slave'} at 127.0.0.1:59510
I20260709 12:29:02.436076 15385 raft_consensus.cc:606] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Received request to transfer leadership to TS fcf7b677a0b646948a96f66b61cc90bb
I20260709 12:29:02.446704 15458 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4c8ccf018614a79bb5fe9f2896231ea"
dest_uuid: "98426102f6f642258ad743626d97034a"
mode: GRACEFUL
new_leader_uuid: "6b0dbefdcaa7411a8877659d441f6413"
 from {username='slave'} at 127.0.0.1:42898
I20260709 12:29:02.447296 15458 raft_consensus.cc:606] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Received request to transfer leadership to TS 6b0dbefdcaa7411a8877659d441f6413
I20260709 12:29:02.469949 15308 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3540108fcddc4ca1b11dc45a9d2ea775"
dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
mode: GRACEFUL
new_leader_uuid: "6b0dbefdcaa7411a8877659d441f6413"
 from {username='slave'} at 127.0.0.1:41790
I20260709 12:29:02.470892 15308 raft_consensus.cc:606] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Received request to transfer leadership to TS 6b0dbefdcaa7411a8877659d441f6413
I20260709 12:29:02.546878 15528 raft_consensus.cc:993] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a: : Instructing follower fcf7b677a0b646948a96f66b61cc90bb to start an election
I20260709 12:29:02.547503 15528 raft_consensus.cc:1081] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Signalling peer fcf7b677a0b646948a96f66b61cc90bb to start an election
I20260709 12:29:02.550043 15308 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e4e50d080484b2bbf21b074105618f6"
dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
 from {username='slave'} at 127.0.0.1:41770
I20260709 12:29:02.550904 15308 raft_consensus.cc:493] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:02.551329 15308 raft_consensus.cc:3060] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.558605 15308 raft_consensus.cc:515] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } } peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } }
I20260709 12:29:02.560875 15308 leader_election.cc:290] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 2 election: Requested vote from peers 98426102f6f642258ad743626d97034a (127.2.4.195:46775), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:02.561995 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4e50d080484b2bbf21b074105618f6" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:02.562398 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4e50d080484b2bbf21b074105618f6" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:02.562724 15458 raft_consensus.cc:3055] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:02.563134 15458 raft_consensus.cc:740] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Running, Role: LEADER
I20260709 12:29:02.563150 15385 raft_consensus.cc:3060] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.564107 15458 consensus_queue.cc:260] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } } peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } }
I20260709 12:29:02.565303 15458 raft_consensus.cc:3060] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.571122 15385 raft_consensus.cc:2468] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 2.
I20260709 12:29:02.572736 15458 raft_consensus.cc:2468] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 2.
I20260709 12:29:02.572777 15273 leader_election.cc:304] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:02.574149 15531 raft_consensus.cc:2804] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.574725 15531 raft_consensus.cc:697] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [term 2 LEADER]: Becoming Leader. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:02.575845 15531 consensus_queue.cc:237] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } } peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } }
I20260709 12:29:02.583536 15205 catalog_manager.cc:5697] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb reported cstate change: term changed from 1 to 2, leader changed from 98426102f6f642258ad743626d97034a (127.2.4.195) to fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193). New cstate: current_term: 2 leader_uuid: "fcf7b677a0b646948a96f66b61cc90bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:02.652873 15531 consensus_queue.cc:1048] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:02.671201 15506 consensus_queue.cc:1048] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:02.672436 15585 consensus_queue.cc:1048] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:02.681811 15506 consensus_queue.cc:1048] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:02.699987 15505 raft_consensus.cc:993] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb: : Instructing follower 6b0dbefdcaa7411a8877659d441f6413 to start an election
I20260709 12:29:02.700678 15505 raft_consensus.cc:1081] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Signalling peer 6b0dbefdcaa7411a8877659d441f6413 to start an election
I20260709 12:29:02.702885 15384 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3540108fcddc4ca1b11dc45a9d2ea775"
dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
 from {username='slave'} at 127.0.0.1:59494
I20260709 12:29:02.703663 15384 raft_consensus.cc:493] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:02.704768 15384 raft_consensus.cc:3060] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.719650 15384 raft_consensus.cc:515] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.721299 15530 raft_consensus.cc:993] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413: : Instructing follower fcf7b677a0b646948a96f66b61cc90bb to start an election
I20260709 12:29:02.721953 15530 raft_consensus.cc:1081] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Signalling peer fcf7b677a0b646948a96f66b61cc90bb to start an election
I20260709 12:29:02.722225 15384 leader_election.cc:290] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 2 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:02.723875 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3540108fcddc4ca1b11dc45a9d2ea775" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:02.724778 15310 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15904f9d56b744f9bf6f12d1fdd2da13"
dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
 from {username='slave'} at 127.0.0.1:41776
I20260709 12:29:02.724640 15547 raft_consensus.cc:993] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a: : Instructing follower 6b0dbefdcaa7411a8877659d441f6413 to start an election
I20260709 12:29:02.724679 15458 raft_consensus.cc:3060] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.725476 15310 raft_consensus.cc:493] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:02.726119 15547 raft_consensus.cc:1081] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Signalling peer 6b0dbefdcaa7411a8877659d441f6413 to start an election
I20260709 12:29:02.726630 15310 raft_consensus.cc:3060] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.729786 15384 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4c8ccf018614a79bb5fe9f2896231ea"
dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
 from {username='slave'} at 127.0.0.1:59488
I20260709 12:29:02.730643 15384 raft_consensus.cc:493] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:02.731084 15384 raft_consensus.cc:3060] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.725207 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3540108fcddc4ca1b11dc45a9d2ea775" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:02.732450 15308 raft_consensus.cc:3055] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:02.732846 15308 raft_consensus.cc:740] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:02.733843 15308 consensus_queue.cc:260] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.735508 15308 raft_consensus.cc:3060] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.736362 15458 raft_consensus.cc:2468] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 2.
I20260709 12:29:02.736588 15310 raft_consensus.cc:515] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.737923 15350 leader_election.cc:304] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, 98426102f6f642258ad743626d97034a; no voters: 
I20260709 12:29:02.738869 15529 raft_consensus.cc:2804] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.739413 15529 raft_consensus.cc:697] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 LEADER]: Becoming Leader. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:02.740305 15529 consensus_queue.cc:237] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.744275 15384 raft_consensus.cc:515] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.745076 15308 raft_consensus.cc:2468] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 2.
I20260709 12:29:02.747187 15384 leader_election.cc:290] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 2 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:02.747395 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15904f9d56b744f9bf6f12d1fdd2da13" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:02.748143 15385 raft_consensus.cc:3055] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:02.748522 15385 raft_consensus.cc:740] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:02.749751 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4c8ccf018614a79bb5fe9f2896231ea" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:02.748657 15205 catalog_manager.cc:5697] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 reported cstate change: term changed from 1 to 2, leader changed from fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193) to 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "6b0dbefdcaa7411a8877659d441f6413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:02.749660 15385 consensus_queue.cc:260] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.750517 15308 raft_consensus.cc:3060] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.751047 15310 leader_election.cc:290] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 2 election: Requested vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:02.751328 15385 raft_consensus.cc:3060] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.753418 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4c8ccf018614a79bb5fe9f2896231ea" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:02.760548 15457 raft_consensus.cc:3055] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:02.760955 15457 raft_consensus.cc:740] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Running, Role: LEADER
I20260709 12:29:02.761026 15308 raft_consensus.cc:2468] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 2.
I20260709 12:29:02.761875 15457 consensus_queue.cc:260] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.762676 15349 leader_election.cc:304] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:02.763178 15457 raft_consensus.cc:3060] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.763767 15529 raft_consensus.cc:2804] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.764335 15529 raft_consensus.cc:697] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [term 2 LEADER]: Becoming Leader. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:02.765241 15529 consensus_queue.cc:237] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.766990 15385 raft_consensus.cc:2468] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 2.
I20260709 12:29:02.767400 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15904f9d56b744f9bf6f12d1fdd2da13" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:02.768237 15458 raft_consensus.cc:3060] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.768630 15273 leader_election.cc:304] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:02.769865 15505 raft_consensus.cc:2804] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.770480 15505 raft_consensus.cc:697] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [term 2 LEADER]: Becoming Leader. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:02.771519 15505 consensus_queue.cc:237] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:02.775507 15205 catalog_manager.cc:5697] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 reported cstate change: term changed from 1 to 2, leader changed from 98426102f6f642258ad743626d97034a (127.2.4.195) to 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "6b0dbefdcaa7411a8877659d441f6413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:02.779011 15458 raft_consensus.cc:2468] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 2.
I20260709 12:29:02.781592 15207 catalog_manager.cc:5697] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb reported cstate change: term changed from 1 to 2, leader changed from 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194) to fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193). New cstate: current_term: 2 leader_uuid: "fcf7b677a0b646948a96f66b61cc90bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:02.783093 15457 raft_consensus.cc:2468] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 2.
I20260709 12:29:03.023268 15457 raft_consensus.cc:1275] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Refusing update from remote peer fcf7b677a0b646948a96f66b61cc90bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.025233 15585 consensus_queue.cc:1048] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.037995 15385 raft_consensus.cc:1275] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Refusing update from remote peer fcf7b677a0b646948a96f66b61cc90bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.039983 15531 consensus_queue.cc:1048] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.164662 15385 raft_consensus.cc:1275] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Refusing update from remote peer fcf7b677a0b646948a96f66b61cc90bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.164671 15457 raft_consensus.cc:1275] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Refusing update from remote peer 6b0dbefdcaa7411a8877659d441f6413: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.166702 15529 consensus_queue.cc:1048] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.166692 15531 consensus_queue.cc:1048] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:03.172721 15458 raft_consensus.cc:1275] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Refusing update from remote peer fcf7b677a0b646948a96f66b61cc90bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.177676 15505 consensus_queue.cc:1048] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:03.178413 15308 raft_consensus.cc:1275] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Refusing update from remote peer 6b0dbefdcaa7411a8877659d441f6413: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.180208 15530 consensus_queue.cc:1048] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.236979 15308 raft_consensus.cc:1275] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Refusing update from remote peer 6b0dbefdcaa7411a8877659d441f6413: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.238400 15529 consensus_queue.cc:1048] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.248597 15457 raft_consensus.cc:1275] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Refusing update from remote peer 6b0dbefdcaa7411a8877659d441f6413: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.250510 15530 consensus_queue.cc:1048] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:05.510679 15207 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56040:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:05.514202 15207 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:05.579350 15298 tablet_service.cc:1511] Processing CreateTablet for tablet a852c8984cff4bc689d66ac14d2b9f57 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:05.580610 15371 tablet_service.cc:1511] Processing CreateTablet for tablet a852c8984cff4bc689d66ac14d2b9f57 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:05.581115 15298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a852c8984cff4bc689d66ac14d2b9f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.582427 15371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a852c8984cff4bc689d66ac14d2b9f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.585016 15447 tablet_service.cc:1511] Processing CreateTablet for tablet a852c8984cff4bc689d66ac14d2b9f57 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:05.586686 15447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a852c8984cff4bc689d66ac14d2b9f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.588330 15368 tablet_service.cc:1511] Processing CreateTablet for tablet 45c1811b66b849c6bd946167274881d0 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:05.590529 15368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45c1811b66b849c6bd946167274881d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.592716 15294 tablet_service.cc:1511] Processing CreateTablet for tablet 28ab369d42af4360ae4adf7b9f8da0e8 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:05.594463 15294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ab369d42af4360ae4adf7b9f8da0e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.595798 15370 tablet_service.cc:1511] Processing CreateTablet for tablet 28ab369d42af4360ae4adf7b9f8da0e8 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:05.596302 15295 tablet_service.cc:1511] Processing CreateTablet for tablet d0a4695b6359499da27a9fdb36b25e89 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:05.600066 15375 tablet_service.cc:1511] Processing CreateTablet for tablet d0a4695b6359499da27a9fdb36b25e89 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:05.601450 15297 tablet_service.cc:1511] Processing CreateTablet for tablet a97f76576621466fae88aa94c348cc56 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:05.600118 15443 tablet_service.cc:1511] Processing CreateTablet for tablet 45c1811b66b849c6bd946167274881d0 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:05.603307 15443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45c1811b66b849c6bd946167274881d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.606405 15446 tablet_service.cc:1511] Processing CreateTablet for tablet d0a4695b6359499da27a9fdb36b25e89 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:05.606842 15442 tablet_service.cc:1511] Processing CreateTablet for tablet 28ab369d42af4360ae4adf7b9f8da0e8 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:05.608078 15446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0a4695b6359499da27a9fdb36b25e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.609333 15375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0a4695b6359499da27a9fdb36b25e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.610163 15300 tablet_service.cc:1511] Processing CreateTablet for tablet ddb4894e18f34fbc9df199e60e76b0a0 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:05.612167 15448 tablet_service.cc:1511] Processing CreateTablet for tablet a97f76576621466fae88aa94c348cc56 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:05.612483 15296 tablet_service.cc:1511] Processing CreateTablet for tablet 3153bae2fd24462a84ec16abfdf26ea1 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:05.589205 15299 tablet_service.cc:1511] Processing CreateTablet for tablet 45c1811b66b849c6bd946167274881d0 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:05.613888 15295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0a4695b6359499da27a9fdb36b25e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.616843 15445 tablet_service.cc:1511] Processing CreateTablet for tablet ddb4894e18f34fbc9df199e60e76b0a0 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:05.625272 15300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddb4894e18f34fbc9df199e60e76b0a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.626098 15299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45c1811b66b849c6bd946167274881d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.626451 15373 tablet_service.cc:1511] Processing CreateTablet for tablet ddb4894e18f34fbc9df199e60e76b0a0 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:05.628088 15373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddb4894e18f34fbc9df199e60e76b0a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.614420 15442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ab369d42af4360ae4adf7b9f8da0e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.632759 15445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddb4894e18f34fbc9df199e60e76b0a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.617286 15374 tablet_service.cc:1511] Processing CreateTablet for tablet a97f76576621466fae88aa94c348cc56 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:05.634025 15297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a97f76576621466fae88aa94c348cc56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.634315 15370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ab369d42af4360ae4adf7b9f8da0e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.635152 15296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3153bae2fd24462a84ec16abfdf26ea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.638098 15448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a97f76576621466fae88aa94c348cc56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.640280 15369 tablet_service.cc:1511] Processing CreateTablet for tablet 3153bae2fd24462a84ec16abfdf26ea1 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:05.641940 15369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3153bae2fd24462a84ec16abfdf26ea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.646205 15374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a97f76576621466fae88aa94c348cc56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.651991 15449 tablet_service.cc:1511] Processing CreateTablet for tablet 3153bae2fd24462a84ec16abfdf26ea1 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:05.654815 15293 tablet_service.cc:1511] Processing CreateTablet for tablet 180dc9f5fe7246d7a31c88b1f2ad6bd2 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:05.656347 15293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180dc9f5fe7246d7a31c88b1f2ad6bd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.653540 15449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3153bae2fd24462a84ec16abfdf26ea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.659960 15444 tablet_service.cc:1511] Processing CreateTablet for tablet 180dc9f5fe7246d7a31c88b1f2ad6bd2 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:05.661676 15444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180dc9f5fe7246d7a31c88b1f2ad6bd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.669343 15372 tablet_service.cc:1511] Processing CreateTablet for tablet 180dc9f5fe7246d7a31c88b1f2ad6bd2 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:05.672216 15292 tablet_service.cc:1511] Processing CreateTablet for tablet 96645810f3ef4083a07dfac9828b13f3 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:05.673676 15372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180dc9f5fe7246d7a31c88b1f2ad6bd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.674983 15441 tablet_service.cc:1511] Processing CreateTablet for tablet 96645810f3ef4083a07dfac9828b13f3 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:05.679700 15441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96645810f3ef4083a07dfac9828b13f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.682003 15292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96645810f3ef4083a07dfac9828b13f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.708479 15606 tablet_bootstrap.cc:492] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:05.720594 15606 tablet_bootstrap.cc:654] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.734748 15371 tablet_service.cc:1511] Processing CreateTablet for tablet 96645810f3ef4083a07dfac9828b13f3 (DEFAULT_TABLE table=soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:05.736523 15371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96645810f3ef4083a07dfac9828b13f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:05.745616 15604 tablet_bootstrap.cc:492] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:05.783283 15604 tablet_bootstrap.cc:654] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.871683 15605 tablet_bootstrap.cc:492] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:05.872570 15606 tablet_bootstrap.cc:492] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:05.873317 15606 ts_tablet_manager.cc:1403] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.165s	user 0.028s	sys 0.035s
I20260709 12:29:05.876955 15606 raft_consensus.cc:359] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:05.879562 15604 tablet_bootstrap.cc:492] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:05.880436 15604 ts_tablet_manager.cc:1403] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.135s	user 0.016s	sys 0.000s
I20260709 12:29:05.882970 15606 raft_consensus.cc:385] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.883427 15606 raft_consensus.cc:740] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.883955 15604 raft_consensus.cc:359] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:05.884335 15606 consensus_queue.cc:260] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:05.884867 15604 raft_consensus.cc:385] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.885329 15604 raft_consensus.cc:740] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.886204 15604 consensus_queue.cc:260] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:05.892740 15604 ts_tablet_manager.cc:1434] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:29:05.894022 15604 tablet_bootstrap.cc:492] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:05.901858 15604 tablet_bootstrap.cc:654] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.912410 15606 ts_tablet_manager.cc:1434] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.002s
I20260709 12:29:05.914702 15605 tablet_bootstrap.cc:654] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.919890 15606 tablet_bootstrap.cc:492] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:05.920027 15604 tablet_bootstrap.cc:492] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:05.921005 15604 ts_tablet_manager.cc:1403] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.009s
I20260709 12:29:05.923797 15604 raft_consensus.cc:359] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:05.924423 15604 raft_consensus.cc:385] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.924719 15604 raft_consensus.cc:740] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.925470 15604 consensus_queue.cc:260] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:05.934514 15604 ts_tablet_manager.cc:1434] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.004s
I20260709 12:29:05.935462 15604 tablet_bootstrap.cc:492] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:05.942185 15604 tablet_bootstrap.cc:654] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.943015 15606 tablet_bootstrap.cc:654] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.944312 15605 tablet_bootstrap.cc:492] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:05.945109 15605 ts_tablet_manager.cc:1403] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.074s	user 0.030s	sys 0.005s
I20260709 12:29:05.948947 15605 raft_consensus.cc:359] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:05.949836 15605 raft_consensus.cc:385] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.950137 15605 raft_consensus.cc:740] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.950995 15605 consensus_queue.cc:260] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:05.953809 15605 ts_tablet_manager.cc:1434] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.008s	user 0.011s	sys 0.000s
I20260709 12:29:05.954730 15604 tablet_bootstrap.cc:492] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:05.955335 15606 tablet_bootstrap.cc:492] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:05.955346 15605 tablet_bootstrap.cc:492] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:05.955415 15604 ts_tablet_manager.cc:1403] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.007s
I20260709 12:29:05.956310 15606 ts_tablet_manager.cc:1403] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.037s	user 0.022s	sys 0.000s
I20260709 12:29:05.960160 15606 raft_consensus.cc:359] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:05.961611 15606 raft_consensus.cc:385] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.961977 15606 raft_consensus.cc:740] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.960330 15604 raft_consensus.cc:359] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:05.962842 15606 consensus_queue.cc:260] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:05.963963 15604 raft_consensus.cc:385] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.964450 15605 tablet_bootstrap.cc:654] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.966075 15606 ts_tablet_manager.cc:1434] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:29:05.967478 15606 tablet_bootstrap.cc:492] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:05.964455 15604 raft_consensus.cc:740] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.969436 15604 consensus_queue.cc:260] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:05.972314 15604 ts_tablet_manager.cc:1434] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.004s
I20260709 12:29:05.976030 15588 raft_consensus.cc:493] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:05.976716 15588 raft_consensus.cc:515] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:05.979607 15604 tablet_bootstrap.cc:492] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:05.980135 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c1811b66b849c6bd946167274881d0" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:05.980546 15588 leader_election.cc:290] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:05.981825 15606 tablet_bootstrap.cc:654] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:05.981901 15423 leader_election.cc:343] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:05.983453 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c1811b66b849c6bd946167274881d0" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
W20260709 12:29:05.985168 15422 leader_election.cc:343] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:05.985618 15422 leader_election.cc:304] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98426102f6f642258ad743626d97034a; no voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb
I20260709 12:29:05.987125 15588 raft_consensus.cc:2749] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:05.988864 15605 tablet_bootstrap.cc:492] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:05.989010 15604 tablet_bootstrap.cc:654] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:05.989650 15605 ts_tablet_manager.cc:1403] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.005s
I20260709 12:29:05.994112 15605 raft_consensus.cc:359] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:05.995273 15605 raft_consensus.cc:385] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:05.995716 15605 raft_consensus.cc:740] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.996778 15605 consensus_queue.cc:260] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.001484 15604 tablet_bootstrap.cc:492] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:06.002444 15604 ts_tablet_manager.cc:1403] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.008s
I20260709 12:29:06.007967 15605 ts_tablet_manager.cc:1434] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.018s	user 0.013s	sys 0.004s
I20260709 12:29:06.008884 15604 raft_consensus.cc:359] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.009560 15605 tablet_bootstrap.cc:492] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:06.009804 15604 raft_consensus.cc:385] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.010298 15604 raft_consensus.cc:740] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.011365 15606 tablet_bootstrap.cc:492] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:06.011263 15604 consensus_queue.cc:260] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.012176 15606 ts_tablet_manager.cc:1403] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.045s	user 0.018s	sys 0.003s
I20260709 12:29:06.014129 15604 ts_tablet_manager.cc:1434] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.007s
I20260709 12:29:06.015587 15604 tablet_bootstrap.cc:492] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:06.016546 15606 raft_consensus.cc:359] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.017432 15606 raft_consensus.cc:385] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.017593 15605 tablet_bootstrap.cc:654] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.017815 15606 raft_consensus.cc:740] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.018918 15606 consensus_queue.cc:260] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.021401 15606 ts_tablet_manager.cc:1434] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:06.024773 15604 tablet_bootstrap.cc:654] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.025045 15606 tablet_bootstrap.cc:492] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:06.033042 15606 tablet_bootstrap.cc:654] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.038205 15605 tablet_bootstrap.cc:492] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:06.038936 15605 ts_tablet_manager.cc:1403] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.012s
I20260709 12:29:06.042588 15605 raft_consensus.cc:359] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.045535 15605 raft_consensus.cc:385] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.046136 15605 raft_consensus.cc:740] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.047379 15604 tablet_bootstrap.cc:492] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:06.048152 15604 ts_tablet_manager.cc:1403] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.019s
I20260709 12:29:06.047422 15605 consensus_queue.cc:260] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.051915 15604 raft_consensus.cc:359] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.052564 15605 ts_tablet_manager.cc:1434] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:29:06.052848 15604 raft_consensus.cc:385] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.053246 15604 raft_consensus.cc:740] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.053902 15605 tablet_bootstrap.cc:492] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:06.054118 15604 consensus_queue.cc:260] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.055199 15606 tablet_bootstrap.cc:492] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:06.058842 15606 ts_tablet_manager.cc:1403] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.034s	user 0.000s	sys 0.020s
I20260709 12:29:06.056960 15604 ts_tablet_manager.cc:1434] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:06.062253 15606 raft_consensus.cc:359] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.063257 15606 raft_consensus.cc:385] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.063685 15606 raft_consensus.cc:740] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.063783 15604 tablet_bootstrap.cc:492] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:06.064595 15606 consensus_queue.cc:260] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.067268 15606 ts_tablet_manager.cc:1434] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:06.068481 15606 tablet_bootstrap.cc:492] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:06.071856 15604 tablet_bootstrap.cc:654] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.072717 15605 tablet_bootstrap.cc:654] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.076151 15606 tablet_bootstrap.cc:654] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.086933 15604 tablet_bootstrap.cc:492] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:06.087586 15604 ts_tablet_manager.cc:1403] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.011s
I20260709 12:29:06.087425 15605 tablet_bootstrap.cc:492] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:06.089147 15605 ts_tablet_manager.cc:1403] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.013s
I20260709 12:29:06.091342 15606 tablet_bootstrap.cc:492] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:06.091998 15606 ts_tablet_manager.cc:1403] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.024s	user 0.004s	sys 0.017s
I20260709 12:29:06.091709 15604 raft_consensus.cc:359] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.092639 15604 raft_consensus.cc:385] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.092963 15604 raft_consensus.cc:740] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.093138 15605 raft_consensus.cc:359] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.094312 15605 raft_consensus.cc:385] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.093886 15604 consensus_queue.cc:260] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.094743 15605 raft_consensus.cc:740] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.095744 15605 consensus_queue.cc:260] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.096705 15604 ts_tablet_manager.cc:1434] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:29:06.097965 15604 tablet_bootstrap.cc:492] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:06.098464 15605 ts_tablet_manager.cc:1434] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 12:29:06.099751 15605 tablet_bootstrap.cc:492] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:06.106447 15604 tablet_bootstrap.cc:654] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.107532 15605 tablet_bootstrap.cc:654] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.107357 15606 raft_consensus.cc:359] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.108268 15606 raft_consensus.cc:385] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.108595 15606 raft_consensus.cc:740] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.109570 15606 consensus_queue.cc:260] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.111948 15606 ts_tablet_manager.cc:1434] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.003s
I20260709 12:29:06.113516 15606 tablet_bootstrap.cc:492] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:06.119903 15604 tablet_bootstrap.cc:492] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:06.120383 15605 tablet_bootstrap.cc:492] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:06.120671 15604 ts_tablet_manager.cc:1403] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.010s
I20260709 12:29:06.121220 15605 ts_tablet_manager.cc:1403] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.006s
I20260709 12:29:06.122211 15606 tablet_bootstrap.cc:654] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.124423 15604 raft_consensus.cc:359] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.125422 15604 raft_consensus.cc:385] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.124918 15605 raft_consensus.cc:359] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.125809 15616 raft_consensus.cc:493] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.125782 15604 raft_consensus.cc:740] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.126628 15616 raft_consensus.cc:515] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.126984 15604 consensus_queue.cc:260] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.129076 15604 ts_tablet_manager.cc:1434] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:29:06.129256 15616 leader_election.cc:290] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.132138 15605 raft_consensus.cc:385] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.132567 15605 raft_consensus.cc:740] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.130693 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a4695b6359499da27a9fdb36b25e89" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:06.131304 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a4695b6359499da27a9fdb36b25e89" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:06.134320 15458 raft_consensus.cc:2468] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 0.
I20260709 12:29:06.133745 15605 consensus_queue.cc:260] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.137408 15605 ts_tablet_manager.cc:1434] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.002s
I20260709 12:29:06.137531 15350 leader_election.cc:304] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, 98426102f6f642258ad743626d97034a; no voters: 
I20260709 12:29:06.138684 15592 raft_consensus.cc:2804] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.139118 15592 raft_consensus.cc:493] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.139011 15308 raft_consensus.cc:2468] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 0.
I20260709 12:29:06.138911 15605 tablet_bootstrap.cc:492] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:06.139788 15592 raft_consensus.cc:3060] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.142336 15606 tablet_bootstrap.cc:492] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:06.142496 15604 tablet_bootstrap.cc:492] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:06.143010 15606 ts_tablet_manager.cc:1403] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.008s
I20260709 12:29:06.146625 15606 raft_consensus.cc:359] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.147431 15606 raft_consensus.cc:385] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.147744 15606 raft_consensus.cc:740] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.148017 15605 tablet_bootstrap.cc:654] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.148887 15606 consensus_queue.cc:260] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.152009 15618 raft_consensus.cc:493] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.152578 15618 raft_consensus.cc:515] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.154054 15604 tablet_bootstrap.cc:654] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.157753 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97f76576621466fae88aa94c348cc56" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
I20260709 12:29:06.152834 15606 ts_tablet_manager.cc:1434] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
W20260709 12:29:06.162953 15422 leader_election.cc:343] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.161008 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97f76576621466fae88aa94c348cc56" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:06.164081 15308 raft_consensus.cc:2468] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 0.
I20260709 12:29:06.164614 15606 tablet_bootstrap.cc:492] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:06.165702 15423 leader_election.cc:304] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 6b0dbefdcaa7411a8877659d441f6413
I20260709 12:29:06.166875 15588 raft_consensus.cc:2804] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.167294 15588 raft_consensus.cc:493] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.167611 15588 raft_consensus.cc:3060] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.169363 15505 raft_consensus.cc:493] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.170135 15505 raft_consensus.cc:515] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.161821 15618 leader_election.cc:290] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:06.173458 15592 raft_consensus.cc:515] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.174832 15505 leader_election.cc:290] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.175235 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c1811b66b849c6bd946167274881d0" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:06.176146 15458 raft_consensus.cc:2468] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 0.
I20260709 12:29:06.176225 15592 leader_election.cc:290] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.178099 15505 raft_consensus.cc:493] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.178794 15505 raft_consensus.cc:515] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.179612 15275 leader_election.cc:304] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:06.180859 15603 raft_consensus.cc:2804] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.181246 15603 raft_consensus.cc:493] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.181550 15603 raft_consensus.cc:3060] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.182765 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180dc9f5fe7246d7a31c88b1f2ad6bd2" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
W20260709 12:29:06.185155 15273 leader_election.cc:343] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.186786 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a4695b6359499da27a9fdb36b25e89" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:06.187144 15505 leader_election.cc:290] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.187610 15458 raft_consensus.cc:3060] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.188053 15606 tablet_bootstrap.cc:654] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.190572 15605 tablet_bootstrap.cc:492] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:06.174850 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c1811b66b849c6bd946167274881d0" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
I20260709 12:29:06.191593 15605 ts_tablet_manager.cc:1403] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.053s	user 0.017s	sys 0.012s
I20260709 12:29:06.192543 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a4695b6359499da27a9fdb36b25e89" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:06.193209 15308 raft_consensus.cc:3060] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.196655 15605 raft_consensus.cc:359] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.190676 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180dc9f5fe7246d7a31c88b1f2ad6bd2" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:06.198107 15605 raft_consensus.cc:385] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.191776 15384 raft_consensus.cc:2468] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 0.
I20260709 12:29:06.198803 15605 raft_consensus.cc:740] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.200706 15308 raft_consensus.cc:2468] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 1.
I20260709 12:29:06.198618 15457 raft_consensus.cc:2468] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 0.
I20260709 12:29:06.202325 15349 leader_election.cc:304] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:06.203783 15275 leader_election.cc:304] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 6b0dbefdcaa7411a8877659d441f6413
I20260709 12:29:06.205801 15505 raft_consensus.cc:2804] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.206439 15505 raft_consensus.cc:493] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.204147 15592 raft_consensus.cc:2804] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.206933 15505 raft_consensus.cc:3060] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.207206 15592 raft_consensus.cc:697] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Becoming Leader. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:06.209216 15592 consensus_queue.cc:237] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.220290 15505 raft_consensus.cc:515] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.200110 15605 consensus_queue.cc:260] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.224110 15605 ts_tablet_manager.cc:1434] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.032s	user 0.011s	sys 0.000s
I20260709 12:29:06.227311 15605 tablet_bootstrap.cc:492] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:06.228273 15458 raft_consensus.cc:2468] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 1.
I20260709 12:29:06.229923 15606 tablet_bootstrap.cc:492] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:06.230624 15606 ts_tablet_manager.cc:1403] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.066s	user 0.008s	sys 0.010s
I20260709 12:29:06.235370 15605 tablet_bootstrap.cc:654] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.235617 15505 leader_election.cc:290] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Requested vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.236461 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180dc9f5fe7246d7a31c88b1f2ad6bd2" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:06.238441 15606 raft_consensus.cc:359] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
W20260709 12:29:06.239092 15273 leader_election.cc:343] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.239624 15606 raft_consensus.cc:385] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.240185 15606 raft_consensus.cc:740] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.241401 15606 consensus_queue.cc:260] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.245164 15606 ts_tablet_manager.cc:1434] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:29:06.246529 15606 tablet_bootstrap.cc:492] T a852c8984cff4bc689d66ac14d2b9f57 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:06.248227 15605 tablet_bootstrap.cc:492] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:06.248931 15605 ts_tablet_manager.cc:1403] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.023s	user 0.007s	sys 0.011s
I20260709 12:29:06.215348 15588 raft_consensus.cc:515] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.252806 15605 raft_consensus.cc:359] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.253609 15588 leader_election.cc:290] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:06.253799 15605 raft_consensus.cc:385] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.221277 15603 raft_consensus.cc:515] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.255533 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97f76576621466fae88aa94c348cc56" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:06.254551 15606 tablet_bootstrap.cc:654] T a852c8984cff4bc689d66ac14d2b9f57 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.256642 15604 tablet_bootstrap.cc:492] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:06.257239 15604 ts_tablet_manager.cc:1403] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.115s	user 0.023s	sys 0.023s
W20260709 12:29:06.257448 15422 leader_election.cc:343] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.258857 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97f76576621466fae88aa94c348cc56" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:06.254444 15605 raft_consensus.cc:740] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.261127 15604 raft_consensus.cc:359] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.262130 15604 raft_consensus.cc:385] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.262460 15308 raft_consensus.cc:3060] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.262564 15605 consensus_queue.cc:260] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.262463 15604 raft_consensus.cc:740] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.266273 15604 consensus_queue.cc:260] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
W20260709 12:29:06.267448 15275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:46775, user_credentials={real_user=slave}} blocked reactor thread for 60856.9us
I20260709 12:29:06.270011 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180dc9f5fe7246d7a31c88b1f2ad6bd2" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:06.270666 15457 raft_consensus.cc:3060] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.272125 15604 ts_tablet_manager.cc:1434] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:29:06.269254 15206 catalog_manager.cc:5697] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "6b0dbefdcaa7411a8877659d441f6413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.273854 15604 tablet_bootstrap.cc:492] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb: Bootstrap starting.
I20260709 12:29:06.266783 15605 ts_tablet_manager.cc:1434] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.005s
I20260709 12:29:06.281893 15605 tablet_bootstrap.cc:492] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:06.289026 15457 raft_consensus.cc:2468] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 1.
I20260709 12:29:06.293165 15605 tablet_bootstrap.cc:654] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.310173 15275 leader_election.cc:304] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 6b0dbefdcaa7411a8877659d441f6413
I20260709 12:29:06.295166 15604 tablet_bootstrap.cc:654] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.301652 15459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c1811b66b849c6bd946167274881d0" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:06.314273 15459 raft_consensus.cc:3060] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.304179 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45c1811b66b849c6bd946167274881d0" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:06.316635 15384 raft_consensus.cc:3060] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.318112 15632 raft_consensus.cc:2804] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.318921 15632 raft_consensus.cc:697] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Becoming Leader. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:06.305846 15603 leader_election.cc:290] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Requested vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.318145 15505 raft_consensus.cc:493] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.319799 15632 consensus_queue.cc:237] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.320574 15505 raft_consensus.cc:515] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.322068 15604 tablet_bootstrap.cc:492] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb: No bootstrap required, opened a new log
I20260709 12:29:06.322929 15604 ts_tablet_manager.cc:1403] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.000s
I20260709 12:29:06.324718 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a852c8984cff4bc689d66ac14d2b9f57" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
I20260709 12:29:06.325467 15385 raft_consensus.cc:2468] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 0.
I20260709 12:29:06.326006 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a852c8984cff4bc689d66ac14d2b9f57" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:06.326567 15604 raft_consensus.cc:359] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.327653 15604 raft_consensus.cc:385] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.328006 15604 raft_consensus.cc:740] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.329137 15592 raft_consensus.cc:493] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.329342 15604 consensus_queue.cc:260] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.329779 15592 raft_consensus.cc:515] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.332579 15604 ts_tablet_manager.cc:1434] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:29:06.308009 15308 raft_consensus.cc:2468] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 1.
I20260709 12:29:06.334187 15592 leader_election.cc:290] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.336097 15423 leader_election.cc:304] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 6b0dbefdcaa7411a8877659d441f6413
I20260709 12:29:06.336416 15384 raft_consensus.cc:2468] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 1.
I20260709 12:29:06.336509 15606 tablet_bootstrap.cc:492] T a852c8984cff4bc689d66ac14d2b9f57 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:06.337424 15606 ts_tablet_manager.cc:1403] T a852c8984cff4bc689d66ac14d2b9f57 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.091s	user 0.032s	sys 0.048s
I20260709 12:29:06.338003 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153bae2fd24462a84ec16abfdf26ea1" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:06.339033 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153bae2fd24462a84ec16abfdf26ea1" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:06.339421 15308 raft_consensus.cc:2468] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 0.
I20260709 12:29:06.327920 15273 leader_election.cc:304] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:06.340683 15349 leader_election.cc:304] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:06.340703 15606 raft_consensus.cc:359] T a852c8984cff4bc689d66ac14d2b9f57 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.341728 15606 raft_consensus.cc:385] T a852c8984cff4bc689d66ac14d2b9f57 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.342018 15606 raft_consensus.cc:740] T a852c8984cff4bc689d66ac14d2b9f57 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.342175 15629 raft_consensus.cc:2804] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.342689 15629 raft_consensus.cc:493] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.343168 15629 raft_consensus.cc:3060] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.342720 15606 consensus_queue.cc:260] T a852c8984cff4bc689d66ac14d2b9f57 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.345870 15606 ts_tablet_manager.cc:1434] T a852c8984cff4bc689d66ac14d2b9f57 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:29:06.346035 15273 leader_election.cc:304] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
W20260709 12:29:06.345863 15350 leader_election.cc:343] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98426102f6f642258ad743626d97034a (127.2.4.195:46775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.348066 15606 tablet_bootstrap.cc:492] T 3153bae2fd24462a84ec16abfdf26ea1 P 98426102f6f642258ad743626d97034a: Bootstrap starting.
I20260709 12:29:06.348564 15632 raft_consensus.cc:2804] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.349118 15632 raft_consensus.cc:697] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Becoming Leader. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:06.350008 15632 consensus_queue.cc:237] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
W20260709 12:29:06.353652 15275 leader_election.cc:343] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98426102f6f642258ad743626d97034a (127.2.4.195:46775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:06.322922 15505 leader_election.cc:290] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.355918 15606 tablet_bootstrap.cc:654] T 3153bae2fd24462a84ec16abfdf26ea1 P 98426102f6f642258ad743626d97034a: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.361835 15592 raft_consensus.cc:2804] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.362532 15592 raft_consensus.cc:493] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.362953 15592 raft_consensus.cc:3060] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.370177 15603 raft_consensus.cc:493] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.386224 15603 raft_consensus.cc:515] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.376266 15588 raft_consensus.cc:2804] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.390123 15588 raft_consensus.cc:697] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Becoming Leader. State: Replica: 98426102f6f642258ad743626d97034a, State: Running, Role: LEADER
I20260709 12:29:06.391358 15588 consensus_queue.cc:237] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.389487 15603 leader_election.cc:290] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.399550 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb4894e18f34fbc9df199e60e76b0a0" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
I20260709 12:29:06.400749 15384 raft_consensus.cc:2468] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 0.
I20260709 12:29:06.400902 15616 raft_consensus.cc:493] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.401651 15616 raft_consensus.cc:515] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
W20260709 12:29:06.403856 15256 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:06.404498 15616 leader_election.cc:290] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.406206 15273 leader_election.cc:304] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:06.407768 15603 raft_consensus.cc:2804] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.408632 15603 raft_consensus.cc:493] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.409262 15603 raft_consensus.cc:3060] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.407858 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb4894e18f34fbc9df199e60e76b0a0" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:06.410312 15618 raft_consensus.cc:493] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.411109 15618 raft_consensus.cc:515] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.413082 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:06.413990 15457 raft_consensus.cc:2468] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 0.
I20260709 12:29:06.414639 15458 raft_consensus.cc:2468] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 0.
I20260709 12:29:06.414762 15618 leader_election.cc:290] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:06.417371 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb4894e18f34fbc9df199e60e76b0a0" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
I20260709 12:29:06.418532 15618 raft_consensus.cc:493] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.418686 15384 raft_consensus.cc:2468] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 0.
I20260709 12:29:06.419175 15505 raft_consensus.cc:493] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.420082 15505 raft_consensus.cc:515] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.420812 15422 leader_election.cc:304] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, 98426102f6f642258ad743626d97034a; no voters: 
I20260709 12:29:06.423276 15505 leader_election.cc:290] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.419224 15618 raft_consensus.cc:515] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.426079 15605 tablet_bootstrap.cc:492] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:06.426652 15605 ts_tablet_manager.cc:1403] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.145s	user 0.018s	sys 0.000s
I20260709 12:29:06.427407 15618 leader_election.cc:290] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:06.428017 15618 raft_consensus.cc:2804] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.428884 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96645810f3ef4083a07dfac9828b13f3" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
I20260709 12:29:06.429566 15385 raft_consensus.cc:2468] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 0.
I20260709 12:29:06.429797 15605 raft_consensus.cc:359] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.430518 15605 raft_consensus.cc:385] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.428365 15618 raft_consensus.cc:493] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.430343 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413" is_pre_election: true
I20260709 12:29:06.431200 15605 raft_consensus.cc:740] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.431399 15618 raft_consensus.cc:3060] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.432044 15605 consensus_queue.cc:260] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.433034 15422 leader_election.cc:304] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, 98426102f6f642258ad743626d97034a; no voters: 
I20260709 12:29:06.434362 15605 ts_tablet_manager.cc:1434] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:06.435424 15605 tablet_bootstrap.cc:492] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413: Bootstrap starting.
I20260709 12:29:06.436313 15647 raft_consensus.cc:2804] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.436818 15647 raft_consensus.cc:493] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.437135 15647 raft_consensus.cc:3060] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.439332 15384 raft_consensus.cc:2468] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 0.
I20260709 12:29:06.441185 15273 leader_election.cc:304] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:06.426146 15350 leader_election.cc:304] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, 98426102f6f642258ad743626d97034a; no voters: 
I20260709 12:29:06.442862 15616 raft_consensus.cc:2804] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.442934 15505 raft_consensus.cc:2804] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:06.443327 15616 raft_consensus.cc:493] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.443989 15606 tablet_bootstrap.cc:492] T 3153bae2fd24462a84ec16abfdf26ea1 P 98426102f6f642258ad743626d97034a: No bootstrap required, opened a new log
I20260709 12:29:06.444245 15505 raft_consensus.cc:493] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.444929 15616 raft_consensus.cc:3060] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.446009 15606 ts_tablet_manager.cc:1403] T 3153bae2fd24462a84ec16abfdf26ea1 P 98426102f6f642258ad743626d97034a: Time spent bootstrapping tablet: real 0.098s	user 0.021s	sys 0.022s
I20260709 12:29:06.446125 15505 raft_consensus.cc:3060] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.449160 15606 raft_consensus.cc:359] T 3153bae2fd24462a84ec16abfdf26ea1 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.454188 15605 tablet_bootstrap.cc:654] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:06.454433 15606 raft_consensus.cc:385] T 3153bae2fd24462a84ec16abfdf26ea1 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.454849 15606 raft_consensus.cc:740] T 3153bae2fd24462a84ec16abfdf26ea1 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.464164 15606 consensus_queue.cc:260] T 3153bae2fd24462a84ec16abfdf26ea1 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.483886 15206 catalog_manager.cc:5697] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a reported cstate change: term changed from 0 to 1, leader changed from <none> to 98426102f6f642258ad743626d97034a (127.2.4.195). New cstate: current_term: 1 leader_uuid: "98426102f6f642258ad743626d97034a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:06.484758 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:06.486063 15457 raft_consensus.cc:2468] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 0.
I20260709 12:29:06.491101 15603 raft_consensus.cc:515] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.494694 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb4894e18f34fbc9df199e60e76b0a0" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:06.495386 15384 raft_consensus.cc:3060] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.476445 15629 raft_consensus.cc:515] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.502022 15629 leader_election.cc:290] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Requested vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.513871 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a852c8984cff4bc689d66ac14d2b9f57" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:06.519416 15606 ts_tablet_manager.cc:1434] T 3153bae2fd24462a84ec16abfdf26ea1 P 98426102f6f642258ad743626d97034a: Time spent starting tablet: real 0.073s	user 0.009s	sys 0.000s
I20260709 12:29:06.503233 15459 raft_consensus.cc:2468] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 1.
I20260709 12:29:06.532452 15384 raft_consensus.cc:2468] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 1.
I20260709 12:29:06.523967 15385 raft_consensus.cc:3060] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.533716 15603 leader_election.cc:290] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Requested vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.534322 15273 leader_election.cc:304] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:06.535513 15603 raft_consensus.cc:2804] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.535360 15592 raft_consensus.cc:515] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.536214 15603 raft_consensus.cc:697] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Becoming Leader. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:06.537046 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a852c8984cff4bc689d66ac14d2b9f57" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:06.536715 15505 raft_consensus.cc:515] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.538674 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb4894e18f34fbc9df199e60e76b0a0" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:06.544636 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb4894e18f34fbc9df199e60e76b0a0" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:06.504045 15616 raft_consensus.cc:515] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.547849 15458 raft_consensus.cc:3060] T a852c8984cff4bc689d66ac14d2b9f57 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.491003 15647 raft_consensus.cc:515] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.548537 15603 consensus_queue.cc:237] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.560940 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96645810f3ef4083a07dfac9828b13f3" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:06.561510 15310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:06.562565 15310 raft_consensus.cc:2393] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b0dbefdcaa7411a8877659d441f6413 in current term 1: Already voted for candidate fcf7b677a0b646948a96f66b61cc90bb in this term.
I20260709 12:29:06.561784 15308 raft_consensus.cc:2468] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 0.
I20260709 12:29:06.564406 15306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153bae2fd24462a84ec16abfdf26ea1" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:06.564530 15649 raft_consensus.cc:493] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.565328 15306 raft_consensus.cc:3060] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.565801 15649 raft_consensus.cc:515] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.566805 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:06.567488 15457 raft_consensus.cc:3060] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.568624 15458 raft_consensus.cc:2468] T a852c8984cff4bc689d66ac14d2b9f57 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 1.
I20260709 12:29:06.571714 15649 leader_election.cc:290] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.575417 15616 leader_election.cc:290] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.571754 15605 tablet_bootstrap.cc:492] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413: No bootstrap required, opened a new log
I20260709 12:29:06.575510 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180dc9f5fe7246d7a31c88b1f2ad6bd2" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:06.577332 15275 leader_election.cc:304] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:06.578120 15457 raft_consensus.cc:2468] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 1.
I20260709 12:29:06.577203 15206 catalog_manager.cc:5697] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193). New cstate: current_term: 1 leader_uuid: "fcf7b677a0b646948a96f66b61cc90bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.578835 15629 raft_consensus.cc:2804] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.579859 15350 leader_election.cc:304] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, 98426102f6f642258ad743626d97034a; no voters: 
I20260709 12:29:06.580766 15206 catalog_manager.cc:5697] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193). New cstate: current_term: 1 leader_uuid: "fcf7b677a0b646948a96f66b61cc90bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.581307 15616 raft_consensus.cc:2804] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.581396 15629 raft_consensus.cc:697] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Becoming Leader. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:06.582515 15616 raft_consensus.cc:697] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Becoming Leader. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:06.577266 15458 raft_consensus.cc:2393] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b0dbefdcaa7411a8877659d441f6413 in current term 1: Already voted for candidate fcf7b677a0b646948a96f66b61cc90bb in this term.
I20260709 12:29:06.581341 15383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96645810f3ef4083a07dfac9828b13f3" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:06.585245 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:06.576449 15605 ts_tablet_manager.cc:1403] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413: Time spent bootstrapping tablet: real 0.141s	user 0.014s	sys 0.022s
I20260709 12:29:06.592926 15308 raft_consensus.cc:2393] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b0dbefdcaa7411a8877659d441f6413 in current term 1: Already voted for candidate fcf7b677a0b646948a96f66b61cc90bb in this term.
I20260709 12:29:06.575971 15385 raft_consensus.cc:2468] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 1.
I20260709 12:29:06.596232 15592 leader_election.cc:290] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.584326 15307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96645810f3ef4083a07dfac9828b13f3" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:06.600199 15307 raft_consensus.cc:3060] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.595157 15383 raft_consensus.cc:3060] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.583472 15616 consensus_queue.cc:237] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.600373 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180dc9f5fe7246d7a31c88b1f2ad6bd2" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:06.597004 15618 raft_consensus.cc:515] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.604782 15456 raft_consensus.cc:2393] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf7b677a0b646948a96f66b61cc90bb in current term 1: Already voted for candidate 98426102f6f642258ad743626d97034a in this term.
I20260709 12:29:06.608011 15618 leader_election.cc:290] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:06.583418 15629 consensus_queue.cc:237] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.610869 15505 leader_election.cc:290] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Requested vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.592831 15647 leader_election.cc:290] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:06.612532 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153bae2fd24462a84ec16abfdf26ea1" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:06.617204 15457 raft_consensus.cc:3060] T 3153bae2fd24462a84ec16abfdf26ea1 P 98426102f6f642258ad743626d97034a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.617331 15592 consensus_queue.cc:1048] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:06.619633 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:06.612948 15306 raft_consensus.cc:2468] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 1.
I20260709 12:29:06.623536 15349 leader_election.cc:304] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413; no voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb
I20260709 12:29:06.624632 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:06.630198 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb4894e18f34fbc9df199e60e76b0a0" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:06.631795 15307 raft_consensus.cc:2468] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 1.
I20260709 12:29:06.629348 15592 raft_consensus.cc:3060] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.636425 15385 raft_consensus.cc:2393] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 98426102f6f642258ad743626d97034a in current term 1: Already voted for candidate fcf7b677a0b646948a96f66b61cc90bb in this term.
I20260709 12:29:06.634482 15458 raft_consensus.cc:2393] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf7b677a0b646948a96f66b61cc90bb in current term 1: Already voted for candidate 6b0dbefdcaa7411a8877659d441f6413 in this term.
I20260709 12:29:06.646297 15275 leader_election.cc:304] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf7b677a0b646948a96f66b61cc90bb; no voters: 6b0dbefdcaa7411a8877659d441f6413, 98426102f6f642258ad743626d97034a
I20260709 12:29:06.647321 15603 raft_consensus.cc:2749] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:06.649520 15383 raft_consensus.cc:2468] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 1.
I20260709 12:29:06.652923 15422 leader_election.cc:304] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, 98426102f6f642258ad743626d97034a; no voters: 
I20260709 12:29:06.654510 15647 raft_consensus.cc:2804] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.656143 15647 raft_consensus.cc:697] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Becoming Leader. State: Replica: 98426102f6f642258ad743626d97034a, State: Running, Role: LEADER
I20260709 12:29:06.660532 15647 consensus_queue.cc:237] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.663532 15457 raft_consensus.cc:2468] T 3153bae2fd24462a84ec16abfdf26ea1 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 1.
I20260709 12:29:06.668022 15350 leader_election.cc:304] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, 98426102f6f642258ad743626d97034a; no voters: 
I20260709 12:29:06.668099 15605 raft_consensus.cc:359] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.669832 15605 raft_consensus.cc:385] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:06.670675 15646 raft_consensus.cc:2804] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:06.671880 15646 raft_consensus.cc:697] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Becoming Leader. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:06.671108 15605 raft_consensus.cc:740] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.678249 15605 consensus_queue.cc:260] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.679563 15208 catalog_manager.cc:5697] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193). New cstate: current_term: 1 leader_uuid: "fcf7b677a0b646948a96f66b61cc90bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.679680 15646 consensus_queue.cc:237] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.680752 15605 ts_tablet_manager.cc:1434] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413: Time spent starting tablet: real 0.087s	user 0.002s	sys 0.004s
I20260709 12:29:06.670976 15206 catalog_manager.cc:5697] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "6b0dbefdcaa7411a8877659d441f6413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.691877 15306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb4894e18f34fbc9df199e60e76b0a0" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:06.683015 15208 catalog_manager.cc:5697] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193). New cstate: current_term: 1 leader_uuid: "fcf7b677a0b646948a96f66b61cc90bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.698020 15423 leader_election.cc:304] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98426102f6f642258ad743626d97034a; no voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb
I20260709 12:29:06.700969 15647 raft_consensus.cc:2749] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:06.711942 15592 raft_consensus.cc:2749] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:06.713737 15592 raft_consensus.cc:493] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.714347 15592 raft_consensus.cc:515] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:06.717726 15592 leader_election.cc:290] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:06.718933 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97f76576621466fae88aa94c348cc56" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98426102f6f642258ad743626d97034a" is_pre_election: true
I20260709 12:29:06.723425 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97f76576621466fae88aa94c348cc56" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb" is_pre_election: true
I20260709 12:29:06.724220 15308 raft_consensus.cc:2393] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b0dbefdcaa7411a8877659d441f6413 in current term 1: Already voted for candidate 98426102f6f642258ad743626d97034a in this term.
I20260709 12:29:06.723940 15208 catalog_manager.cc:5697] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a reported cstate change: term changed from 0 to 1, leader changed from <none> to 98426102f6f642258ad743626d97034a (127.2.4.195). New cstate: current_term: 1 leader_uuid: "98426102f6f642258ad743626d97034a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:06.726295 15207 catalog_manager.cc:5697] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "6b0dbefdcaa7411a8877659d441f6413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.728279 15349 leader_election.cc:304] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413; no voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb
I20260709 12:29:06.729766 15592 raft_consensus.cc:3060] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:06.730815 15656 consensus_queue.cc:1048] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:06.740603 15592 raft_consensus.cc:2749] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:06.784674 15629 consensus_queue.cc:1048] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:06.800040 15641 consensus_queue.cc:1048] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:06.873721 15647 consensus_queue.cc:1048] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:06.886608 15632 consensus_queue.cc:1048] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:06.888263 15641 consensus_queue.cc:1048] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:06.892131 15618 consensus_queue.cc:1048] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:06.934300 15385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8"
dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
mode: GRACEFUL
new_leader_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
 from {username='slave'} at 127.0.0.1:59510
I20260709 12:29:06.934935 15385 raft_consensus.cc:606] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Received request to transfer leadership to TS fcf7b677a0b646948a96f66b61cc90bb
I20260709 12:29:06.942189 15385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0a4695b6359499da27a9fdb36b25e89"
dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
mode: GRACEFUL
new_leader_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
 from {username='slave'} at 127.0.0.1:59510
I20260709 12:29:06.942873 15385 raft_consensus.cc:606] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Received request to transfer leadership to TS fcf7b677a0b646948a96f66b61cc90bb
I20260709 12:29:06.949869 15457 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a97f76576621466fae88aa94c348cc56"
dest_uuid: "98426102f6f642258ad743626d97034a"
mode: GRACEFUL
new_leader_uuid: "6b0dbefdcaa7411a8877659d441f6413"
 from {username='slave'} at 127.0.0.1:42898
I20260709 12:29:06.950487 15457 raft_consensus.cc:606] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Received request to transfer leadership to TS 6b0dbefdcaa7411a8877659d441f6413
I20260709 12:29:06.958127 15306 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddb4894e18f34fbc9df199e60e76b0a0"
dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
mode: GRACEFUL
new_leader_uuid: "6b0dbefdcaa7411a8877659d441f6413"
 from {username='slave'} at 127.0.0.1:41790
I20260709 12:29:06.958724 15306 raft_consensus.cc:606] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Received request to transfer leadership to TS 6b0dbefdcaa7411a8877659d441f6413
I20260709 12:29:06.969830 15306 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "180dc9f5fe7246d7a31c88b1f2ad6bd2"
dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
mode: GRACEFUL
new_leader_uuid: "6b0dbefdcaa7411a8877659d441f6413"
 from {username='slave'} at 127.0.0.1:41790
I20260709 12:29:06.970468 15306 raft_consensus.cc:606] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Received request to transfer leadership to TS 6b0dbefdcaa7411a8877659d441f6413
I20260709 12:29:07.032038 15592 consensus_queue.cc:1048] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:07.042507 15616 raft_consensus.cc:993] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413: : Instructing follower fcf7b677a0b646948a96f66b61cc90bb to start an election
I20260709 12:29:07.043013 15616 raft_consensus.cc:1081] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Signalling peer fcf7b677a0b646948a96f66b61cc90bb to start an election
I20260709 12:29:07.045588 15308 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8"
dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
 from {username='slave'} at 127.0.0.1:41776
I20260709 12:29:07.052873 15656 consensus_queue.cc:1048] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:07.053886 15308 raft_consensus.cc:493] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:07.054400 15308 raft_consensus.cc:3060] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.068953 15308 raft_consensus.cc:515] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:07.078189 15632 consensus_queue.cc:1048] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:07.088243 15656 consensus_queue.cc:1048] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:07.089038 15632 consensus_queue.cc:1048] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:07.092268 15505 raft_consensus.cc:993] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb: : Instructing follower 6b0dbefdcaa7411a8877659d441f6413 to start an election
I20260709 12:29:07.092922 15505 raft_consensus.cc:1081] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Signalling peer 6b0dbefdcaa7411a8877659d441f6413 to start an election
I20260709 12:29:07.097451 15308 leader_election.cc:290] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 2 election: Requested vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:07.099264 15383 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ddb4894e18f34fbc9df199e60e76b0a0"
dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
 from {username='slave'} at 127.0.0.1:59494
I20260709 12:29:07.100205 15383 raft_consensus.cc:493] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:07.100673 15383 raft_consensus.cc:3060] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.105083 15649 consensus_queue.cc:1048] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:07.108091 15384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:07.108788 15384 raft_consensus.cc:3055] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:07.109167 15384 raft_consensus.cc:740] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:07.110106 15384 consensus_queue.cc:260] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:07.111521 15384 raft_consensus.cc:3060] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.112774 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:07.114033 15457 raft_consensus.cc:3060] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.145488 15618 consensus_queue.cc:1048] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:07.161255 15383 raft_consensus.cc:515] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:07.215886 15457 raft_consensus.cc:2468] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 2.
I20260709 12:29:07.269137 15275 leader_election.cc:304] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:07.299654 15632 raft_consensus.cc:2804] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:07.300253 15632 raft_consensus.cc:697] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 2 LEADER]: Becoming Leader. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:07.301064 15632 consensus_queue.cc:237] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:07.368206 15384 raft_consensus.cc:2468] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 2.
I20260709 12:29:07.379788 15383 leader_election.cc:290] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 2 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:07.409638 15385 raft_consensus.cc:1240] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Rejecting Update request from peer fcf7b677a0b646948a96f66b61cc90bb for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:07.493160 15641 raft_consensus.cc:993] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb: : Instructing follower 6b0dbefdcaa7411a8877659d441f6413 to start an election
I20260709 12:29:07.494130 15641 raft_consensus.cc:1081] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Signalling peer 6b0dbefdcaa7411a8877659d441f6413 to start an election
I20260709 12:29:07.512858 15382 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "180dc9f5fe7246d7a31c88b1f2ad6bd2"
dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
 from {username='slave'} at 127.0.0.1:59494
I20260709 12:29:07.513723 15382 raft_consensus.cc:493] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:07.514207 15382 raft_consensus.cc:3060] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.520476 15618 consensus_queue.cc:1048] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:29:07.649178 15422 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:34701, user_credentials={real_user=slave}} blocked reactor thread for 196290us
I20260709 12:29:07.651695 15603 consensus_queue.cc:1048] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:29:07.660046 15423 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:34773, user_credentials={real_user=slave}} blocked reactor thread for 85447.2us
I20260709 12:29:07.672392 15459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb4894e18f34fbc9df199e60e76b0a0" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:07.673183 15459 raft_consensus.cc:3060] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.697866 15456 raft_consensus.cc:1275] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Refusing update from remote peer fcf7b677a0b646948a96f66b61cc90bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.703500 15632 consensus_queue.cc:1059] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:29:07.709774 15306 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.016s	sys 0.010s
W20260709 12:29:07.718261 15306 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.016s	sys 0.010s
I20260709 12:29:07.705374 15632 raft_consensus.cc:3055] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:07.733356 15632 raft_consensus.cc:740] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:07.721618 15382 raft_consensus.cc:515] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:07.781376 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb4894e18f34fbc9df199e60e76b0a0" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:07.795941 15459 raft_consensus.cc:2468] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 2.
I20260709 12:29:07.763687 15505 consensus_queue.cc:1048] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:29:07.808857 15618 raft_consensus.cc:993] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a: : Instructing follower 6b0dbefdcaa7411a8877659d441f6413 to start an election
I20260709 12:29:07.811020 15678 raft_consensus.cc:1081] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Signalling peer 6b0dbefdcaa7411a8877659d441f6413 to start an election
I20260709 12:29:07.822119 15592 raft_consensus.cc:993] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413: : Instructing follower fcf7b677a0b646948a96f66b61cc90bb to start an election
I20260709 12:29:07.822829 15592 raft_consensus.cc:1081] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Signalling peer fcf7b677a0b646948a96f66b61cc90bb to start an election
I20260709 12:29:07.816470 15603 consensus_queue.cc:1048] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:29:07.802198 15632 consensus_queue.cc:260] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:07.832533 15382 leader_election.cc:290] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 2 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:07.832858 15306 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0a4695b6359499da27a9fdb36b25e89"
dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
 from {username='slave'} at 127.0.0.1:41776
I20260709 12:29:07.833384 15632 raft_consensus.cc:3060] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.833757 15306 raft_consensus.cc:493] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:07.834501 15306 raft_consensus.cc:3060] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.823549 15350 leader_election.cc:304] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, 98426102f6f642258ad743626d97034a; no voters: 
I20260709 12:29:07.843336 15592 raft_consensus.cc:2804] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:07.845047 15592 raft_consensus.cc:697] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 LEADER]: Becoming Leader. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:07.846057 15592 consensus_queue.cc:237] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:07.842676 15383 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a97f76576621466fae88aa94c348cc56"
dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
 from {username='slave'} at 127.0.0.1:59488
I20260709 12:29:07.855113 15383 raft_consensus.cc:493] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:07.855544 15383 raft_consensus.cc:3060] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.861138 15459 raft_consensus.cc:1240] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Rejecting Update request from peer fcf7b677a0b646948a96f66b61cc90bb for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:07.870018 15207 catalog_manager.cc:5697] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb reported cstate change: term changed from 1 to 2, leader changed from 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194) to fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193). New cstate: current_term: 2 leader_uuid: "fcf7b677a0b646948a96f66b61cc90bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:07.871388 15384 raft_consensus.cc:1275] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Refusing update from remote peer fcf7b677a0b646948a96f66b61cc90bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:07.881240 15632 consensus_queue.cc:1048] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:07.894366 15459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180dc9f5fe7246d7a31c88b1f2ad6bd2" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:07.895416 15459 raft_consensus.cc:3060] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.895757 15307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180dc9f5fe7246d7a31c88b1f2ad6bd2" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:07.896808 15307 raft_consensus.cc:3055] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:07.897238 15307 raft_consensus.cc:740] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:07.918993 15308 raft_consensus.cc:2468] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 2.
I20260709 12:29:07.923868 15307 consensus_queue.cc:260] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:07.935401 15307 raft_consensus.cc:3060] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.933009 15306 raft_consensus.cc:515] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:07.929071 15383 raft_consensus.cc:515] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:07.941696 15385 raft_consensus.cc:1240] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Rejecting Update request from peer fcf7b677a0b646948a96f66b61cc90bb for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:07.944958 15383 leader_election.cc:290] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 2 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:07.947724 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97f76576621466fae88aa94c348cc56" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:07.949616 15308 raft_consensus.cc:3060] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.947378 15458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97f76576621466fae88aa94c348cc56" candidate_uuid: "6b0dbefdcaa7411a8877659d441f6413" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:07.957711 15308 raft_consensus.cc:2468] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 2.
I20260709 12:29:07.959571 15349 leader_election.cc:304] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:07.960753 15686 raft_consensus.cc:2804] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:07.961288 15686 raft_consensus.cc:697] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 LEADER]: Becoming Leader. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:07.962260 15686 consensus_queue.cc:237] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:07.964253 15307 raft_consensus.cc:2468] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 2.
I20260709 12:29:07.965797 15349 leader_election.cc:304] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:07.966672 15685 raft_consensus.cc:2804] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:07.967268 15685 raft_consensus.cc:697] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 LEADER]: Becoming Leader. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:07.968216 15685 consensus_queue.cc:237] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:07.976320 15459 raft_consensus.cc:2468] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 2.
I20260709 12:29:07.977178 15457 raft_consensus.cc:1240] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Rejecting Update request from peer fcf7b677a0b646948a96f66b61cc90bb for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:07.953184 15458 raft_consensus.cc:3055] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:07.977723 15458 raft_consensus.cc:740] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 98426102f6f642258ad743626d97034a, State: Running, Role: LEADER
I20260709 12:29:07.978845 15458 consensus_queue.cc:260] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:07.980265 15458 raft_consensus.cc:3060] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.990007 15306 leader_election.cc:290] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 2 election: Requested vote from peers 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701), 98426102f6f642258ad743626d97034a (127.2.4.195:46775)
I20260709 12:29:07.993892 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a4695b6359499da27a9fdb36b25e89" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "98426102f6f642258ad743626d97034a"
I20260709 12:29:07.996858 15381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a4695b6359499da27a9fdb36b25e89" candidate_uuid: "fcf7b677a0b646948a96f66b61cc90bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:07.998113 15456 raft_consensus.cc:3060] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.998442 15381 raft_consensus.cc:3055] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:07.998848 15381 raft_consensus.cc:740] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:07.999752 15381 consensus_queue.cc:260] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:08.001001 15381 raft_consensus.cc:3060] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:08.006682 15456 raft_consensus.cc:2468] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 2.
I20260709 12:29:08.008188 15275 leader_election.cc:304] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:08.009087 15632 raft_consensus.cc:2804] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:08.009557 15632 raft_consensus.cc:697] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [term 2 LEADER]: Becoming Leader. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:08.010456 15632 consensus_queue.cc:237] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:08.014256 15458 raft_consensus.cc:2468] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0dbefdcaa7411a8877659d441f6413 in term 2.
I20260709 12:29:08.020237 15381 raft_consensus.cc:2468] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf7b677a0b646948a96f66b61cc90bb in term 2.
I20260709 12:29:08.031363 15206 catalog_manager.cc:5697] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 reported cstate change: term changed from 1 to 2, leader changed from fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193) to 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "6b0dbefdcaa7411a8877659d441f6413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:08.053795 15208 catalog_manager.cc:5697] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb reported cstate change: term changed from 1 to 2, leader changed from 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194) to fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193). New cstate: current_term: 2 leader_uuid: "fcf7b677a0b646948a96f66b61cc90bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:08.066685 15206 catalog_manager.cc:5697] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 reported cstate change: term changed from 1 to 2, leader changed from fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193) to 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "6b0dbefdcaa7411a8877659d441f6413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:08.068264 15206 catalog_manager.cc:5697] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 reported cstate change: term changed from 1 to 2, leader changed from 98426102f6f642258ad743626d97034a (127.2.4.195) to 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "6b0dbefdcaa7411a8877659d441f6413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:08.231278 15306 raft_consensus.cc:1275] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Refusing update from remote peer 6b0dbefdcaa7411a8877659d441f6413: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:08.233119 15683 consensus_queue.cc:1048] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:08.247185 15456 raft_consensus.cc:1275] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Refusing update from remote peer 6b0dbefdcaa7411a8877659d441f6413: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:08.249588 15686 consensus_queue.cc:1048] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:08.275784 15239 debug-util.cc:398] Leaking SignalData structure 0x7b08001a2ca0 after lost signal to thread 2070
W20260709 12:29:08.277594 15239 debug-util.cc:398] Leaking SignalData structure 0x7b080003e4c0 after lost signal to thread 15242
W20260709 12:29:08.279433 15239 debug-util.cc:398] Leaking SignalData structure 0x7b08000faec0 after lost signal to thread 15334
W20260709 12:29:08.281067 15239 debug-util.cc:398] Leaking SignalData structure 0x7b0800375aa0 after lost signal to thread 15409
W20260709 12:29:08.282769 15239 debug-util.cc:398] Leaking SignalData structure 0x7b08003996a0 after lost signal to thread 15483
I20260709 12:29:08.360085 15456 raft_consensus.cc:1275] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Refusing update from remote peer 6b0dbefdcaa7411a8877659d441f6413: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:08.361181 15306 raft_consensus.cc:1275] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Refusing update from remote peer 6b0dbefdcaa7411a8877659d441f6413: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:08.362038 15686 consensus_queue.cc:1048] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:08.363587 15683 consensus_queue.cc:1048] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:08.379295 15306 raft_consensus.cc:1275] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Refusing update from remote peer 6b0dbefdcaa7411a8877659d441f6413: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:08.381574 15592 consensus_queue.cc:1048] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:29:08.387570 15458 raft_consensus.cc:1275] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Refusing update from remote peer 6b0dbefdcaa7411a8877659d441f6413: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:08.390513 15685 consensus_queue.cc:1048] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:08.604220 15381 raft_consensus.cc:1275] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Refusing update from remote peer fcf7b677a0b646948a96f66b61cc90bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:08.605890 15505 consensus_queue.cc:1048] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:08.634718 15458 raft_consensus.cc:1275] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Refusing update from remote peer fcf7b677a0b646948a96f66b61cc90bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:08.637229 15505 consensus_queue.cc:1048] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:09.983732 15206 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56040:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:29:10.015985 15206 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56040:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:29:10.178848  2067 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 12:29:10.229471 15422 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:34701, user_credentials={real_user=slave}} blocked reactor thread for 99150us
W20260709 12:29:10.246115 15728 log.cc:927] Time spent T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:29:10.441023 15770 tablet.cc:1724] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.457106 15765 tablet.cc:1724] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.496662 15772 tablet.cc:1724] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.470939 15766 tablet.cc:1724] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.515594 15773 tablet.cc:1724] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:10.546877 15411 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:a97f76576621466fae88aa94c348cc56)
I20260709 12:29:10.546970 15774 tablet.cc:1724] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.558246 15775 tablet.cc:1724] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:10.548645 15771 log.cc:927] Time spent T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
W20260709 12:29:10.553321 15745 log.cc:927] Time spent T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a: Append to log took a long time: real 0.258s	user 0.005s	sys 0.001s
W20260709 12:29:10.568172 15411 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:180dc9f5fe7246d7a31c88b1f2ad6bd2)
W20260709 12:29:10.571905 15411 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:180dc9f5fe7246d7a31c88b1f2ad6bd2)
I20260709 12:29:10.542402 15777 tablet.cc:1724] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.597944 15776 tablet.cc:1724] T a852c8984cff4bc689d66ac14d2b9f57 P 98426102f6f642258ad743626d97034a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.613981 15782 tablet.cc:1724] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.651690 15773 tablet.cc:1724] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.658689 15777 tablet.cc:1724] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.678239 15308 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8"
dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
mode: GRACEFUL
new_leader_uuid: "98426102f6f642258ad743626d97034a"
 from {username='slave'} at 127.0.0.1:50270
I20260709 12:29:10.678985 15308 raft_consensus.cc:606] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 2 LEADER]: Received request to transfer leadership to TS 98426102f6f642258ad743626d97034a
I20260709 12:29:10.678583 15772 tablet.cc:1724] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.656919 15776 tablet.cc:1724] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.690856 15765 tablet.cc:1724] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.649171 15770 tablet.cc:1724] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.688838 15774 tablet.cc:1724] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.702526 15766 tablet.cc:1724] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.691908 15775 tablet.cc:1724] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.734511 15782 tablet.cc:1724] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.745040 15777 tablet.cc:1724] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.751447 15775 tablet.cc:1724] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.754675 15765 tablet.cc:1724] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.771577 15765 tablet.cc:1724] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.783032 15772 tablet.cc:1724] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.797195 15776 tablet.cc:1724] T 3153bae2fd24462a84ec16abfdf26ea1 P 98426102f6f642258ad743626d97034a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.809320 15787 tablet.cc:1724] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:10.832919 15381 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a97f76576621466fae88aa94c348cc56"
dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
mode: GRACEFUL
new_leader_uuid: "98426102f6f642258ad743626d97034a"
 from {username='slave'} at 127.0.0.1:42432
I20260709 12:29:10.833773 15381 raft_consensus.cc:606] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 LEADER]: Received request to transfer leadership to TS 98426102f6f642258ad743626d97034a
I20260709 12:29:10.836755  2067 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:29:10.837683  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:10.850479 15381 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3919d8a786804398a997ab35ffd8d848"
dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
mode: GRACEFUL
new_leader_uuid: "98426102f6f642258ad743626d97034a"
 from {username='slave'} at 127.0.0.1:42432
I20260709 12:29:10.851125 15381 raft_consensus.cc:606] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Received request to transfer leadership to TS 98426102f6f642258ad743626d97034a
I20260709 12:29:10.854430 15308 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8db3593e4616490a977852f03cfb8752"
dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
mode: GRACEFUL
new_leader_uuid: "98426102f6f642258ad743626d97034a"
 from {username='slave'} at 127.0.0.1:50270
I20260709 12:29:10.855070 15308 raft_consensus.cc:606] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Received request to transfer leadership to TS 98426102f6f642258ad743626d97034a
I20260709 12:29:10.856730  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:29:10.857501  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:10.867002 15740 raft_consensus.cc:993] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb: : Instructing follower 98426102f6f642258ad743626d97034a to start an election
I20260709 12:29:10.867717 15740 raft_consensus.cc:1081] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Signalling peer 98426102f6f642258ad743626d97034a to start an election
I20260709 12:29:10.869676 15457 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8db3593e4616490a977852f03cfb8752"
dest_uuid: "98426102f6f642258ad743626d97034a"
 from {username='slave'} at 127.0.0.1:42874
I20260709 12:29:10.870230 15457 raft_consensus.cc:493] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:10.870652 15457 raft_consensus.cc:3060] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:10.872934 15206 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca: soft_deleted_table [id=b47bc59fb4794d47a0f1c47294dde955] alter complete (version 1)
I20260709 12:29:10.890933 15457 raft_consensus.cc:515] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:10.894449 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db3593e4616490a977852f03cfb8752" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:10.895107 15308 raft_consensus.cc:3055] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:10.895431 15308 raft_consensus.cc:740] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:10.896240 15308 consensus_queue.cc:260] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:10.897310 15308 raft_consensus.cc:3060] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:10.906334 15457 leader_election.cc:290] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 2 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:10.907899 15381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db3593e4616490a977852f03cfb8752" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:10.908661 15381 raft_consensus.cc:3060] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:10.910501 15308 raft_consensus.cc:2468] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 2.
I20260709 12:29:10.911923 15423 leader_election.cc:304] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:10.912755 15742 raft_consensus.cc:2804] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:10.913249 15742 raft_consensus.cc:697] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 2 LEADER]: Becoming Leader. State: Replica: 98426102f6f642258ad743626d97034a, State: Running, Role: LEADER
I20260709 12:29:10.914253 15742 consensus_queue.cc:237] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:10.919521 15381 raft_consensus.cc:2468] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 2.
I20260709 12:29:10.927386 15205 catalog_manager.cc:5697] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a reported cstate change: term changed from 1 to 2, leader changed from fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193) to 98426102f6f642258ad743626d97034a (127.2.4.195). New cstate: current_term: 2 leader_uuid: "98426102f6f642258ad743626d97034a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:10.946419 15754 raft_consensus.cc:993] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb: : Instructing follower 98426102f6f642258ad743626d97034a to start an election
I20260709 12:29:10.947016 15754 raft_consensus.cc:1081] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 2 LEADER]: Signalling peer 98426102f6f642258ad743626d97034a to start an election
I20260709 12:29:10.948976 15457 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8"
dest_uuid: "98426102f6f642258ad743626d97034a"
 from {username='slave'} at 127.0.0.1:42874
I20260709 12:29:10.949443 15457 raft_consensus.cc:493] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:10.949808 15457 raft_consensus.cc:3060] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:10.951520 15769 raft_consensus.cc:993] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413: : Instructing follower 98426102f6f642258ad743626d97034a to start an election
I20260709 12:29:10.952389 15759 raft_consensus.cc:1081] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Signalling peer 98426102f6f642258ad743626d97034a to start an election
I20260709 12:29:10.955176 15458 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3919d8a786804398a997ab35ffd8d848"
dest_uuid: "98426102f6f642258ad743626d97034a"
 from {username='slave'} at 127.0.0.1:42886
I20260709 12:29:10.955858 15458 raft_consensus.cc:493] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:10.956234 15458 raft_consensus.cc:3060] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:10.957967 15457 raft_consensus.cc:515] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:10.960071 15457 leader_election.cc:290] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 3 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:10.961730 15381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:10.961781 15308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ab369d42af4360ae4adf7b9f8da0e8" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:10.962507 15381 raft_consensus.cc:3060] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:10.962775 15308 raft_consensus.cc:3055] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:10.963215 15308 raft_consensus.cc:740] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fcf7b677a0b646948a96f66b61cc90bb, State: Running, Role: LEADER
I20260709 12:29:10.964356 15308 consensus_queue.cc:260] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:10.966132 15308 raft_consensus.cc:3060] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:10.968055 15458 raft_consensus.cc:515] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:10.971593 15381 raft_consensus.cc:2468] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 3.
I20260709 12:29:10.972147 15309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3919d8a786804398a997ab35ffd8d848" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:10.973058 15309 raft_consensus.cc:3060] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:10.973996 15422 leader_election.cc:304] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, 98426102f6f642258ad743626d97034a; no voters: 
I20260709 12:29:10.975629 15742 raft_consensus.cc:2804] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:10.976220 15742 raft_consensus.cc:697] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 3 LEADER]: Becoming Leader. State: Replica: 98426102f6f642258ad743626d97034a, State: Running, Role: LEADER
I20260709 12:29:10.977213 15742 consensus_queue.cc:237] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:10.983063 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3919d8a786804398a997ab35ffd8d848" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:10.983970 15385 raft_consensus.cc:3055] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:10.984309 15385 raft_consensus.cc:740] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:10.985159 15385 consensus_queue.cc:260] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:10.986245 15309 raft_consensus.cc:2468] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 2.
I20260709 12:29:10.986316 15385 raft_consensus.cc:3060] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:10.972419 15458 leader_election.cc:290] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 2 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:10.995045 15308 raft_consensus.cc:2468] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 3.
I20260709 12:29:10.996582 15205 catalog_manager.cc:5697] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a reported cstate change: term changed from 2 to 3, leader changed from fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193) to 98426102f6f642258ad743626d97034a (127.2.4.195). New cstate: current_term: 3 leader_uuid: "98426102f6f642258ad743626d97034a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:10.998773 15423 leader_election.cc:304] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98426102f6f642258ad743626d97034a, fcf7b677a0b646948a96f66b61cc90bb; no voters: 
I20260709 12:29:10.999830 15742 raft_consensus.cc:2804] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:11.000355 15742 raft_consensus.cc:697] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [term 2 LEADER]: Becoming Leader. State: Replica: 98426102f6f642258ad743626d97034a, State: Running, Role: LEADER
I20260709 12:29:11.001246 15742 consensus_queue.cc:237] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:11.001722 15385 raft_consensus.cc:2468] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 2.
I20260709 12:29:11.017251 15205 catalog_manager.cc:5697] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a reported cstate change: term changed from 1 to 2, leader changed from 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194) to 98426102f6f642258ad743626d97034a (127.2.4.195). New cstate: current_term: 2 leader_uuid: "98426102f6f642258ad743626d97034a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:11.026077 15769 raft_consensus.cc:993] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413: : Instructing follower 98426102f6f642258ad743626d97034a to start an election
I20260709 12:29:11.026706 15769 raft_consensus.cc:1081] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 LEADER]: Signalling peer 98426102f6f642258ad743626d97034a to start an election
I20260709 12:29:11.029458 15458 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a97f76576621466fae88aa94c348cc56"
dest_uuid: "98426102f6f642258ad743626d97034a"
 from {username='slave'} at 127.0.0.1:42886
I20260709 12:29:11.030143 15458 raft_consensus.cc:493] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:11.030586 15458 raft_consensus.cc:3060] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:11.039122 15458 raft_consensus.cc:515] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:11.041343 15458 leader_election.cc:290] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 3 election: Requested vote from peers fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773), 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701)
I20260709 12:29:11.042547 15309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97f76576621466fae88aa94c348cc56" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "fcf7b677a0b646948a96f66b61cc90bb"
I20260709 12:29:11.042631 15385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97f76576621466fae88aa94c348cc56" candidate_uuid: "98426102f6f642258ad743626d97034a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "6b0dbefdcaa7411a8877659d441f6413"
I20260709 12:29:11.043561 15309 raft_consensus.cc:3060] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:11.043648 15385 raft_consensus.cc:3055] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:11.044335 15385 raft_consensus.cc:740] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6b0dbefdcaa7411a8877659d441f6413, State: Running, Role: LEADER
I20260709 12:29:11.045234 15385 consensus_queue.cc:260] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:11.046439 15385 raft_consensus.cc:3060] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:11.052904 15385 raft_consensus.cc:2468] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 3.
I20260709 12:29:11.053815 15309 raft_consensus.cc:2468] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98426102f6f642258ad743626d97034a in term 3.
I20260709 12:29:11.054088 15422 leader_election.cc:304] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0dbefdcaa7411a8877659d441f6413, 98426102f6f642258ad743626d97034a; no voters: 
I20260709 12:29:11.055433 15742 raft_consensus.cc:2804] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:11.056071 15742 raft_consensus.cc:697] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 3 LEADER]: Becoming Leader. State: Replica: 98426102f6f642258ad743626d97034a, State: Running, Role: LEADER
I20260709 12:29:11.056967 15742 consensus_queue.cc:237] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } }
I20260709 12:29:11.067580 15205 catalog_manager.cc:5697] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a reported cstate change: term changed from 2 to 3, leader changed from 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194) to 98426102f6f642258ad743626d97034a (127.2.4.195). New cstate: current_term: 3 leader_uuid: "98426102f6f642258ad743626d97034a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98426102f6f642258ad743626d97034a" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 46775 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:11.433504 15309 raft_consensus.cc:1275] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Refusing update from remote peer 98426102f6f642258ad743626d97034a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:11.435220 15742 consensus_queue.cc:1048] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:11.452620 15385 raft_consensus.cc:1275] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Refusing update from remote peer 98426102f6f642258ad743626d97034a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:11.454794 15618 consensus_queue.cc:1048] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:11.483757 15309 raft_consensus.cc:1275] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Refusing update from remote peer 98426102f6f642258ad743626d97034a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:11.486192 15618 consensus_queue.cc:1048] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:11.502188 15385 raft_consensus.cc:1275] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Refusing update from remote peer 98426102f6f642258ad743626d97034a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:11.508010 15618 consensus_queue.cc:1048] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:11.578510 15309 raft_consensus.cc:1275] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 3 FOLLOWER]: Refusing update from remote peer 98426102f6f642258ad743626d97034a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:29:11.580433 15618 consensus_queue.cc:1048] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:11.596382 15381 raft_consensus.cc:1275] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 3 FOLLOWER]: Refusing update from remote peer 98426102f6f642258ad743626d97034a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:29:11.598196 15742 consensus_queue.cc:1048] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:11.630586 15309 raft_consensus.cc:1275] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [term 3 FOLLOWER]: Refusing update from remote peer 98426102f6f642258ad743626d97034a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:29:11.632272 15742 consensus_queue.cc:1048] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf7b677a0b646948a96f66b61cc90bb" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:29:11.642829 15381 raft_consensus.cc:1275] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 3 FOLLOWER]: Refusing update from remote peer 98426102f6f642258ad743626d97034a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:29:11.651958 15742 consensus_queue.cc:1048] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0dbefdcaa7411a8877659d441f6413" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:11.860469  2067 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:11.876029  2067 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:11.876703  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:11.888038  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:11.888758  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:12.890594  2067 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:12.901387  2067 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:12.902163  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:12.911075  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:12.911738  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:13.913087  2067 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:13.919756  2067 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:13.920466  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:13.931435  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:13.932005  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:14.933511  2067 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:14.940260  2067 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:14.940833  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:14.946471  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:14.947119  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:15.948338  2067 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:15.953800  2067 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:15.954274  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:15.959425  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:15.959887  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:16.961072  2067 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:16.966069  2067 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:16.966492  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:16.971079  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:16.971498  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:17.972589  2067 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:17.978761  2067 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:17.979291  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:17.984721  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:17.985309  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:18.986577  2067 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:18.992671  2067 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:18.993225  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:18.998309  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:18.998865  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:20.000545  2067 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:20.007121  2067 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:20.007705  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:20.013523  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:20.014062  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:21.088447  2067 tablet_server.cc:179] TabletServer@127.2.4.193:0 shutting down...
W20260709 12:29:21.100698 15349 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 8 similar messages]
W20260709 12:29:21.106444 15349 consensus_peers.cc:597] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 -> Peer fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773): Couldn't send request to peer fcf7b677a0b646948a96f66b61cc90bb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:21.151401 15423 consensus_peers.cc:597] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a -> Peer fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773): Couldn't send request to peer fcf7b677a0b646948a96f66b61cc90bb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:21.155447  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:21.156733  2067 tablet_replica.cc:333] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.158145  2067 raft_consensus.cc:2243] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.159160  2067 raft_consensus.cc:2272] T 28ab369d42af4360ae4adf7b9f8da0e8 P fcf7b677a0b646948a96f66b61cc90bb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.163475  2067 tablet_replica.cc:333] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.164467  2067 raft_consensus.cc:2243] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.165410  2067 raft_consensus.cc:2272] T d0a4695b6359499da27a9fdb36b25e89 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.168532  2067 tablet_replica.cc:333] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.169373  2067 raft_consensus.cc:2243] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.170176  2067 raft_consensus.cc:2272] T 3153bae2fd24462a84ec16abfdf26ea1 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:21.173054 15423 consensus_peers.cc:597] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a -> Peer fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773): Couldn't send request to peer fcf7b677a0b646948a96f66b61cc90bb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:21.174242  2067 tablet_replica.cc:333] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.175319  2067 raft_consensus.cc:2243] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.176137  2067 raft_consensus.cc:2272] T 96645810f3ef4083a07dfac9828b13f3 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.179293  2067 tablet_replica.cc:333] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.179913  2067 raft_consensus.cc:2243] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.180559  2067 raft_consensus.cc:2272] T a97f76576621466fae88aa94c348cc56 P fcf7b677a0b646948a96f66b61cc90bb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.183130  2067 tablet_replica.cc:333] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.183832  2067 raft_consensus.cc:2243] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.184744  2067 raft_consensus.cc:2272] T 45c1811b66b849c6bd946167274881d0 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.187433  2067 tablet_replica.cc:333] T 02a7fd763e6d4686aba882cfae13b560 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.188494  2067 raft_consensus.cc:2243] T 02a7fd763e6d4686aba882cfae13b560 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.189244  2067 raft_consensus.cc:2272] T 02a7fd763e6d4686aba882cfae13b560 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:21.192929 15349 consensus_peers.cc:597] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 -> Peer fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773): Couldn't send request to peer fcf7b677a0b646948a96f66b61cc90bb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:21.193013  2067 tablet_replica.cc:333] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.194177  2067 raft_consensus.cc:2243] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.194733  2067 raft_consensus.cc:2272] T f4c8ccf018614a79bb5fe9f2896231ea P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.197189  2067 tablet_replica.cc:333] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.197881  2067 raft_consensus.cc:2243] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.198388  2067 raft_consensus.cc:2272] T 3540108fcddc4ca1b11dc45a9d2ea775 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.200873  2067 tablet_replica.cc:333] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.201431  2067 raft_consensus.cc:2243] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.201916  2067 raft_consensus.cc:2272] T 3919d8a786804398a997ab35ffd8d848 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.205469  2067 tablet_replica.cc:333] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.206390  2067 raft_consensus.cc:2243] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.207052  2067 raft_consensus.cc:2272] T 81e70726b51a4dabb3e47f6074f33bb4 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:21.208786 15423 consensus_peers.cc:597] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a -> Peer fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773): Couldn't send request to peer fcf7b677a0b646948a96f66b61cc90bb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:21.210376  2067 tablet_replica.cc:333] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.211340  2067 raft_consensus.cc:2243] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.212453  2067 raft_consensus.cc:2272] T 15904f9d56b744f9bf6f12d1fdd2da13 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.214980  2067 tablet_replica.cc:333] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.215551  2067 raft_consensus.cc:2243] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.216187  2067 raft_consensus.cc:2272] T 8db3593e4616490a977852f03cfb8752 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.218729  2067 tablet_replica.cc:333] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.219390  2067 raft_consensus.cc:2243] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.220000  2067 raft_consensus.cc:2272] T ddb4894e18f34fbc9df199e60e76b0a0 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.222515  2067 tablet_replica.cc:333] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.223065  2067 raft_consensus.cc:2243] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.224020  2067 raft_consensus.cc:2272] T a852c8984cff4bc689d66ac14d2b9f57 P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.226729  2067 tablet_replica.cc:333] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.227624  2067 raft_consensus.cc:2243] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.228820  2067 raft_consensus.cc:2272] T 4e4e50d080484b2bbf21b074105618f6 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.232165  2067 tablet_replica.cc:333] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.232733  2067 raft_consensus.cc:2243] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.233364  2067 raft_consensus.cc:2272] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P fcf7b677a0b646948a96f66b61cc90bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.235821  2067 tablet_replica.cc:333] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb: stopping tablet replica
I20260709 12:29:21.236377  2067 raft_consensus.cc:2243] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.237350  2067 raft_consensus.cc:2272] T 14354702f32a45e2b59f53620e8f40da P fcf7b677a0b646948a96f66b61cc90bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:21.256523 15349 consensus_peers.cc:597] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 -> Peer fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773): Couldn't send request to peer fcf7b677a0b646948a96f66b61cc90bb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:21.289321  2067 tablet_server.cc:196] TabletServer@127.2.4.193:0 shutdown complete.
I20260709 12:29:21.336086  2067 tablet_server.cc:179] TabletServer@127.2.4.194:0 shutting down...
W20260709 12:29:21.347209 15422 consensus_peers.cc:597] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a -> Peer 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701): Couldn't send request to peer 6b0dbefdcaa7411a8877659d441f6413. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:21.360493 15422 consensus_peers.cc:597] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a -> Peer 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701): Couldn't send request to peer 6b0dbefdcaa7411a8877659d441f6413. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:21.380504  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:21.381755  2067 tablet_replica.cc:333] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.382690  2067 raft_consensus.cc:2243] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.383955  2067 raft_consensus.cc:2272] T 3153bae2fd24462a84ec16abfdf26ea1 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.388082  2067 tablet_replica.cc:333] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.389220  2067 raft_consensus.cc:2243] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:29:21.390319 15423 consensus_peers.cc:597] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a -> Peer fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773): Couldn't send request to peer fcf7b677a0b646948a96f66b61cc90bb. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:34773: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:21.390645  2067 raft_consensus.cc:2272] T ddb4894e18f34fbc9df199e60e76b0a0 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.393778  2067 tablet_replica.cc:333] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.394594  2067 raft_consensus.cc:2243] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.395090  2067 raft_consensus.cc:2272] T a97f76576621466fae88aa94c348cc56 P 6b0dbefdcaa7411a8877659d441f6413 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.398088  2067 tablet_replica.cc:333] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.399082  2067 raft_consensus.cc:2243] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.399690  2067 raft_consensus.cc:2272] T 96645810f3ef4083a07dfac9828b13f3 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.403347  2067 tablet_replica.cc:333] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.404147  2067 raft_consensus.cc:2243] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.404953  2067 raft_consensus.cc:2272] T a852c8984cff4bc689d66ac14d2b9f57 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.408754  2067 tablet_replica.cc:333] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
W20260709 12:29:21.409689 15349 consensus_peers.cc:597] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 -> Peer fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773): Couldn't send request to peer fcf7b677a0b646948a96f66b61cc90bb. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:34773: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:21.409888  2067 raft_consensus.cc:2243] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.410812  2067 raft_consensus.cc:2272] T d0a4695b6359499da27a9fdb36b25e89 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.414157  2067 tablet_replica.cc:333] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.414844  2067 raft_consensus.cc:2243] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.415935  2067 raft_consensus.cc:2272] T 3540108fcddc4ca1b11dc45a9d2ea775 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.418784  2067 tablet_replica.cc:333] T 02a7fd763e6d4686aba882cfae13b560 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.419443  2067 raft_consensus.cc:2243] T 02a7fd763e6d4686aba882cfae13b560 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.420066  2067 raft_consensus.cc:2272] T 02a7fd763e6d4686aba882cfae13b560 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.422884  2067 tablet_replica.cc:333] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.423614  2067 raft_consensus.cc:2243] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.424196  2067 raft_consensus.cc:2272] T 45c1811b66b849c6bd946167274881d0 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.426949  2067 tablet_replica.cc:333] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.427618  2067 raft_consensus.cc:2243] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.428647  2067 raft_consensus.cc:2272] T f4c8ccf018614a79bb5fe9f2896231ea P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.431257  2067 tablet_replica.cc:333] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.431922  2067 raft_consensus.cc:2243] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.432516  2067 raft_consensus.cc:2272] T 8db3593e4616490a977852f03cfb8752 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.435199  2067 tablet_replica.cc:333] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.435853  2067 raft_consensus.cc:2243] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.436885  2067 raft_consensus.cc:2272] T 81e70726b51a4dabb3e47f6074f33bb4 P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.439642  2067 tablet_replica.cc:333] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.440302  2067 raft_consensus.cc:2243] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.440850  2067 raft_consensus.cc:2272] T 3919d8a786804398a997ab35ffd8d848 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.443542  2067 tablet_replica.cc:333] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.444181  2067 raft_consensus.cc:2243] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.444742  2067 raft_consensus.cc:2272] T 15904f9d56b744f9bf6f12d1fdd2da13 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.447391  2067 tablet_replica.cc:333] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.448000  2067 raft_consensus.cc:2243] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.448585  2067 raft_consensus.cc:2272] T 28ab369d42af4360ae4adf7b9f8da0e8 P 6b0dbefdcaa7411a8877659d441f6413 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.451221  2067 tablet_replica.cc:333] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.451825  2067 raft_consensus.cc:2243] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.452418  2067 raft_consensus.cc:2272] T 4e4e50d080484b2bbf21b074105618f6 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.454874  2067 tablet_replica.cc:333] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.455442  2067 raft_consensus.cc:2243] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.456408  2067 raft_consensus.cc:2272] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 6b0dbefdcaa7411a8877659d441f6413 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.458905  2067 tablet_replica.cc:333] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413: stopping tablet replica
I20260709 12:29:21.459438  2067 raft_consensus.cc:2243] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.459962  2067 raft_consensus.cc:2272] T 14354702f32a45e2b59f53620e8f40da P 6b0dbefdcaa7411a8877659d441f6413 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:21.500177 15422 consensus_peers.cc:597] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a -> Peer 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701): Couldn't send request to peer 6b0dbefdcaa7411a8877659d441f6413. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:34701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:21.501170 15423 consensus_peers.cc:597] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a -> Peer fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773): Couldn't send request to peer fcf7b677a0b646948a96f66b61cc90bb. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:34773: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:21.504164 15423 consensus_peers.cc:597] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a -> Peer fcf7b677a0b646948a96f66b61cc90bb (127.2.4.193:34773): Couldn't send request to peer fcf7b677a0b646948a96f66b61cc90bb. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:34773: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:21.513228  2067 tablet_server.cc:196] TabletServer@127.2.4.194:0 shutdown complete.
I20260709 12:29:21.555569  2067 tablet_server.cc:179] TabletServer@127.2.4.195:0 shutting down...
I20260709 12:29:21.601501  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:21.602586  2067 tablet_replica.cc:333] T 3153bae2fd24462a84ec16abfdf26ea1 P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.604000  2067 raft_consensus.cc:2243] T 3153bae2fd24462a84ec16abfdf26ea1 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.604736  2067 raft_consensus.cc:2272] T 3153bae2fd24462a84ec16abfdf26ea1 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.607419  2067 tablet_replica.cc:333] T a852c8984cff4bc689d66ac14d2b9f57 P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.608131  2067 raft_consensus.cc:2243] T a852c8984cff4bc689d66ac14d2b9f57 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.608649  2067 raft_consensus.cc:2272] T a852c8984cff4bc689d66ac14d2b9f57 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.611182  2067 tablet_replica.cc:333] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.611871  2067 raft_consensus.cc:2243] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.612828  2067 raft_consensus.cc:2272] T 28ab369d42af4360ae4adf7b9f8da0e8 P 98426102f6f642258ad743626d97034a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.615491  2067 tablet_replica.cc:333] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.616179  2067 raft_consensus.cc:2243] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.616734  2067 raft_consensus.cc:2272] T d0a4695b6359499da27a9fdb36b25e89 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.619613  2067 tablet_replica.cc:333] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.620527  2067 raft_consensus.cc:2243] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.621232  2067 raft_consensus.cc:2272] T ddb4894e18f34fbc9df199e60e76b0a0 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:21.624634 15422 consensus_peers.cc:597] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a -> Peer 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701): Couldn't send request to peer 6b0dbefdcaa7411a8877659d441f6413. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:34701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:21.624716  2067 tablet_replica.cc:333] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a: stopping tablet replica
W20260709 12:29:21.625289 15422 consensus_peers.cc:597] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a -> Peer 6b0dbefdcaa7411a8877659d441f6413 (127.2.4.194:34701): Couldn't send request to peer 6b0dbefdcaa7411a8877659d441f6413. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:34701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:21.625944  2067 raft_consensus.cc:2243] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.626948  2067 raft_consensus.cc:2272] T 96645810f3ef4083a07dfac9828b13f3 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.629482  2067 tablet_replica.cc:333] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.630206  2067 raft_consensus.cc:2243] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.631129  2067 raft_consensus.cc:2272] T a97f76576621466fae88aa94c348cc56 P 98426102f6f642258ad743626d97034a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.633406  2067 tablet_replica.cc:333] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.634048  2067 raft_consensus.cc:2243] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.634580  2067 raft_consensus.cc:2272] T f4c8ccf018614a79bb5fe9f2896231ea P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.636904  2067 tablet_replica.cc:333] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.637516  2067 raft_consensus.cc:2243] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.638547  2067 raft_consensus.cc:2272] T 02a7fd763e6d4686aba882cfae13b560 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.640920  2067 tablet_replica.cc:333] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.641527  2067 raft_consensus.cc:2243] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.642136  2067 raft_consensus.cc:2272] T 45c1811b66b849c6bd946167274881d0 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.644292  2067 tablet_replica.cc:333] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.644977  2067 raft_consensus.cc:2243] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.645462  2067 raft_consensus.cc:2272] T 3540108fcddc4ca1b11dc45a9d2ea775 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.647703  2067 tablet_replica.cc:333] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.648276  2067 raft_consensus.cc:2243] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.649165  2067 raft_consensus.cc:2272] T 3919d8a786804398a997ab35ffd8d848 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.651506  2067 tablet_replica.cc:333] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.652105  2067 raft_consensus.cc:2243] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.652570  2067 raft_consensus.cc:2272] T 81e70726b51a4dabb3e47f6074f33bb4 P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.655293  2067 tablet_replica.cc:333] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.656008  2067 raft_consensus.cc:2243] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.656563  2067 raft_consensus.cc:2272] T 15904f9d56b744f9bf6f12d1fdd2da13 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.659152  2067 tablet_replica.cc:333] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.660302  2067 raft_consensus.cc:2243] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.661953  2067 raft_consensus.cc:2272] T 8db3593e4616490a977852f03cfb8752 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.665478  2067 tablet_replica.cc:333] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.666152  2067 raft_consensus.cc:2243] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.666745  2067 raft_consensus.cc:2272] T 180dc9f5fe7246d7a31c88b1f2ad6bd2 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.668907  2067 tablet_replica.cc:333] T 14354702f32a45e2b59f53620e8f40da P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.669461  2067 raft_consensus.cc:2243] T 14354702f32a45e2b59f53620e8f40da P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.669957  2067 raft_consensus.cc:2272] T 14354702f32a45e2b59f53620e8f40da P 98426102f6f642258ad743626d97034a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.672129  2067 tablet_replica.cc:333] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a: stopping tablet replica
I20260709 12:29:21.672672  2067 raft_consensus.cc:2243] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.673177  2067 raft_consensus.cc:2272] T 4e4e50d080484b2bbf21b074105618f6 P 98426102f6f642258ad743626d97034a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.720333  2067 tablet_server.cc:196] TabletServer@127.2.4.195:0 shutdown complete.
I20260709 12:29:21.768024  2067 master.cc:562] Master@127.2.4.254:42861 shutting down...
I20260709 12:29:21.831844  2067 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:21.833173  2067 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.833595  2067 tablet_replica.cc:333] T 00000000000000000000000000000000 P 244a03776aa247b2b722430191ed6aca: stopping tablet replica
I20260709 12:29:21.891064  2067 master.cc:584] Master@127.2.4.254:42861 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (21817 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:29:21.978014  2067 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.4.254:37151
I20260709 12:29:21.979264  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:21.987089 15831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:21.987192 15832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:21.987488 15834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:21.989049  2067 server_base.cc:1061] running on GCE node
I20260709 12:29:21.990345  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:21.990536  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:21.990753  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600161990717 us; error 0 us; skew 500 ppm
I20260709 12:29:21.991375  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:21.994601  2067 webserver.cc:533] Webserver started at http://127.2.4.254:42419/ using document root <none> and password file <none>
I20260709 12:29:21.995155  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:21.995360  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:21.995697  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:21.997259  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/master-0-root/instance:
uuid: "ef17addbd962435aa4431544d3522350"
format_stamp: "Formatted at 2026-07-09 12:29:21 on dist-test-slave-4qvp"
I20260709 12:29:22.003046  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.003s
I20260709 12:29:22.007529 15839 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:22.008639  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:29:22.009079  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/master-0-root
uuid: "ef17addbd962435aa4431544d3522350"
format_stamp: "Formatted at 2026-07-09 12:29:21 on dist-test-slave-4qvp"
I20260709 12:29:22.009406  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:22.028862  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:22.030536  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:22.077450  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.254:37151
I20260709 12:29:22.077703 15890 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.254:37151 every 8 connection(s)
I20260709 12:29:22.082245 15891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.101365 15891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350: Bootstrap starting.
I20260709 12:29:22.107547 15891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:22.113242 15891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350: No bootstrap required, opened a new log
I20260709 12:29:22.116698 15891 raft_consensus.cc:359] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef17addbd962435aa4431544d3522350" member_type: VOTER }
I20260709 12:29:22.117208 15891 raft_consensus.cc:385] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:22.117486 15891 raft_consensus.cc:740] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef17addbd962435aa4431544d3522350, State: Initialized, Role: FOLLOWER
I20260709 12:29:22.118284 15891 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef17addbd962435aa4431544d3522350" member_type: VOTER }
I20260709 12:29:22.118845 15891 raft_consensus.cc:399] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:29:22.119073 15891 raft_consensus.cc:493] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:29:22.119390 15891 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:22.126214 15891 raft_consensus.cc:515] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef17addbd962435aa4431544d3522350" member_type: VOTER }
I20260709 12:29:22.126902 15891 leader_election.cc:304] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef17addbd962435aa4431544d3522350; no voters: 
I20260709 12:29:22.128332 15891 leader_election.cc:290] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:29:22.128859 15894 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:22.130550 15894 raft_consensus.cc:697] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [term 1 LEADER]: Becoming Leader. State: Replica: ef17addbd962435aa4431544d3522350, State: Running, Role: LEADER
I20260709 12:29:22.131356 15894 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef17addbd962435aa4431544d3522350" member_type: VOTER }
I20260709 12:29:22.131916 15891 sys_catalog.cc:565] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:29:22.138764 15896 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef17addbd962435aa4431544d3522350. Latest consensus state: current_term: 1 leader_uuid: "ef17addbd962435aa4431544d3522350" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef17addbd962435aa4431544d3522350" member_type: VOTER } }
I20260709 12:29:22.139561 15896 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:22.139392 15895 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef17addbd962435aa4431544d3522350" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef17addbd962435aa4431544d3522350" member_type: VOTER } }
I20260709 12:29:22.140254 15895 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:22.142264 15902 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:29:22.149430 15902 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:29:22.150614  2067 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:29:22.160676 15902 catalog_manager.cc:1383] Generated new cluster ID: 27e67605972d4b8f8274f563faa1d02d
I20260709 12:29:22.161010 15902 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:29:22.178987 15902 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:29:22.180727 15902 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:29:22.200507 15902 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350: Generated new TSK 0
I20260709 12:29:22.201200 15902 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:29:22.218043  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:22.225878 15912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:22.227959 15913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:22.230242 15915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:22.230289  2067 server_base.cc:1061] running on GCE node
I20260709 12:29:22.231772  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:22.231981  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:22.232231  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600162232208 us; error 0 us; skew 500 ppm
I20260709 12:29:22.233217  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:22.235880  2067 webserver.cc:533] Webserver started at http://127.2.4.193:35047/ using document root <none> and password file <none>
I20260709 12:29:22.236436  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:22.236686  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:22.236938  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:22.238402  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/ts-0-root/instance:
uuid: "f742bdd975884e759c43e1311e22ac23"
format_stamp: "Formatted at 2026-07-09 12:29:22 on dist-test-slave-4qvp"
I20260709 12:29:22.243564  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.002s
I20260709 12:29:22.247848 15920 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:22.249035  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:29:22.249493  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/ts-0-root
uuid: "f742bdd975884e759c43e1311e22ac23"
format_stamp: "Formatted at 2026-07-09 12:29:22 on dist-test-slave-4qvp"
I20260709 12:29:22.249953  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:22.269101  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:22.270569  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:22.272418  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:22.275574  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:22.275832  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:22.276105  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:22.276298  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:22.330312  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.193:45651
I20260709 12:29:22.330482 15982 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.193:45651 every 8 connection(s)
I20260709 12:29:22.336076  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:22.345235 15987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:22.348241 15988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:22.351933 15990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:22.353214 15983 heartbeater.cc:344] Connected to a master server at 127.2.4.254:37151
I20260709 12:29:22.353258  2067 server_base.cc:1061] running on GCE node
I20260709 12:29:22.353960 15983 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:22.355073 15983 heartbeater.cc:507] Master 127.2.4.254:37151 requested a full tablet report, sending...
I20260709 12:29:22.355163  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:22.355579  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:22.355767  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600162355749 us; error 0 us; skew 500 ppm
I20260709 12:29:22.356460  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:22.358477 15856 ts_manager.cc:194] Registered new tserver with Master: f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651)
I20260709 12:29:22.360312  2067 webserver.cc:533] Webserver started at http://127.2.4.194:44195/ using document root <none> and password file <none>
I20260709 12:29:22.361132  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:22.361370  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:22.361513 15856 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36358
I20260709 12:29:22.361758  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:22.363497  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/ts-1-root/instance:
uuid: "f8085f7497804cc096027b14b64f95b7"
format_stamp: "Formatted at 2026-07-09 12:29:22 on dist-test-slave-4qvp"
I20260709 12:29:22.368822  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 12:29:22.373795 15995 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:22.374898  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:29:22.375301  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/ts-1-root
uuid: "f8085f7497804cc096027b14b64f95b7"
format_stamp: "Formatted at 2026-07-09 12:29:22 on dist-test-slave-4qvp"
I20260709 12:29:22.375597  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:22.403455  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:22.404837  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:22.406713  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:22.409340  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:22.409596  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:22.409922  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:22.410099  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:22.460451  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.194:33263
I20260709 12:29:22.460628 16057 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.194:33263 every 8 connection(s)
I20260709 12:29:22.466166  2067 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:22.476699 16061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:22.480275 16062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:22.484535 16058 heartbeater.cc:344] Connected to a master server at 127.2.4.254:37151
I20260709 12:29:22.485143 16058 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:22.486102 16058 heartbeater.cc:507] Master 127.2.4.254:37151 requested a full tablet report, sending...
I20260709 12:29:22.486246  2067 server_base.cc:1061] running on GCE node
W20260709 12:29:22.488449 16064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:22.488452 15856 ts_manager.cc:194] Registered new tserver with Master: f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:22.489856  2067 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:22.490121  2067 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:22.490334  2067 hybrid_clock.cc:648] HybridClock initialized: now 1783600162490279 us; error 0 us; skew 500 ppm
I20260709 12:29:22.491014  2067 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:22.491050 15856 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36360
I20260709 12:29:22.494145  2067 webserver.cc:533] Webserver started at http://127.2.4.195:36605/ using document root <none> and password file <none>
I20260709 12:29:22.494809  2067 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:22.495036  2067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:22.495368  2067 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:22.496820  2067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/ts-2-root/instance:
uuid: "cd106f11cb4c43f19e686df9f9a365f9"
format_stamp: "Formatted at 2026-07-09 12:29:22 on dist-test-slave-4qvp"
I20260709 12:29:22.502596  2067 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 12:29:22.507238 16069 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:22.508342  2067 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.004s
I20260709 12:29:22.508895  2067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/ts-2-root
uuid: "cd106f11cb4c43f19e686df9f9a365f9"
format_stamp: "Formatted at 2026-07-09 12:29:22 on dist-test-slave-4qvp"
I20260709 12:29:22.509294  2067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task1MeDuD/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599727094521-2067-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:22.534567  2067 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:22.535876  2067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:22.537807  2067 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:22.540316  2067 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:22.540518  2067 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:22.540848  2067 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:22.540997  2067 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:22.590384  2067 rpc_server.cc:307] RPC server started. Bound to: 127.2.4.195:33947
I20260709 12:29:22.590518 16131 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.4.195:33947 every 8 connection(s)
I20260709 12:29:22.607544 16132 heartbeater.cc:344] Connected to a master server at 127.2.4.254:37151
I20260709 12:29:22.608196 16132 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:22.609208 16132 heartbeater.cc:507] Master 127.2.4.254:37151 requested a full tablet report, sending...
I20260709 12:29:22.611773 15856 ts_manager.cc:194] Registered new tserver with Master: cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:22.611968  2067 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017854691s
I20260709 12:29:22.613987 15856 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36364
I20260709 12:29:22.645226 15856 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36366:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:22.648490 15856 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:22.737133 16023 tablet_service.cc:1511] Processing CreateTablet for tablet dfdf24842682422dab37b9ccc8ab0c24 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:22.739066 16023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfdf24842682422dab37b9ccc8ab0c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.743083 16022 tablet_service.cc:1511] Processing CreateTablet for tablet 85fc54f820da47d0951fabd0d60156d0 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:22.745033 16022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85fc54f820da47d0951fabd0d60156d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.753886 16021 tablet_service.cc:1511] Processing CreateTablet for tablet d37749a2508249b2afaf4467cc56d190 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:22.755750 16021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d37749a2508249b2afaf4467cc56d190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.757432 16097 tablet_service.cc:1511] Processing CreateTablet for tablet dfdf24842682422dab37b9ccc8ab0c24 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:22.759305 16097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfdf24842682422dab37b9ccc8ab0c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.759085 16020 tablet_service.cc:1511] Processing CreateTablet for tablet 11a00e1f25ac46bf8df46d1a30c6f75b (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:22.760742 16020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11a00e1f25ac46bf8df46d1a30c6f75b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.760560 16092 tablet_service.cc:1511] Processing CreateTablet for tablet 87b688925a5c41e1b41d0a4d8652e834 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:22.762351 16092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87b688925a5c41e1b41d0a4d8652e834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.762004 16096 tablet_service.cc:1511] Processing CreateTablet for tablet 85fc54f820da47d0951fabd0d60156d0 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:22.763019 16019 tablet_service.cc:1511] Processing CreateTablet for tablet 72091658e1c047c08f409622ac3910b3 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:22.763787 16096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85fc54f820da47d0951fabd0d60156d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.764647 16019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72091658e1c047c08f409622ac3910b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.767374 16095 tablet_service.cc:1511] Processing CreateTablet for tablet d37749a2508249b2afaf4467cc56d190 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:22.767827 15947 tablet_service.cc:1511] Processing CreateTablet for tablet 85fc54f820da47d0951fabd0d60156d0 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:22.769182 16095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d37749a2508249b2afaf4467cc56d190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.770977 16093 tablet_service.cc:1511] Processing CreateTablet for tablet 72091658e1c047c08f409622ac3910b3 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:22.772706 16093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72091658e1c047c08f409622ac3910b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.769191 15947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85fc54f820da47d0951fabd0d60156d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.775054 16094 tablet_service.cc:1511] Processing CreateTablet for tablet 11a00e1f25ac46bf8df46d1a30c6f75b (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:22.775851 16091 tablet_service.cc:1511] Processing CreateTablet for tablet 2a65d88797ab489db65de91babc2828f (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:22.776757 16094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11a00e1f25ac46bf8df46d1a30c6f75b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.779044 15944 tablet_service.cc:1511] Processing CreateTablet for tablet 72091658e1c047c08f409622ac3910b3 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:22.780762 15944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72091658e1c047c08f409622ac3910b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.783160 15942 tablet_service.cc:1511] Processing CreateTablet for tablet 2a65d88797ab489db65de91babc2828f (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:22.783722 16089 tablet_service.cc:1511] Processing CreateTablet for tablet 3630e85b83b04193a66ec5acd5611eeb (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:22.786019 16017 tablet_service.cc:1511] Processing CreateTablet for tablet 2a65d88797ab489db65de91babc2828f (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:22.787632 16017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a65d88797ab489db65de91babc2828f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.784646 15942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a65d88797ab489db65de91babc2828f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.790993 15941 tablet_service.cc:1511] Processing CreateTablet for tablet 84514ec125fd474ba0c033b11a66d145 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:22.792060 16016 tablet_service.cc:1511] Processing CreateTablet for tablet 84514ec125fd474ba0c033b11a66d145 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:22.792641 15941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84514ec125fd474ba0c033b11a66d145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.793893 16016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84514ec125fd474ba0c033b11a66d145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.796048 16018 tablet_service.cc:1511] Processing CreateTablet for tablet 87b688925a5c41e1b41d0a4d8652e834 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:22.798729 15943 tablet_service.cc:1511] Processing CreateTablet for tablet 87b688925a5c41e1b41d0a4d8652e834 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:22.800215 15943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87b688925a5c41e1b41d0a4d8652e834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.802958 16015 tablet_service.cc:1511] Processing CreateTablet for tablet 3630e85b83b04193a66ec5acd5611eeb (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:22.805233 15940 tablet_service.cc:1511] Processing CreateTablet for tablet 3630e85b83b04193a66ec5acd5611eeb (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:22.807118 16091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a65d88797ab489db65de91babc2828f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.808331 15946 tablet_service.cc:1511] Processing CreateTablet for tablet d37749a2508249b2afaf4467cc56d190 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:22.810220 16089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3630e85b83b04193a66ec5acd5611eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.810582 15945 tablet_service.cc:1511] Processing CreateTablet for tablet 11a00e1f25ac46bf8df46d1a30c6f75b (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:22.813035 15948 tablet_service.cc:1511] Processing CreateTablet for tablet dfdf24842682422dab37b9ccc8ab0c24 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:22.780303 16090 tablet_service.cc:1511] Processing CreateTablet for tablet 84514ec125fd474ba0c033b11a66d145 (DEFAULT_TABLE table=test-workload [id=f49dbcf35972439c926e5bd134364913]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:22.815155 16090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84514ec125fd474ba0c033b11a66d145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.797564 16018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87b688925a5c41e1b41d0a4d8652e834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.819271 16015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3630e85b83b04193a66ec5acd5611eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.831879 15940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3630e85b83b04193a66ec5acd5611eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.833766 15945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11a00e1f25ac46bf8df46d1a30c6f75b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.835925 15948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfdf24842682422dab37b9ccc8ab0c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:22.837560 15946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d37749a2508249b2afaf4467cc56d190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:23.090483 16149 tablet_bootstrap.cc:492] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:23.091624 16148 tablet_bootstrap.cc:492] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:23.102816 16149 tablet_bootstrap.cc:654] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.120237 16148 tablet_bootstrap.cc:654] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.133761 16147 tablet_bootstrap.cc:492] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:23.144084 16147 tablet_bootstrap.cc:654] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:23.145299 15904 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:23.150550 16149 tablet_bootstrap.cc:492] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:23.151316 16149 ts_tablet_manager.cc:1403] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.019s
I20260709 12:29:23.154815 16149 raft_consensus.cc:359] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.155699 16149 raft_consensus.cc:385] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.156036 16149 raft_consensus.cc:740] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.157027 16149 consensus_queue.cc:260] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.166276 16147 tablet_bootstrap.cc:492] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:23.167321 16147 ts_tablet_manager.cc:1403] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.015s
I20260709 12:29:23.172948 16147 raft_consensus.cc:359] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.174166 16147 raft_consensus.cc:385] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.174661 16147 raft_consensus.cc:740] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.175900 15983 heartbeater.cc:499] Master 127.2.4.254:37151 was elected leader, sending a full tablet report...
I20260709 12:29:23.176645 16149 ts_tablet_manager.cc:1434] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.009s
I20260709 12:29:23.176667 16148 tablet_bootstrap.cc:492] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:23.177959 16148 ts_tablet_manager.cc:1403] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.087s	user 0.016s	sys 0.006s
I20260709 12:29:23.177057 16147 consensus_queue.cc:260] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.178507 16149 tablet_bootstrap.cc:492] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:23.182701 16148 raft_consensus.cc:359] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.184038 16148 raft_consensus.cc:385] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.184515 16148 raft_consensus.cc:740] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.185783 16148 consensus_queue.cc:260] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.187254 16149 tablet_bootstrap.cc:654] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.200834 16149 tablet_bootstrap.cc:492] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:23.201589 16149 ts_tablet_manager.cc:1403] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.006s
I20260709 12:29:23.205535 16149 raft_consensus.cc:359] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.206774 16149 raft_consensus.cc:385] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.207178 16149 raft_consensus.cc:740] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.208156 16149 consensus_queue.cc:260] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.211212 16149 ts_tablet_manager.cc:1434] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 12:29:23.212466 16149 tablet_bootstrap.cc:492] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:23.221904 16149 tablet_bootstrap.cc:654] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.223192 16132 heartbeater.cc:499] Master 127.2.4.254:37151 was elected leader, sending a full tablet report...
I20260709 12:29:23.223649 16148 ts_tablet_manager.cc:1434] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.045s	user 0.023s	sys 0.019s
I20260709 12:29:23.223335 16058 heartbeater.cc:499] Master 127.2.4.254:37151 was elected leader, sending a full tablet report...
I20260709 12:29:23.225039 16148 tablet_bootstrap.cc:492] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:23.230424 16152 raft_consensus.cc:493] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.231098 16152 raft_consensus.cc:515] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.234043 16148 tablet_bootstrap.cc:654] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.235343 16154 raft_consensus.cc:493] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.236066 16154 raft_consensus.cc:515] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.257287 16155 raft_consensus.cc:493] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.257997 16147 ts_tablet_manager.cc:1434] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.090s	user 0.016s	sys 0.034s
I20260709 12:29:23.258044 16155 raft_consensus.cc:515] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.259372 16152 leader_election.cc:290] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.260120 16147 tablet_bootstrap.cc:492] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:23.267104 16155 leader_election.cc:290] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:23.270237 16154 leader_election.cc:290] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.270936 16147 tablet_bootstrap.cc:654] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.286293 16148 tablet_bootstrap.cc:492] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:23.287019 16148 ts_tablet_manager.cc:1403] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.062s	user 0.019s	sys 0.014s
I20260709 12:29:23.290930 16148 raft_consensus.cc:359] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.291996 16148 raft_consensus.cc:385] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.292362 16148 raft_consensus.cc:740] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.294587 16148 consensus_queue.cc:260] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.298736 16149 tablet_bootstrap.cc:492] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:23.302659 16149 ts_tablet_manager.cc:1403] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.090s	user 0.019s	sys 0.023s
I20260709 12:29:23.304060 16148 ts_tablet_manager.cc:1434] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.004s
I20260709 12:29:23.305434 16148 tablet_bootstrap.cc:492] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:23.306846 16149 raft_consensus.cc:359] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.307802 16149 raft_consensus.cc:385] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.308117 16149 raft_consensus.cc:740] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.309018 16149 consensus_queue.cc:260] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.326663 16149 ts_tablet_manager.cc:1434] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 12:29:23.333731 16149 tablet_bootstrap.cc:492] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
W20260709 12:29:23.339797 15984 tablet.cc:2406] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:29:23.341390 16148 tablet_bootstrap.cc:654] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.346983 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfdf24842682422dab37b9ccc8ab0c24" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:23.349048 15958 raft_consensus.cc:2468] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 0.
I20260709 12:29:23.355400 16147 tablet_bootstrap.cc:492] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:23.355652 15996 leader_election.cc:304] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:23.359992 16149 tablet_bootstrap.cc:654] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.356113 16147 ts_tablet_manager.cc:1403] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.096s	user 0.015s	sys 0.007s
I20260709 12:29:23.361351 16154 raft_consensus.cc:2804] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:23.362053 16154 raft_consensus.cc:493] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.362567 16154 raft_consensus.cc:3060] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.370464 16147 raft_consensus.cc:359] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.371703 16147 raft_consensus.cc:385] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.372076 16147 raft_consensus.cc:740] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.373160 16147 consensus_queue.cc:260] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.380391 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfdf24842682422dab37b9ccc8ab0c24" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:23.381490 16148 tablet_bootstrap.cc:492] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:23.384136 16148 ts_tablet_manager.cc:1403] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.079s	user 0.020s	sys 0.005s
W20260709 12:29:23.386731 15998 leader_election.cc:343] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.388139 16148 raft_consensus.cc:359] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.389299 16148 raft_consensus.cc:385] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.391736 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87b688925a5c41e1b41d0a4d8652e834" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
W20260709 12:29:23.394105 16070 leader_election.cc:343] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.400520 16152 raft_consensus.cc:493] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.401289 16152 raft_consensus.cc:515] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.405653 16147 ts_tablet_manager.cc:1434] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.044s	user 0.013s	sys 0.013s
I20260709 12:29:23.405831 16149 tablet_bootstrap.cc:492] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:23.406216 16152 leader_election.cc:290] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.406729 16149 ts_tablet_manager.cc:1403] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.073s	user 0.021s	sys 0.010s
I20260709 12:29:23.411895 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfdf24842682422dab37b9ccc8ab0c24" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
W20260709 12:29:23.413969 15923 leader_election.cc:343] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.414773 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fc54f820da47d0951fabd0d60156d0" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
W20260709 12:29:23.416848 15923 leader_election.cc:343] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.417371 16033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfdf24842682422dab37b9ccc8ab0c24" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:23.418329 16032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fc54f820da47d0951fabd0d60156d0" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:23.410543 16149 raft_consensus.cc:359] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.419512 16149 raft_consensus.cc:385] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.419821 16149 raft_consensus.cc:740] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
W20260709 12:29:23.420258 15921 leader_election.cc:343] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.420957 15921 leader_election.cc:304] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f742bdd975884e759c43e1311e22ac23; no voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7
I20260709 12:29:23.420701 16149 consensus_queue.cc:260] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.422106 16152 raft_consensus.cc:2749] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:23.407356 16147 tablet_bootstrap.cc:492] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:23.410903 16154 raft_consensus.cc:515] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.423650 16032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87b688925a5c41e1b41d0a4d8652e834" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:23.423904 16033 raft_consensus.cc:2393] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f742bdd975884e759c43e1311e22ac23 in current term 1: Already voted for candidate f8085f7497804cc096027b14b64f95b7 in this term.
W20260709 12:29:23.425580 16070 leader_election.cc:343] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.426268 16070 leader_election.cc:304] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9; no voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7
I20260709 12:29:23.427675 16155 raft_consensus.cc:2749] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:23.430364 15921 leader_election.cc:304] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f742bdd975884e759c43e1311e22ac23; no voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7
I20260709 12:29:23.411032 16148 raft_consensus.cc:740] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.431654 16147 tablet_bootstrap.cc:654] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.432134 16148 consensus_queue.cc:260] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.432785 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfdf24842682422dab37b9ccc8ab0c24" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:23.433665 15958 raft_consensus.cc:3060] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.434296 16154 leader_election.cc:290] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.438221 16155 raft_consensus.cc:493] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.438958 16155 raft_consensus.cc:515] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.441537 16155 leader_election.cc:290] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:23.442838 16148 ts_tablet_manager.cc:1434] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.001s
I20260709 12:29:23.444208 16148 tablet_bootstrap.cc:492] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:23.447045 16149 ts_tablet_manager.cc:1434] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 12:29:23.452348 16148 tablet_bootstrap.cc:654] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.458724 16148 tablet_bootstrap.cc:492] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:23.459290 16148 ts_tablet_manager.cc:1403] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 12:29:23.459931 16149 tablet_bootstrap.cc:492] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:23.461990 16148 raft_consensus.cc:359] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.462754 16148 raft_consensus.cc:385] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.463032 16148 raft_consensus.cc:740] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.463842 16148 consensus_queue.cc:260] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.466317 16148 ts_tablet_manager.cc:1434] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:29:23.467504 16148 tablet_bootstrap.cc:492] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:23.468036 15958 raft_consensus.cc:2468] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 1.
I20260709 12:29:23.469102 16175 raft_consensus.cc:2749] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:23.470233 15996 leader_election.cc:304] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:23.471998 16154 raft_consensus.cc:2804] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:23.472740 16154 raft_consensus.cc:697] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Becoming Leader. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:23.473970 16154 consensus_queue.cc:237] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.477527 16148 tablet_bootstrap.cc:654] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.483167 16033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3630e85b83b04193a66ec5acd5611eeb" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:23.484300 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3630e85b83b04193a66ec5acd5611eeb" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
W20260709 12:29:23.485828 16070 leader_election.cc:343] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.486568 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfdf24842682422dab37b9ccc8ab0c24" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:23.490733 16147 tablet_bootstrap.cc:492] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:23.491452 16147 ts_tablet_manager.cc:1403] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.084s	user 0.015s	sys 0.017s
I20260709 12:29:23.491076 15854 catalog_manager.cc:5697] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8085f7497804cc096027b14b64f95b7 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "f8085f7497804cc096027b14b64f95b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:23.492560 16149 tablet_bootstrap.cc:654] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:23.495257 15998 leader_election.cc:343] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:23.495428 16070 leader_election.cc:343] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.496229 16070 leader_election.cc:304] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9; no voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7
I20260709 12:29:23.496431 16147 raft_consensus.cc:359] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.497219 16147 raft_consensus.cc:385] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.497462 16176 raft_consensus.cc:2749] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:23.497598 16147 raft_consensus.cc:740] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.498936 16147 consensus_queue.cc:260] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.501420 16148 tablet_bootstrap.cc:492] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:23.502332 16148 ts_tablet_manager.cc:1403] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.035s	user 0.006s	sys 0.016s
I20260709 12:29:23.502508 16147 ts_tablet_manager.cc:1434] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 12:29:23.504194 16147 tablet_bootstrap.cc:492] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:23.505887 16148 raft_consensus.cc:359] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.506639 16148 raft_consensus.cc:385] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.506902 16148 raft_consensus.cc:740] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.507685 16148 consensus_queue.cc:260] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.509434 16148 ts_tablet_manager.cc:1434] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:29:23.510844 16148 tablet_bootstrap.cc:492] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:23.513312 16147 tablet_bootstrap.cc:654] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.516995 16148 tablet_bootstrap.cc:654] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.531641 16147 tablet_bootstrap.cc:492] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:23.532297 16147 ts_tablet_manager.cc:1403] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.028s	user 0.020s	sys 0.005s
I20260709 12:29:23.535704 16147 raft_consensus.cc:359] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.536552 16147 raft_consensus.cc:385] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.536859 16147 raft_consensus.cc:740] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.537796 16149 tablet_bootstrap.cc:492] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:23.537804 16147 consensus_queue.cc:260] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.538719 16149 ts_tablet_manager.cc:1403] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.079s	user 0.018s	sys 0.019s
I20260709 12:29:23.540921 16147 ts_tablet_manager.cc:1434] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.002s
I20260709 12:29:23.541410 16175 raft_consensus.cc:493] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.542174 16149 raft_consensus.cc:359] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.542164 16175 raft_consensus.cc:515] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.542172 16147 tablet_bootstrap.cc:492] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:23.546043 16033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a65d88797ab489db65de91babc2828f" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
W20260709 12:29:23.547937 15921 leader_election.cc:343] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.550587 16147 tablet_bootstrap.cc:654] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.550458 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a65d88797ab489db65de91babc2828f" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:23.550969 16175 leader_election.cc:290] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.551218 16106 raft_consensus.cc:2468] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 0.
I20260709 12:29:23.552882 15923 leader_election.cc:304] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: f8085f7497804cc096027b14b64f95b7
I20260709 12:29:23.554327 16175 raft_consensus.cc:2804] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:23.554716 16175 raft_consensus.cc:493] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.554909 16148 tablet_bootstrap.cc:492] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:23.555063 16175 raft_consensus.cc:3060] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.555559 16148 ts_tablet_manager.cc:1403] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.013s
I20260709 12:29:23.556399 16149 raft_consensus.cc:385] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.556843 16149 raft_consensus.cc:740] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.557917 16149 consensus_queue.cc:260] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.559005 16147 tablet_bootstrap.cc:492] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:23.559677 16147 ts_tablet_manager.cc:1403] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.003s
I20260709 12:29:23.559468 16148 raft_consensus.cc:359] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.560375 16148 raft_consensus.cc:385] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.560724 16148 raft_consensus.cc:740] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.561612 16148 consensus_queue.cc:260] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.562755 16149 ts_tablet_manager.cc:1434] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 12:29:23.563536 16147 raft_consensus.cc:359] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.564035 16149 tablet_bootstrap.cc:492] T 84514ec125fd474ba0c033b11a66d145 P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:23.564483 16147 raft_consensus.cc:385] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.564875 16147 raft_consensus.cc:740] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.565758 16147 consensus_queue.cc:260] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.566490 16148 ts_tablet_manager.cc:1434] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 12:29:23.567759 16148 tablet_bootstrap.cc:492] T 84514ec125fd474ba0c033b11a66d145 P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:23.568327 16147 ts_tablet_manager.cc:1434] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:29:23.571894 16149 tablet_bootstrap.cc:654] T 84514ec125fd474ba0c033b11a66d145 P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.572898 16147 tablet_bootstrap.cc:492] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:23.575655 16148 tablet_bootstrap.cc:654] T 84514ec125fd474ba0c033b11a66d145 P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.582914 16147 tablet_bootstrap.cc:654] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.611176 16154 raft_consensus.cc:493] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.611967 16154 raft_consensus.cc:515] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.615772 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37749a2508249b2afaf4467cc56d190" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:23.616559 16106 raft_consensus.cc:2468] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 0.
I20260709 12:29:23.615525 16175 raft_consensus.cc:515] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.618680 15998 leader_election.cc:304] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:23.618844 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37749a2508249b2afaf4467cc56d190" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:23.619627 16154 leader_election.cc:290] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.619722 16173 raft_consensus.cc:2804] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:23.620337 16149 tablet_bootstrap.cc:492] T 84514ec125fd474ba0c033b11a66d145 P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:23.621222 16173 raft_consensus.cc:493] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.621718 16149 ts_tablet_manager.cc:1403] T 84514ec125fd474ba0c033b11a66d145 P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.058s	user 0.035s	sys 0.020s
I20260709 12:29:23.621737 16173 raft_consensus.cc:3060] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.622225 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a65d88797ab489db65de91babc2828f" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:23.622865 16106 raft_consensus.cc:3060] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.623055 16147 tablet_bootstrap.cc:492] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:23.620756 16033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a65d88797ab489db65de91babc2828f" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7"
W20260709 12:29:23.620697 15996 leader_election.cc:343] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.623775 16147 ts_tablet_manager.cc:1403] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.051s	user 0.033s	sys 0.009s
W20260709 12:29:23.625026 15921 leader_election.cc:343] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.625164 16149 raft_consensus.cc:359] T 84514ec125fd474ba0c033b11a66d145 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.626053 16149 raft_consensus.cc:385] T 84514ec125fd474ba0c033b11a66d145 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.626351 16149 raft_consensus.cc:740] T 84514ec125fd474ba0c033b11a66d145 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.620886 16175 leader_election.cc:290] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 election: Requested vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.627029 16147 raft_consensus.cc:359] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.627853 16147 raft_consensus.cc:385] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.627595 16149 consensus_queue.cc:260] T 84514ec125fd474ba0c033b11a66d145 P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.628197 16147 raft_consensus.cc:740] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.629017 16147 consensus_queue.cc:260] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.630411 16149 ts_tablet_manager.cc:1434] T 84514ec125fd474ba0c033b11a66d145 P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:23.631342 16147 ts_tablet_manager.cc:1434] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:29:23.631814 16149 tablet_bootstrap.cc:492] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:23.632373 16147 tablet_bootstrap.cc:492] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:23.634529 16173 raft_consensus.cc:515] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.638139 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37749a2508249b2afaf4467cc56d190" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23"
W20260709 12:29:23.639765 15996 leader_election.cc:343] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.640112 16149 tablet_bootstrap.cc:654] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.641978 16148 tablet_bootstrap.cc:492] T 84514ec125fd474ba0c033b11a66d145 P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:23.642655 16148 ts_tablet_manager.cc:1403] T 84514ec125fd474ba0c033b11a66d145 P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.075s	user 0.022s	sys 0.020s
I20260709 12:29:23.640134 16147 tablet_bootstrap.cc:654] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.646169 16148 raft_consensus.cc:359] T 84514ec125fd474ba0c033b11a66d145 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.647176 16148 raft_consensus.cc:385] T 84514ec125fd474ba0c033b11a66d145 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.647521 16148 raft_consensus.cc:740] T 84514ec125fd474ba0c033b11a66d145 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.648612 16148 consensus_queue.cc:260] T 84514ec125fd474ba0c033b11a66d145 P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.652556 16173 leader_election.cc:290] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.653594 16148 ts_tablet_manager.cc:1434] T 84514ec125fd474ba0c033b11a66d145 P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.004s
I20260709 12:29:23.654595 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37749a2508249b2afaf4467cc56d190" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:23.654909 16148 tablet_bootstrap.cc:492] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:23.655414 16107 raft_consensus.cc:3060] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.656934 16149 tablet_bootstrap.cc:492] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:23.657536 16147 tablet_bootstrap.cc:492] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:23.658337 16147 ts_tablet_manager.cc:1403] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.003s
I20260709 12:29:23.662082 16147 raft_consensus.cc:359] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.662851 16148 tablet_bootstrap.cc:654] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.663060 16147 raft_consensus.cc:385] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.663584 16147 raft_consensus.cc:740] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.664623 16147 consensus_queue.cc:260] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.657573 16149 ts_tablet_manager.cc:1403] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.004s
I20260709 12:29:23.667372 16147 ts_tablet_manager.cc:1434] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.009s	user 0.000s	sys 0.007s
I20260709 12:29:23.668776 16147 tablet_bootstrap.cc:492] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:23.671731 16149 raft_consensus.cc:359] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.672683 16149 raft_consensus.cc:385] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.672988 16149 raft_consensus.cc:740] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.673830 16149 consensus_queue.cc:260] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.674997 16148 tablet_bootstrap.cc:492] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:23.675590 16148 ts_tablet_manager.cc:1403] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.000s
I20260709 12:29:23.677160 16147 tablet_bootstrap.cc:654] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.677829 16106 raft_consensus.cc:2468] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 1.
I20260709 12:29:23.678478 16148 raft_consensus.cc:359] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.679275 16148 raft_consensus.cc:385] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.679342 15923 leader_election.cc:304] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: f8085f7497804cc096027b14b64f95b7
I20260709 12:29:23.679626 16148 raft_consensus.cc:740] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.680608 16148 consensus_queue.cc:260] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.683066 16148 ts_tablet_manager.cc:1434] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:29:23.684113 16148 tablet_bootstrap.cc:492] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:23.684111 16175 raft_consensus.cc:2804] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:23.689862 16152 raft_consensus.cc:493] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.690373 16152 raft_consensus.cc:515] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.692003 16149 ts_tablet_manager.cc:1434] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 12:29:23.692649 16152 leader_election.cc:290] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.694504 16033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11a00e1f25ac46bf8df46d1a30c6f75b" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:23.695242 16149 tablet_bootstrap.cc:492] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
W20260709 12:29:23.696909 15921 leader_election.cc:343] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.698180 16148 tablet_bootstrap.cc:654] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.698777 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11a00e1f25ac46bf8df46d1a30c6f75b" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:23.699753 16106 raft_consensus.cc:2468] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 0.
I20260709 12:29:23.701362 15923 leader_election.cc:304] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: f8085f7497804cc096027b14b64f95b7
I20260709 12:29:23.703037 16152 raft_consensus.cc:2804] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:23.703500 16152 raft_consensus.cc:493] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.704107 16152 raft_consensus.cc:3060] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.705843 16147 tablet_bootstrap.cc:492] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:23.706097 16107 raft_consensus.cc:2468] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 1.
I20260709 12:29:23.706652 16147 ts_tablet_manager.cc:1403] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.038s	user 0.019s	sys 0.014s
I20260709 12:29:23.707652 15998 leader_election.cc:304] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7; no voters: f742bdd975884e759c43e1311e22ac23
I20260709 12:29:23.709141 16173 raft_consensus.cc:2804] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:23.709692 16173 raft_consensus.cc:697] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Becoming Leader. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:23.710584 16173 consensus_queue.cc:237] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.711673 16175 raft_consensus.cc:697] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Becoming Leader. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Running, Role: LEADER
I20260709 12:29:23.712728 16175 consensus_queue.cc:237] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.714021 16149 tablet_bootstrap.cc:654] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.710532 16147 raft_consensus.cc:359] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.715838 16152 raft_consensus.cc:515] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.717425 16147 raft_consensus.cc:385] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.718175 16147 raft_consensus.cc:740] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.720815 16147 consensus_queue.cc:260] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.722319 16173 raft_consensus.cc:493] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.723219 16148 tablet_bootstrap.cc:492] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:23.723129 16173 raft_consensus.cc:515] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.723815 16148 ts_tablet_manager.cc:1403] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.001s
I20260709 12:29:23.725979 16147 ts_tablet_manager.cc:1434] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:29:23.726742 16154 raft_consensus.cc:493] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.726832 16148 raft_consensus.cc:359] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.727865 16148 raft_consensus.cc:385] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.727479 16154 raft_consensus.cc:515] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.728171 16148 raft_consensus.cc:740] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.729156 16148 consensus_queue.cc:260] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.731479 16148 ts_tablet_manager.cc:1434] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:29:23.727043 16147 tablet_bootstrap.cc:492] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:23.734340 16154 leader_election.cc:290] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.734975 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11a00e1f25ac46bf8df46d1a30c6f75b" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:23.735735 16106 raft_consensus.cc:3060] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.738471 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fc54f820da47d0951fabd0d60156d0" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:23.739756 16107 raft_consensus.cc:2468] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 0.
I20260709 12:29:23.742293 16147 tablet_bootstrap.cc:654] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.742578 16033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11a00e1f25ac46bf8df46d1a30c6f75b" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:23.743667 16152 leader_election.cc:290] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 election: Requested vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.745924 16173 leader_election.cc:290] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.746346 15998 leader_election.cc:304] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:23.746587 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fc54f820da47d0951fabd0d60156d0" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:23.747413 15957 raft_consensus.cc:2468] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 0.
W20260709 12:29:23.748682 15921 leader_election.cc:343] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:23.749141 16105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84514ec125fd474ba0c033b11a66d145" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:23.750720 16105 raft_consensus.cc:2468] T 84514ec125fd474ba0c033b11a66d145 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 0.
I20260709 12:29:23.752607 15998 leader_election.cc:304] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:23.753189 16173 raft_consensus.cc:2804] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:23.753976 16154 raft_consensus.cc:493] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.754753 16154 raft_consensus.cc:515] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.755153 16173 raft_consensus.cc:493] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.754959 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84514ec125fd474ba0c033b11a66d145" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:23.755684 16173 raft_consensus.cc:3060] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.755928 15957 raft_consensus.cc:2468] T 84514ec125fd474ba0c033b11a66d145 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 0.
I20260709 12:29:23.757259 16154 leader_election.cc:290] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.760922 16154 raft_consensus.cc:2804] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:23.761336 16154 raft_consensus.cc:493] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.761710 16154 raft_consensus.cc:3060] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.762544 16149 tablet_bootstrap.cc:492] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:23.763264 16149 ts_tablet_manager.cc:1403] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.068s	user 0.020s	sys 0.011s
I20260709 12:29:23.766151 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3630e85b83b04193a66ec5acd5611eeb" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:23.736384 15855 catalog_manager.cc:5697] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8085f7497804cc096027b14b64f95b7 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "f8085f7497804cc096027b14b64f95b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:23.766650 16106 raft_consensus.cc:2468] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 1.
I20260709 12:29:23.766876 16149 raft_consensus.cc:359] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.759295 15856 catalog_manager.cc:5697] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 reported cstate change: term changed from 0 to 1, leader changed from <none> to f742bdd975884e759c43e1311e22ac23 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "f742bdd975884e759c43e1311e22ac23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:23.768946 15923 leader_election.cc:304] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: f8085f7497804cc096027b14b64f95b7
I20260709 12:29:23.770478 16152 raft_consensus.cc:2804] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:23.771044 16152 raft_consensus.cc:697] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Becoming Leader. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Running, Role: LEADER
I20260709 12:29:23.770560 16173 raft_consensus.cc:515] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.772073 16152 consensus_queue.cc:237] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.773945 16173 leader_election.cc:290] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.768025 16149 raft_consensus.cc:385] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.776288 16149 raft_consensus.cc:740] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.777246 16149 consensus_queue.cc:260] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.778702 15957 raft_consensus.cc:2468] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 0.
I20260709 12:29:23.778900 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fc54f820da47d0951fabd0d60156d0" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:23.779480 16149 ts_tablet_manager.cc:1434] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:29:23.779704 15958 raft_consensus.cc:3060] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.780834 16149 tablet_bootstrap.cc:492] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:23.788336 16176 raft_consensus.cc:493] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.789070 16176 raft_consensus.cc:515] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.789825 16149 tablet_bootstrap.cc:654] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:23.791708 15958 raft_consensus.cc:2468] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 1.
I20260709 12:29:23.793949 15956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfdf24842682422dab37b9ccc8ab0c24" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:23.804935 15956 raft_consensus.cc:2393] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd106f11cb4c43f19e686df9f9a365f9 in current term 1: Already voted for candidate f8085f7497804cc096027b14b64f95b7 in this term.
I20260709 12:29:23.791761 16176 leader_election.cc:290] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651)
I20260709 12:29:23.796627 16175 raft_consensus.cc:493] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.806759 16175 raft_consensus.cc:515] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.800978 15996 leader_election.cc:304] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:23.811196 16176 raft_consensus.cc:493] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.811534 15996 leader_election.cc:304] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:23.811967 16176 raft_consensus.cc:515] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.813088 16173 raft_consensus.cc:2804] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:23.813680 16173 raft_consensus.cc:493] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.814128 16173 raft_consensus.cc:3060] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.815850 16176 leader_election.cc:290] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:23.816133 16175 leader_election.cc:290] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.818840 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72091658e1c047c08f409622ac3910b3" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:23.819075 16200 raft_consensus.cc:2804] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:23.819690 16033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87b688925a5c41e1b41d0a4d8652e834" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:23.819857 15958 raft_consensus.cc:2468] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 0.
I20260709 12:29:23.820556 16033 raft_consensus.cc:2468] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 0.
I20260709 12:29:23.821470 16070 leader_election.cc:304] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
I20260709 12:29:23.818878 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87b688925a5c41e1b41d0a4d8652e834" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:23.822695 16176 raft_consensus.cc:2804] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:23.823132 16176 raft_consensus.cc:493] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.823479 16176 raft_consensus.cc:3060] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.823673 15921 leader_election.cc:304] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:23.824123 15855 catalog_manager.cc:5697] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 reported cstate change: term changed from 0 to 1, leader changed from <none> to f742bdd975884e759c43e1311e22ac23 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "f742bdd975884e759c43e1311e22ac23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:23.825030 16175 raft_consensus.cc:2804] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:23.825781 16175 raft_consensus.cc:493] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.822713 16106 raft_consensus.cc:2468] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 0.
I20260709 12:29:23.826587 16175 raft_consensus.cc:3060] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.832705 16173 raft_consensus.cc:515] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.825661 16154 raft_consensus.cc:515] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
W20260709 12:29:23.818034 15998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.195:33947, user_credentials={real_user=slave}} blocked reactor thread for 65552.1us
I20260709 12:29:23.837023 16149 tablet_bootstrap.cc:492] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:23.837694 16149 ts_tablet_manager.cc:1403] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.057s	user 0.025s	sys 0.011s
I20260709 12:29:23.837656 16033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfdf24842682422dab37b9ccc8ab0c24" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:23.839334 16032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72091658e1c047c08f409622ac3910b3" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:23.840049 16032 raft_consensus.cc:2468] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 0.
I20260709 12:29:23.840962 16149 raft_consensus.cc:359] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.841940 16149 raft_consensus.cc:385] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.842238 16149 raft_consensus.cc:740] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.842072 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3630e85b83b04193a66ec5acd5611eeb" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:23.843081 16149 consensus_queue.cc:260] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.845353 16154 leader_election.cc:290] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.842985 16105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fc54f820da47d0951fabd0d60156d0" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:23.851321 16105 raft_consensus.cc:3060] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.840855 16175 raft_consensus.cc:515] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.858083 16149 ts_tablet_manager.cc:1434] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:29:23.860114 16147 tablet_bootstrap.cc:492] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:23.860960 16147 ts_tablet_manager.cc:1403] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.134s	user 0.031s	sys 0.000s
I20260709 12:29:23.863946 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84514ec125fd474ba0c033b11a66d145" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:23.857264 16106 raft_consensus.cc:2468] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 0.
I20260709 12:29:23.864897 15958 raft_consensus.cc:3060] T 84514ec125fd474ba0c033b11a66d145 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.867919 16032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87b688925a5c41e1b41d0a4d8652e834" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:23.868783 16032 raft_consensus.cc:3060] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.844431 16070 leader_election.cc:304] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9; no voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7
I20260709 12:29:23.848124 16176 raft_consensus.cc:515] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.874652 16155 raft_consensus.cc:3060] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.877486 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84514ec125fd474ba0c033b11a66d145" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:23.878319 16106 raft_consensus.cc:3060] T 84514ec125fd474ba0c033b11a66d145 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.879532 15956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72091658e1c047c08f409622ac3910b3" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:23.880375 15956 raft_consensus.cc:3060] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.869459 16147 raft_consensus.cc:359] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.882153 16147 raft_consensus.cc:385] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:23.882229 16200 raft_consensus.cc:697] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Becoming Leader. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:23.882498 16147 raft_consensus.cc:740] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:23.883420 16200 consensus_queue.cc:237] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.883520 16147 consensus_queue.cc:260] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.886852 16147 ts_tablet_manager.cc:1434] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.002s
I20260709 12:29:23.881374 16105 raft_consensus.cc:2468] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 1.
I20260709 12:29:23.888712 15958 raft_consensus.cc:2468] T 84514ec125fd474ba0c033b11a66d145 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 1.
I20260709 12:29:23.894686 15956 raft_consensus.cc:2468] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 1.
I20260709 12:29:23.890967 16155 raft_consensus.cc:2749] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:23.892405 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3630e85b83b04193a66ec5acd5611eeb" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:23.893477 16173 leader_election.cc:290] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.895174 15996 leader_election.cc:304] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:23.898178 15957 raft_consensus.cc:3060] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.899561 16173 raft_consensus.cc:2804] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:23.900174 16173 raft_consensus.cc:697] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Becoming Leader. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:23.900211 16106 raft_consensus.cc:2468] T 84514ec125fd474ba0c033b11a66d145 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 1.
I20260709 12:29:23.896606 16175 leader_election.cc:290] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 election: Requested vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:23.901265 16173 consensus_queue.cc:237] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.902331 16070 leader_election.cc:304] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
I20260709 12:29:23.902976 16033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72091658e1c047c08f409622ac3910b3" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:23.903687 16033 raft_consensus.cc:3060] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.909204 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3630e85b83b04193a66ec5acd5611eeb" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:23.888254 16176 leader_election.cc:290] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:23.911909 16107 raft_consensus.cc:3060] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.913231 16155 raft_consensus.cc:2804] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:23.922859 16155 raft_consensus.cc:697] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 LEADER]: Becoming Leader. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Running, Role: LEADER
I20260709 12:29:23.931530 16173 consensus_queue.cc:1048] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:23.933936 16200 consensus_queue.cc:1048] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:23.934283 16105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87b688925a5c41e1b41d0a4d8652e834" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:23.935145 16105 raft_consensus.cc:3060] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:23.939850 16155 consensus_queue.cc:237] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.947576 15856 catalog_manager.cc:5697] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8085f7497804cc096027b14b64f95b7 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "f8085f7497804cc096027b14b64f95b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:23.943428 15957 raft_consensus.cc:2468] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 1.
I20260709 12:29:23.951126 16033 raft_consensus.cc:2468] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 1.
I20260709 12:29:23.950402 16105 raft_consensus.cc:2468] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 1.
I20260709 12:29:23.962965 15923 leader_election.cc:304] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
I20260709 12:29:23.964555 16175 raft_consensus.cc:2804] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:23.953565 15996 leader_election.cc:304] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:23.965246 16175 raft_consensus.cc:697] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Becoming Leader. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Running, Role: LEADER
I20260709 12:29:23.967221 16173 raft_consensus.cc:2804] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:23.968137 16173 raft_consensus.cc:697] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Becoming Leader. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:23.967952 16175 consensus_queue.cc:237] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.974229 16032 raft_consensus.cc:2468] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 1.
I20260709 12:29:23.970402 16173 consensus_queue.cc:237] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:23.990757 16107 raft_consensus.cc:2468] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 1.
I20260709 12:29:24.020342 15855 catalog_manager.cc:5697] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "cd106f11cb4c43f19e686df9f9a365f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:24.028707 15854 catalog_manager.cc:5697] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 reported cstate change: term changed from 0 to 1, leader changed from <none> to f742bdd975884e759c43e1311e22ac23 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "f742bdd975884e759c43e1311e22ac23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:24.032965 15856 catalog_manager.cc:5697] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8085f7497804cc096027b14b64f95b7 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "f8085f7497804cc096027b14b64f95b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:24.036074 15856 catalog_manager.cc:5697] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8085f7497804cc096027b14b64f95b7 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "f8085f7497804cc096027b14b64f95b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:24.078925 16175 raft_consensus.cc:493] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:24.079690 16175 raft_consensus.cc:515] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.081614 16175 leader_election.cc:290] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:24.082947 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37749a2508249b2afaf4467cc56d190" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:24.082877 16032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37749a2508249b2afaf4467cc56d190" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:24.083984 16106 raft_consensus.cc:2393] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f742bdd975884e759c43e1311e22ac23 in current term 1: Already voted for candidate f8085f7497804cc096027b14b64f95b7 in this term.
I20260709 12:29:24.085094 15923 leader_election.cc:304] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f742bdd975884e759c43e1311e22ac23; no voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7
I20260709 12:29:24.085892 16175 raft_consensus.cc:3060] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:24.091760 16175 raft_consensus.cc:2749] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:24.092650 16204 raft_consensus.cc:493] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:24.093386 16204 raft_consensus.cc:515] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.095585 16204 leader_election.cc:290] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:24.096763 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a65d88797ab489db65de91babc2828f" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:24.096966 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a65d88797ab489db65de91babc2828f" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:24.098119 16106 raft_consensus.cc:2393] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f8085f7497804cc096027b14b64f95b7 in current term 1: Already voted for candidate f742bdd975884e759c43e1311e22ac23 in this term.
I20260709 12:29:24.099345 15998 leader_election.cc:304] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8085f7497804cc096027b14b64f95b7; no voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23
I20260709 12:29:24.100112 16204 raft_consensus.cc:3060] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:24.108520 16204 raft_consensus.cc:2749] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:24.108779 16173 consensus_queue.cc:1048] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:24.126695 16200 consensus_queue.cc:1048] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:24.139065 16204 raft_consensus.cc:493] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:24.139732 16204 raft_consensus.cc:515] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.142195 16204 leader_election.cc:290] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:24.143241 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11a00e1f25ac46bf8df46d1a30c6f75b" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:24.143558 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11a00e1f25ac46bf8df46d1a30c6f75b" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:24.144982 16106 raft_consensus.cc:2393] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f8085f7497804cc096027b14b64f95b7 in current term 1: Already voted for candidate f742bdd975884e759c43e1311e22ac23 in this term.
I20260709 12:29:24.146042 15998 leader_election.cc:304] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8085f7497804cc096027b14b64f95b7; no voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23
I20260709 12:29:24.146938 16204 raft_consensus.cc:3060] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:24.150876 16032 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfdf24842682422dab37b9ccc8ab0c24"
dest_uuid: "f8085f7497804cc096027b14b64f95b7"
mode: GRACEFUL
new_leader_uuid: "f742bdd975884e759c43e1311e22ac23"
 from {username='slave'} at 127.0.0.1:43196
I20260709 12:29:24.151854 16032 raft_consensus.cc:606] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Received request to transfer leadership to TS f742bdd975884e759c43e1311e22ac23
I20260709 12:29:24.159799 16204 raft_consensus.cc:2749] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:24.159832 16175 consensus_queue.cc:1048] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:24.162932 16032 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85fc54f820da47d0951fabd0d60156d0"
dest_uuid: "f8085f7497804cc096027b14b64f95b7"
mode: GRACEFUL
new_leader_uuid: "f742bdd975884e759c43e1311e22ac23"
 from {username='slave'} at 127.0.0.1:43196
I20260709 12:29:24.164006 16032 raft_consensus.cc:606] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Received request to transfer leadership to TS f742bdd975884e759c43e1311e22ac23
I20260709 12:29:24.181243 16175 consensus_queue.cc:1048] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:24.185141 16033 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d37749a2508249b2afaf4467cc56d190"
dest_uuid: "f8085f7497804cc096027b14b64f95b7"
mode: GRACEFUL
new_leader_uuid: "f742bdd975884e759c43e1311e22ac23"
 from {username='slave'} at 127.0.0.1:43196
I20260709 12:29:24.186007 16033 raft_consensus.cc:606] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Received request to transfer leadership to TS f742bdd975884e759c43e1311e22ac23
I20260709 12:29:24.209121 16106 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72091658e1c047c08f409622ac3910b3"
dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
mode: GRACEFUL
new_leader_uuid: "f8085f7497804cc096027b14b64f95b7"
 from {username='slave'} at 127.0.0.1:48628
I20260709 12:29:24.209856 16106 raft_consensus.cc:606] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 LEADER]: Received request to transfer leadership to TS f8085f7497804cc096027b14b64f95b7
I20260709 12:29:24.227017 15958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87b688925a5c41e1b41d0a4d8652e834"
dest_uuid: "f742bdd975884e759c43e1311e22ac23"
mode: GRACEFUL
new_leader_uuid: "f8085f7497804cc096027b14b64f95b7"
 from {username='slave'} at 127.0.0.1:33802
I20260709 12:29:24.227694 15958 raft_consensus.cc:606] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Received request to transfer leadership to TS f8085f7497804cc096027b14b64f95b7
I20260709 12:29:24.234486 15958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a65d88797ab489db65de91babc2828f"
dest_uuid: "f742bdd975884e759c43e1311e22ac23"
mode: GRACEFUL
new_leader_uuid: "f8085f7497804cc096027b14b64f95b7"
 from {username='slave'} at 127.0.0.1:33802
I20260709 12:29:24.235112 15958 raft_consensus.cc:606] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Received request to transfer leadership to TS f8085f7497804cc096027b14b64f95b7
I20260709 12:29:24.240690 16220 consensus_queue.cc:1048] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:24.246768 16033 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3630e85b83b04193a66ec5acd5611eeb"
dest_uuid: "f8085f7497804cc096027b14b64f95b7"
mode: GRACEFUL
new_leader_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
 from {username='slave'} at 127.0.0.1:43196
I20260709 12:29:24.247536 16033 raft_consensus.cc:606] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Received request to transfer leadership to TS cd106f11cb4c43f19e686df9f9a365f9
I20260709 12:29:24.257239 16175 consensus_queue.cc:1048] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:24.313318 16204 consensus_queue.cc:1048] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:24.327226 16154 consensus_queue.cc:1048] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:24.336127 16173 raft_consensus.cc:993] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7: : Instructing follower f742bdd975884e759c43e1311e22ac23 to start an election
I20260709 12:29:24.336632 16154 raft_consensus.cc:1081] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Signalling peer f742bdd975884e759c43e1311e22ac23 to start an election
I20260709 12:29:24.338305 15958 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85fc54f820da47d0951fabd0d60156d0"
dest_uuid: "f742bdd975884e759c43e1311e22ac23"
 from {username='slave'} at 127.0.0.1:33778
I20260709 12:29:24.338899 15958 raft_consensus.cc:493] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:24.339206 15958 raft_consensus.cc:3060] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.344040 16154 consensus_queue.cc:1048] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:24.347404 15958 raft_consensus.cc:515] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.351356 16033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fc54f820da47d0951fabd0d60156d0" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:24.351666 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fc54f820da47d0951fabd0d60156d0" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:24.352348 16107 raft_consensus.cc:3060] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.358355 16033 raft_consensus.cc:3055] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:24.358801 16033 raft_consensus.cc:740] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:24.360463 16200 consensus_queue.cc:1048] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:24.364830 16033 consensus_queue.cc:260] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.366487 16033 raft_consensus.cc:3060] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.351322 15958 leader_election.cc:290] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 2 election: Requested vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:24.373476 16107 raft_consensus.cc:2468] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 2.
I20260709 12:29:24.375116 15923 leader_election.cc:304] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
I20260709 12:29:24.376152 16175 raft_consensus.cc:2804] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:24.376747 16175 raft_consensus.cc:697] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 2 LEADER]: Becoming Leader. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Running, Role: LEADER
I20260709 12:29:24.377580 16033 raft_consensus.cc:2468] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 2.
I20260709 12:29:24.377774 16175 consensus_queue.cc:237] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.387904 15856 catalog_manager.cc:5697] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 reported cstate change: term changed from 1 to 2, leader changed from f8085f7497804cc096027b14b64f95b7 (127.2.4.194) to f742bdd975884e759c43e1311e22ac23 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "f742bdd975884e759c43e1311e22ac23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:24.424783 16154 raft_consensus.cc:993] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7: : Instructing follower f742bdd975884e759c43e1311e22ac23 to start an election
I20260709 12:29:24.425395 16200 raft_consensus.cc:1081] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Signalling peer f742bdd975884e759c43e1311e22ac23 to start an election
I20260709 12:29:24.427281 15957 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dfdf24842682422dab37b9ccc8ab0c24"
dest_uuid: "f742bdd975884e759c43e1311e22ac23"
 from {username='slave'} at 127.0.0.1:33778
I20260709 12:29:24.427922 15957 raft_consensus.cc:493] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:24.428288 15957 raft_consensus.cc:3060] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.435132 15957 raft_consensus.cc:515] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.437047 15957 leader_election.cc:290] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 2 election: Requested vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:24.438373 16033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfdf24842682422dab37b9ccc8ab0c24" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:24.438437 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfdf24842682422dab37b9ccc8ab0c24" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:24.439491 16033 raft_consensus.cc:3055] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:24.439510 16107 raft_consensus.cc:3060] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.440111 16033 raft_consensus.cc:740] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:24.441123 16033 consensus_queue.cc:260] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.442655 16033 raft_consensus.cc:3060] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.449039 16107 raft_consensus.cc:2468] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 2.
I20260709 12:29:24.450933 15923 leader_election.cc:304] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
I20260709 12:29:24.451112 16033 raft_consensus.cc:2468] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 2.
I20260709 12:29:24.452123 16175 raft_consensus.cc:2804] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:24.452726 16175 raft_consensus.cc:697] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 2 LEADER]: Becoming Leader. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Running, Role: LEADER
I20260709 12:29:24.453732 16175 consensus_queue.cc:237] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.461182 15856 catalog_manager.cc:5697] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 reported cstate change: term changed from 1 to 2, leader changed from f8085f7497804cc096027b14b64f95b7 (127.2.4.194) to f742bdd975884e759c43e1311e22ac23 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "f742bdd975884e759c43e1311e22ac23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:24.489715 16200 consensus_queue.cc:1048] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:24.502558 16155 consensus_queue.cc:1048] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:24.504492 16200 consensus_queue.cc:1048] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:24.518841 16200 raft_consensus.cc:993] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7: : Instructing follower cd106f11cb4c43f19e686df9f9a365f9 to start an election
I20260709 12:29:24.519430 16204 raft_consensus.cc:1081] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Signalling peer cd106f11cb4c43f19e686df9f9a365f9 to start an election
I20260709 12:29:24.524063 16155 consensus_queue.cc:1048] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:24.524906 16107 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3630e85b83b04193a66ec5acd5611eeb"
dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
 from {username='slave'} at 127.0.0.1:48616
I20260709 12:29:24.525813 16107 raft_consensus.cc:493] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:24.526306 16107 raft_consensus.cc:3060] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.537106 16234 raft_consensus.cc:993] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9: : Instructing follower f8085f7497804cc096027b14b64f95b7 to start an election
I20260709 12:29:24.537045 16107 raft_consensus.cc:515] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.537873 16234 raft_consensus.cc:1081] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 LEADER]: Signalling peer f8085f7497804cc096027b14b64f95b7 to start an election
I20260709 12:29:24.539492 16107 leader_election.cc:290] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 2 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:24.541352 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3630e85b83b04193a66ec5acd5611eeb" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:24.542224 15958 raft_consensus.cc:3060] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.542176 16033 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72091658e1c047c08f409622ac3910b3"
dest_uuid: "f8085f7497804cc096027b14b64f95b7"
 from {username='slave'} at 127.0.0.1:43192
I20260709 12:29:24.543380 16033 raft_consensus.cc:493] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:24.543838 16033 raft_consensus.cc:3060] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.541589 16032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3630e85b83b04193a66ec5acd5611eeb" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:24.546204 16032 raft_consensus.cc:3055] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:24.546605 16032 raft_consensus.cc:740] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:24.547578 16032 consensus_queue.cc:260] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.549264 16032 raft_consensus.cc:3060] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.552986 15958 raft_consensus.cc:2468] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 2.
I20260709 12:29:24.553493 16033 raft_consensus.cc:515] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.555066 16070 leader_election.cc:304] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
I20260709 12:29:24.556154 16234 raft_consensus.cc:2804] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:24.556708 16234 raft_consensus.cc:697] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 2 LEADER]: Becoming Leader. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Running, Role: LEADER
I20260709 12:29:24.557328 16033 leader_election.cc:290] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 2 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:24.557904 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72091658e1c047c08f409622ac3910b3" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:24.557719 16234 consensus_queue.cc:237] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.558779 15958 raft_consensus.cc:3060] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.559453 16032 raft_consensus.cc:2468] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 2.
I20260709 12:29:24.559813 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72091658e1c047c08f409622ac3910b3" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:24.560487 16107 raft_consensus.cc:3055] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:24.560827 16107 raft_consensus.cc:740] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Running, Role: LEADER
I20260709 12:29:24.561743 16107 consensus_queue.cc:260] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.563710 16107 raft_consensus.cc:3060] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.565841 16175 consensus_queue.cc:1048] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:24.569233 15958 raft_consensus.cc:2468] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 2.
I20260709 12:29:24.568588 15856 catalog_manager.cc:5697] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 reported cstate change: term changed from 1 to 2, leader changed from f8085f7497804cc096027b14b64f95b7 (127.2.4.194) to cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195). New cstate: current_term: 2 leader_uuid: "cd106f11cb4c43f19e686df9f9a365f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:24.571007 15996 leader_election.cc:304] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:24.572418 16204 raft_consensus.cc:2804] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:24.572957 16204 raft_consensus.cc:697] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 2 LEADER]: Becoming Leader. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:24.574965 16107 raft_consensus.cc:2468] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 2.
I20260709 12:29:24.577960 16204 consensus_queue.cc:237] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.583545 16152 raft_consensus.cc:993] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23: : Instructing follower f8085f7497804cc096027b14b64f95b7 to start an election
I20260709 12:29:24.583609 16195 raft_consensus.cc:993] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23: : Instructing follower f8085f7497804cc096027b14b64f95b7 to start an election
I20260709 12:29:24.584076 16152 raft_consensus.cc:1081] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Signalling peer f8085f7497804cc096027b14b64f95b7 to start an election
I20260709 12:29:24.584270 16195 raft_consensus.cc:1081] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Signalling peer f8085f7497804cc096027b14b64f95b7 to start an election
I20260709 12:29:24.586305 16033 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a65d88797ab489db65de91babc2828f"
dest_uuid: "f8085f7497804cc096027b14b64f95b7"
 from {username='slave'} at 127.0.0.1:43188
I20260709 12:29:24.586766 16031 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87b688925a5c41e1b41d0a4d8652e834"
dest_uuid: "f8085f7497804cc096027b14b64f95b7"
 from {username='slave'} at 127.0.0.1:43188
I20260709 12:29:24.586964 16033 raft_consensus.cc:493] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:24.587517 16033 raft_consensus.cc:3060] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.587548 16031 raft_consensus.cc:493] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:24.588150 16031 raft_consensus.cc:3060] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.589686 15856 catalog_manager.cc:5697] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 reported cstate change: term changed from 1 to 2, leader changed from cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195) to f8085f7497804cc096027b14b64f95b7 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "f8085f7497804cc096027b14b64f95b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:24.595585 16175 consensus_queue.cc:1048] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:24.596632 16031 raft_consensus.cc:515] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.600436 16031 leader_election.cc:290] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 2 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:24.601824 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87b688925a5c41e1b41d0a4d8652e834" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:24.603660 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87b688925a5c41e1b41d0a4d8652e834" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:24.602124 16032 raft_consensus.cc:1240] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Rejecting Update request from peer f742bdd975884e759c43e1311e22ac23 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:24.604478 15958 raft_consensus.cc:3055] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:24.605233 15958 raft_consensus.cc:740] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Running, Role: LEADER
I20260709 12:29:24.599553 16033 raft_consensus.cc:515] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.606416 15958 consensus_queue.cc:260] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.607925 15958 raft_consensus.cc:3060] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.609026 16033 leader_election.cc:290] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 2 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:24.611084 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a65d88797ab489db65de91babc2828f" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:24.612056 16107 raft_consensus.cc:3060] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.612421 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a65d88797ab489db65de91babc2828f" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:24.613131 15957 raft_consensus.cc:3055] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:24.613466 15957 raft_consensus.cc:740] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Running, Role: LEADER
I20260709 12:29:24.614367 15957 consensus_queue.cc:260] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.615717 15957 raft_consensus.cc:3060] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.621138 16107 raft_consensus.cc:2468] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 2.
I20260709 12:29:24.622342 15958 raft_consensus.cc:2468] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 2.
I20260709 12:29:24.622757 15998 leader_election.cc:304] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:24.623857 16204 raft_consensus.cc:2804] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:24.624128 15996 leader_election.cc:304] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: cd106f11cb4c43f19e686df9f9a365f9
I20260709 12:29:24.624147 15957 raft_consensus.cc:2468] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 2.
I20260709 12:29:24.624402 16204 raft_consensus.cc:697] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [term 2 LEADER]: Becoming Leader. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:24.625479 16204 consensus_queue.cc:237] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.625906 16200 raft_consensus.cc:2804] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:24.626551 16200 raft_consensus.cc:697] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [term 2 LEADER]: Becoming Leader. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:24.627524 16200 consensus_queue.cc:237] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.634069 15854 catalog_manager.cc:5697] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 reported cstate change: term changed from 1 to 2, leader changed from f742bdd975884e759c43e1311e22ac23 (127.2.4.193) to f8085f7497804cc096027b14b64f95b7 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "f8085f7497804cc096027b14b64f95b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:24.649246 15854 catalog_manager.cc:5697] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 reported cstate change: term changed from 1 to 2, leader changed from f742bdd975884e759c43e1311e22ac23 (127.2.4.193) to f8085f7497804cc096027b14b64f95b7 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "f8085f7497804cc096027b14b64f95b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:24.720885 16173 raft_consensus.cc:993] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7: : Instructing follower f742bdd975884e759c43e1311e22ac23 to start an election
I20260709 12:29:24.721503 16200 raft_consensus.cc:1081] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Signalling peer f742bdd975884e759c43e1311e22ac23 to start an election
I20260709 12:29:24.723575 15957 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d37749a2508249b2afaf4467cc56d190"
dest_uuid: "f742bdd975884e759c43e1311e22ac23"
 from {username='slave'} at 127.0.0.1:33778
I20260709 12:29:24.724229 15957 raft_consensus.cc:493] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:24.724629 15957 raft_consensus.cc:3060] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.732084 15957 raft_consensus.cc:515] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.734426 15957 leader_election.cc:290] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 2 election: Requested vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:24.735579 16033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37749a2508249b2afaf4467cc56d190" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:24.736222 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37749a2508249b2afaf4467cc56d190" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:24.736567 16033 raft_consensus.cc:3055] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:24.736905 16107 raft_consensus.cc:3060] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.737038 16033 raft_consensus.cc:740] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:24.738190 16033 consensus_queue.cc:260] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.739719 16033 raft_consensus.cc:3060] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.746183 16107 raft_consensus.cc:2468] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 2.
I20260709 12:29:24.748081 15923 leader_election.cc:304] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
I20260709 12:29:24.748549 16033 raft_consensus.cc:2468] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 2.
I20260709 12:29:24.749361 16175 raft_consensus.cc:2804] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:24.750022 16175 raft_consensus.cc:697] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 2 LEADER]: Becoming Leader. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Running, Role: LEADER
I20260709 12:29:24.750918 16175 consensus_queue.cc:237] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:24.760152 15854 catalog_manager.cc:5697] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 reported cstate change: term changed from 1 to 2, leader changed from f8085f7497804cc096027b14b64f95b7 (127.2.4.194) to f742bdd975884e759c43e1311e22ac23 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "f742bdd975884e759c43e1311e22ac23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:24.761286 16107 raft_consensus.cc:1275] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Refusing update from remote peer f742bdd975884e759c43e1311e22ac23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:24.763324 16195 consensus_queue.cc:1048] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:24.774048 16033 raft_consensus.cc:1275] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Refusing update from remote peer f742bdd975884e759c43e1311e22ac23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:24.776253 16152 consensus_queue.cc:1048] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:24.840209 16033 raft_consensus.cc:1275] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Refusing update from remote peer f742bdd975884e759c43e1311e22ac23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:24.842067 16152 consensus_queue.cc:1048] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:24.846241 16107 raft_consensus.cc:1275] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Refusing update from remote peer f742bdd975884e759c43e1311e22ac23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:24.848140 16220 consensus_queue.cc:1048] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:25.051595 15957 raft_consensus.cc:1275] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Refusing update from remote peer f8085f7497804cc096027b14b64f95b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:25.054153 16173 consensus_queue.cc:1048] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:25.058631 15957 raft_consensus.cc:1275] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Refusing update from remote peer cd106f11cb4c43f19e686df9f9a365f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:25.061138 16234 consensus_queue.cc:1048] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:25.066061 16107 raft_consensus.cc:1275] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Refusing update from remote peer f8085f7497804cc096027b14b64f95b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:25.068305 16200 consensus_queue.cc:1048] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:25.086591 16033 raft_consensus.cc:1275] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Refusing update from remote peer cd106f11cb4c43f19e686df9f9a365f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:25.088406 16239 consensus_queue.cc:1048] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:25.119544 15957 raft_consensus.cc:1275] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Refusing update from remote peer f8085f7497804cc096027b14b64f95b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:25.121106 16200 consensus_queue.cc:1048] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:25.131834 16107 raft_consensus.cc:1275] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Refusing update from remote peer f8085f7497804cc096027b14b64f95b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:25.133595 16200 consensus_queue.cc:1048] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:25.168098 15957 raft_consensus.cc:1275] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Refusing update from remote peer f8085f7497804cc096027b14b64f95b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:25.169840 16200 consensus_queue.cc:1048] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:25.179633 16107 raft_consensus.cc:3060] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.190294 16107 raft_consensus.cc:1275] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Refusing update from remote peer f8085f7497804cc096027b14b64f95b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:25.192067 16200 consensus_queue.cc:1048] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:25.290617 16032 raft_consensus.cc:1275] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Refusing update from remote peer f742bdd975884e759c43e1311e22ac23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:25.291918 16220 consensus_queue.cc:1048] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:25.302569 16107 raft_consensus.cc:1275] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Refusing update from remote peer f742bdd975884e759c43e1311e22ac23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:25.304155 16152 consensus_queue.cc:1048] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:27.280390 15854 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36370:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:27.283715 15854 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:27.346068 15946 tablet_service.cc:1511] Processing CreateTablet for tablet abfa827185b748668746d469553068cb (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:27.347846 15946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abfa827185b748668746d469553068cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.350466 15940 tablet_service.cc:1511] Processing CreateTablet for tablet f2ae049343d64dad9f168e68d13b4687 (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:27.347442 16020 tablet_service.cc:1511] Processing CreateTablet for tablet abfa827185b748668746d469553068cb (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:27.352035 16020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abfa827185b748668746d469553068cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.351666 16017 tablet_service.cc:1511] Processing CreateTablet for tablet f2ae049343d64dad9f168e68d13b4687 (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:27.349061 16097 tablet_service.cc:1511] Processing CreateTablet for tablet abfa827185b748668746d469553068cb (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:27.353355 16017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2ae049343d64dad9f168e68d13b4687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.354166 16097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abfa827185b748668746d469553068cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.354568 16094 tablet_service.cc:1511] Processing CreateTablet for tablet f2ae049343d64dad9f168e68d13b4687 (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:27.355934 16094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2ae049343d64dad9f168e68d13b4687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.356267 16022 tablet_service.cc:1511] Processing CreateTablet for tablet 2ee293a1e4364c2e8c2d72fa2c0c6ab9 (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:27.359097 15944 tablet_service.cc:1511] Processing CreateTablet for tablet 2ee293a1e4364c2e8c2d72fa2c0c6ab9 (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:27.359258 16090 tablet_service.cc:1511] Processing CreateTablet for tablet 2ee293a1e4364c2e8c2d72fa2c0c6ab9 (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:27.360716 16090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee293a1e4364c2e8c2d72fa2c0c6ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.362638 16018 tablet_service.cc:1511] Processing CreateTablet for tablet e4daed21d1e44c07a6b92afea49cbb6d (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:27.365403 15945 tablet_service.cc:1511] Processing CreateTablet for tablet e4daed21d1e44c07a6b92afea49cbb6d (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:27.366400 16091 tablet_service.cc:1511] Processing CreateTablet for tablet fa801227038b4ca9a3bb2a634b0d39ef (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:27.366822 16093 tablet_service.cc:1511] Processing CreateTablet for tablet e4daed21d1e44c07a6b92afea49cbb6d (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:27.367802 16091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa801227038b4ca9a3bb2a634b0d39ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.352037 15940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2ae049343d64dad9f168e68d13b4687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.370224 15945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4daed21d1e44c07a6b92afea49cbb6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.370481 16093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4daed21d1e44c07a6b92afea49cbb6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.371104 15944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee293a1e4364c2e8c2d72fa2c0c6ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.371690 16018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4daed21d1e44c07a6b92afea49cbb6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.378119 16022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee293a1e4364c2e8c2d72fa2c0c6ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.382556 16096 tablet_service.cc:1511] Processing CreateTablet for tablet 9d60dd6f617e4b6ca66f0573bb2aaa82 (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:27.384019 16096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d60dd6f617e4b6ca66f0573bb2aaa82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.388561 16021 tablet_service.cc:1511] Processing CreateTablet for tablet fa801227038b4ca9a3bb2a634b0d39ef (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:27.388808 16015 tablet_service.cc:1511] Processing CreateTablet for tablet 9d60dd6f617e4b6ca66f0573bb2aaa82 (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:27.391397 15941 tablet_service.cc:1511] Processing CreateTablet for tablet fa801227038b4ca9a3bb2a634b0d39ef (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:27.391597 15943 tablet_service.cc:1511] Processing CreateTablet for tablet 9d60dd6f617e4b6ca66f0573bb2aaa82 (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:27.393199 16021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa801227038b4ca9a3bb2a634b0d39ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.395578 16092 tablet_service.cc:1511] Processing CreateTablet for tablet 9c88e98f796f438f82b5615a926aa4c1 (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:27.396955 16092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c88e98f796f438f82b5615a926aa4c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.400420 16095 tablet_service.cc:1511] Processing CreateTablet for tablet 53c13ce3b5a040a394b6cb64be5ec33a (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:27.401913 16095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c13ce3b5a040a394b6cb64be5ec33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.402321 16015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d60dd6f617e4b6ca66f0573bb2aaa82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.406279 15941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa801227038b4ca9a3bb2a634b0d39ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.410207 15943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d60dd6f617e4b6ca66f0573bb2aaa82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.419695 16016 tablet_service.cc:1511] Processing CreateTablet for tablet 9c88e98f796f438f82b5615a926aa4c1 (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:27.427870 16019 tablet_service.cc:1511] Processing CreateTablet for tablet 53c13ce3b5a040a394b6cb64be5ec33a (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:27.421528 16016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c88e98f796f438f82b5615a926aa4c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.432279 16019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c13ce3b5a040a394b6cb64be5ec33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.437304 15942 tablet_service.cc:1511] Processing CreateTablet for tablet 9c88e98f796f438f82b5615a926aa4c1 (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:27.439539 15942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c88e98f796f438f82b5615a926aa4c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.438946 15948 tablet_service.cc:1511] Processing CreateTablet for tablet 53c13ce3b5a040a394b6cb64be5ec33a (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:27.442054 15947 tablet_service.cc:1511] Processing CreateTablet for tablet b6798d36196f40d882bab16f264150ed (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:27.442684 15948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c13ce3b5a040a394b6cb64be5ec33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.446511 15947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6798d36196f40d882bab16f264150ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.494917 16258 tablet_bootstrap.cc:492] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:27.503965 16256 tablet_bootstrap.cc:492] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:27.516028 16258 tablet_bootstrap.cc:654] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.523533 16257 tablet_bootstrap.cc:492] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:27.525205 16020 tablet_service.cc:1511] Processing CreateTablet for tablet b6798d36196f40d882bab16f264150ed (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:27.527132 16020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6798d36196f40d882bab16f264150ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.531291 16256 tablet_bootstrap.cc:654] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.547629 16257 tablet_bootstrap.cc:654] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.547586 16094 tablet_service.cc:1511] Processing CreateTablet for tablet b6798d36196f40d882bab16f264150ed (DEFAULT_TABLE table=soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:27.558382 16094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6798d36196f40d882bab16f264150ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:27.576453 16258 tablet_bootstrap.cc:492] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:27.577307 16258 ts_tablet_manager.cc:1403] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.083s	user 0.012s	sys 0.017s
I20260709 12:29:27.580667 16258 raft_consensus.cc:359] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.581493 16258 raft_consensus.cc:385] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.581882 16258 raft_consensus.cc:740] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.582858 16258 consensus_queue.cc:260] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.620854 16258 ts_tablet_manager.cc:1434] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 12:29:27.622107 16256 tablet_bootstrap.cc:492] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:27.622212 16258 tablet_bootstrap.cc:492] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:27.622859 16256 ts_tablet_manager.cc:1403] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.119s	user 0.015s	sys 0.000s
I20260709 12:29:27.629792 16258 tablet_bootstrap.cc:654] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.629993 16256 raft_consensus.cc:359] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.631127 16256 raft_consensus.cc:385] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.631464 16256 raft_consensus.cc:740] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.632417 16256 consensus_queue.cc:260] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.636241 16257 tablet_bootstrap.cc:492] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:27.636896 16257 ts_tablet_manager.cc:1403] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.114s	user 0.007s	sys 0.009s
I20260709 12:29:27.640416 16257 raft_consensus.cc:359] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.641228 16257 raft_consensus.cc:385] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.641510 16256 ts_tablet_manager.cc:1434] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.003s
I20260709 12:29:27.641510 16257 raft_consensus.cc:740] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.644239 16257 consensus_queue.cc:260] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.646093 16256 tablet_bootstrap.cc:492] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:27.655669 16257 ts_tablet_manager.cc:1434] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.018s	user 0.002s	sys 0.008s
I20260709 12:29:27.657331 16173 raft_consensus.cc:493] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.658231 16257 tablet_bootstrap.cc:492] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:27.657902 16173 raft_consensus.cc:515] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.665560 16257 tablet_bootstrap.cc:654] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.667872 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abfa827185b748668746d469553068cb" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:27.669459 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abfa827185b748668746d469553068cb" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
W20260709 12:29:27.671229 15998 leader_election.cc:343] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:27.672631 15996 leader_election.cc:343] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.673067 15996 leader_election.cc:304] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8085f7497804cc096027b14b64f95b7; no voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23
I20260709 12:29:27.665479 16173 leader_election.cc:290] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:27.675697 16173 raft_consensus.cc:2749] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:27.683162 16258 tablet_bootstrap.cc:492] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:27.683830 16256 tablet_bootstrap.cc:654] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.686195 16258 ts_tablet_manager.cc:1403] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.064s	user 0.026s	sys 0.012s
I20260709 12:29:27.690356 16258 raft_consensus.cc:359] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.693465 16257 tablet_bootstrap.cc:492] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:27.693761 16258 raft_consensus.cc:385] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.694144 16258 raft_consensus.cc:740] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.694499 16257 ts_tablet_manager.cc:1403] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.000s
I20260709 12:29:27.695075 16258 consensus_queue.cc:260] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.697867 16258 ts_tablet_manager.cc:1434] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:29:27.698378 16257 raft_consensus.cc:359] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.699019 16258 tablet_bootstrap.cc:492] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:27.699191 16257 raft_consensus.cc:385] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.699528 16257 raft_consensus.cc:740] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.700379 16257 consensus_queue.cc:260] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.702919 16257 ts_tablet_manager.cc:1434] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:27.706606 16257 tablet_bootstrap.cc:492] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:27.710335 16258 tablet_bootstrap.cc:654] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.712159 16256 tablet_bootstrap.cc:492] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:27.712891 16256 ts_tablet_manager.cc:1403] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.067s	user 0.016s	sys 0.008s
I20260709 12:29:27.714506 16257 tablet_bootstrap.cc:654] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.716732 16256 raft_consensus.cc:359] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.717610 16256 raft_consensus.cc:385] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.717943 16256 raft_consensus.cc:740] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.718786 16256 consensus_queue.cc:260] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.722901 16256 ts_tablet_manager.cc:1434] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 12:29:27.724365 16256 tablet_bootstrap.cc:492] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:27.725682 16258 tablet_bootstrap.cc:492] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:27.726331 16258 ts_tablet_manager.cc:1403] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.007s
I20260709 12:29:27.729599 16258 raft_consensus.cc:359] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.730396 16258 raft_consensus.cc:385] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.730690 16258 raft_consensus.cc:740] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.731396 16258 consensus_queue.cc:260] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.732039 16256 tablet_bootstrap.cc:654] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.735112 16258 ts_tablet_manager.cc:1434] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:29:27.737327 16258 tablet_bootstrap.cc:492] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:27.744139 16258 tablet_bootstrap.cc:654] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.748879 16257 tablet_bootstrap.cc:492] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:27.749799 16257 ts_tablet_manager.cc:1403] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.001s
I20260709 12:29:27.754914 16257 raft_consensus.cc:359] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.756026 16257 raft_consensus.cc:385] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.756251 16258 tablet_bootstrap.cc:492] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:27.756414 16257 raft_consensus.cc:740] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.756884 16258 ts_tablet_manager.cc:1403] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.003s
I20260709 12:29:27.757895 16257 consensus_queue.cc:260] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.761399 16258 raft_consensus.cc:359] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.761571 16257 ts_tablet_manager.cc:1434] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:29:27.762315 16258 raft_consensus.cc:385] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.763780 16258 raft_consensus.cc:740] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.764467 16257 tablet_bootstrap.cc:492] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:27.764889 16256 tablet_bootstrap.cc:492] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:27.764599 16258 consensus_queue.cc:260] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.765497 16256 ts_tablet_manager.cc:1403] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.041s	user 0.019s	sys 0.019s
I20260709 12:29:27.770917 16258 ts_tablet_manager.cc:1434] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 12:29:27.772562 16258 tablet_bootstrap.cc:492] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:27.774530 16257 tablet_bootstrap.cc:654] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.769275 16256 raft_consensus.cc:359] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.777354 16256 raft_consensus.cc:385] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.778031 16256 raft_consensus.cc:740] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.779109 16256 consensus_queue.cc:260] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.781127 16258 tablet_bootstrap.cc:654] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.781679 16256 ts_tablet_manager.cc:1434] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:29:27.782934 16256 tablet_bootstrap.cc:492] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:27.790007 16258 tablet_bootstrap.cc:492] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:27.790504 16256 tablet_bootstrap.cc:654] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.790747 16258 ts_tablet_manager.cc:1403] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 12:29:27.794281 16258 raft_consensus.cc:359] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.794889 16258 raft_consensus.cc:385] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.795145 16258 raft_consensus.cc:740] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.795817 16258 consensus_queue.cc:260] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.801965 16257 tablet_bootstrap.cc:492] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:27.802968 16257 ts_tablet_manager.cc:1403] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.000s
I20260709 12:29:27.798454 16258 ts_tablet_manager.cc:1434] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:27.807744 16257 raft_consensus.cc:359] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.808898 16257 raft_consensus.cc:385] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.809324 16257 raft_consensus.cc:740] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.811046 16257 consensus_queue.cc:260] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.811962 16258 tablet_bootstrap.cc:492] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:27.813884 16257 ts_tablet_manager.cc:1434] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
I20260709 12:29:27.815274 16257 tablet_bootstrap.cc:492] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:27.813177 16256 tablet_bootstrap.cc:492] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:27.818475 16256 ts_tablet_manager.cc:1403] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.019s
I20260709 12:29:27.821578 16258 tablet_bootstrap.cc:654] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.822556 16257 tablet_bootstrap.cc:654] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.822213 16256 raft_consensus.cc:359] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.823670 16256 raft_consensus.cc:385] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.824014 16256 raft_consensus.cc:740] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.824837 16256 consensus_queue.cc:260] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.830391 16256 ts_tablet_manager.cc:1434] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:29:27.831574 16256 tablet_bootstrap.cc:492] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:27.840137 16256 tablet_bootstrap.cc:654] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.851642 16258 tablet_bootstrap.cc:492] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:27.852317 16258 ts_tablet_manager.cc:1403] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.041s	user 0.028s	sys 0.009s
I20260709 12:29:27.852779 16256 tablet_bootstrap.cc:492] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:27.853530 16256 ts_tablet_manager.cc:1403] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.002s
I20260709 12:29:27.854745 16257 tablet_bootstrap.cc:492] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:27.855373 16257 ts_tablet_manager.cc:1403] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.040s	user 0.022s	sys 0.012s
I20260709 12:29:27.856081 16258 raft_consensus.cc:359] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.857081 16258 raft_consensus.cc:385] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.857085 16256 raft_consensus.cc:359] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.857542 16258 raft_consensus.cc:740] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.858048 16256 raft_consensus.cc:385] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.858420 16257 raft_consensus.cc:359] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.858398 16256 raft_consensus.cc:740] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.859335 16257 raft_consensus.cc:385] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.858704 16258 consensus_queue.cc:260] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.859678 16257 raft_consensus.cc:740] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.859876 16256 consensus_queue.cc:260] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.860590 16257 consensus_queue.cc:260] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.861866 16258 ts_tablet_manager.cc:1434] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 12:29:27.863055 16258 tablet_bootstrap.cc:492] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:27.863204 16257 ts_tablet_manager.cc:1434] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:29:27.864428 16257 tablet_bootstrap.cc:492] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:27.866605 16256 ts_tablet_manager.cc:1434] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.003s
I20260709 12:29:27.868053 16256 tablet_bootstrap.cc:492] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:27.870918 16258 tablet_bootstrap.cc:654] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.871930 16257 tablet_bootstrap.cc:654] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.876186 16256 tablet_bootstrap.cc:654] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.878510 16173 raft_consensus.cc:493] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.879319 16173 raft_consensus.cc:515] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.871886 16239 raft_consensus.cc:493] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.885329 16257 tablet_bootstrap.cc:492] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:27.885982 16257 ts_tablet_manager.cc:1403] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.004s
I20260709 12:29:27.886448 16258 tablet_bootstrap.cc:492] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:27.887333 16258 ts_tablet_manager.cc:1403] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.001s
I20260709 12:29:27.886132 16239 raft_consensus.cc:515] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.889303 16257 raft_consensus.cc:359] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.890172 16257 raft_consensus.cc:385] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.890455 16257 raft_consensus.cc:740] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.890672 16256 tablet_bootstrap.cc:492] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:27.891477 16256 ts_tablet_manager.cc:1403] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.007s
I20260709 12:29:27.891649 16257 consensus_queue.cc:260] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.893054 16032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ae049343d64dad9f168e68d13b4687" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:27.893590 16258 raft_consensus.cc:359] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.895295 16256 raft_consensus.cc:359] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.896147 16256 raft_consensus.cc:385] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.895131 16258 raft_consensus.cc:385] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.896219 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ae049343d64dad9f168e68d13b4687" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:27.897230 16256 raft_consensus.cc:740] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.896960 16258 raft_consensus.cc:740] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.898928 16256 consensus_queue.cc:260] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.899276 16258 consensus_queue.cc:260] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.902639 16256 ts_tablet_manager.cc:1434] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.005s
I20260709 12:29:27.902995 16258 ts_tablet_manager.cc:1434] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.000s
I20260709 12:29:27.904136 16256 tablet_bootstrap.cc:492] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:27.904268 16258 tablet_bootstrap.cc:492] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:27.894721 16032 raft_consensus.cc:2468] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 0.
I20260709 12:29:27.913443 16256 tablet_bootstrap.cc:654] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.913986 16258 tablet_bootstrap.cc:654] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.915442 15957 raft_consensus.cc:2468] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 0.
I20260709 12:29:27.924825 16283 raft_consensus.cc:493] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.925463 16283 raft_consensus.cc:515] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.928429 16283 leader_election.cc:290] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:27.928617 16258 tablet_bootstrap.cc:492] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:27.928893 16283 raft_consensus.cc:493] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.929311 16258 ts_tablet_manager.cc:1403] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.006s
I20260709 12:29:27.929492 16283 raft_consensus.cc:515] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.930805 16173 leader_election.cc:290] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:27.931767 16283 leader_election.cc:290] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:27.932719 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:27.932631 16258 raft_consensus.cc:359] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.933773 16258 raft_consensus.cc:385] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.934188 16258 raft_consensus.cc:740] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.934970 16258 consensus_queue.cc:260] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
W20260709 12:29:27.935688 15996 leader_election.cc:343] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.936993 16256 tablet_bootstrap.cc:492] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:27.937865 16256 ts_tablet_manager.cc:1403] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.034s	user 0.021s	sys 0.004s
I20260709 12:29:27.941550 16256 raft_consensus.cc:359] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.942425 16256 raft_consensus.cc:385] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:27.942727 16256 raft_consensus.cc:740] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:27.943584 16256 consensus_queue.cc:260] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.946738 16256 ts_tablet_manager.cc:1434] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:29:27.937496 16258 ts_tablet_manager.cc:1434] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:29:27.950075 16256 tablet_bootstrap.cc:492] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:27.949855 16264 raft_consensus.cc:493] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.950239 16152 raft_consensus.cc:493] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.950651 16264 raft_consensus.cc:515] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.951496 16152 raft_consensus.cc:515] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.954890 16264 leader_election.cc:290] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:27.958700 16256 tablet_bootstrap.cc:654] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:27.893767 16239 leader_election.cc:290] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:27.966279 16258 tablet_bootstrap.cc:492] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7: Bootstrap starting.
I20260709 12:29:27.971731 16152 leader_election.cc:290] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:27.973470 15956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4daed21d1e44c07a6b92afea49cbb6d" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:27.974488 16033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4daed21d1e44c07a6b92afea49cbb6d" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:27.973470 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d60dd6f617e4b6ca66f0573bb2aaa82" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:27.975281 16033 raft_consensus.cc:2468] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 0.
I20260709 12:29:27.975481 15958 raft_consensus.cc:2468] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 0.
I20260709 12:29:27.977352 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88e98f796f438f82b5615a926aa4c1" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:27.977483 15996 leader_election.cc:304] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
W20260709 12:29:27.979178 15923 leader_election.cc:343] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:27.980831 16032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88e98f796f438f82b5615a926aa4c1" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:27.984791 16032 raft_consensus.cc:2468] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 0.
I20260709 12:29:27.984963 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:27.982721 16264 raft_consensus.cc:2804] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:27.990100 16264 raft_consensus.cc:493] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:27.990651 16264 raft_consensus.cc:3060] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:27.992642 16256 tablet_bootstrap.cc:492] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:27.993583 16256 ts_tablet_manager.cc:1403] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.012s
I20260709 12:29:27.994518 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d60dd6f617e4b6ca66f0573bb2aaa82" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:27.990414 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:27.996002 16070 leader_election.cc:304] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
I20260709 12:29:27.996246 16107 raft_consensus.cc:2468] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 0.
I20260709 12:29:27.998340 16256 raft_consensus.cc:359] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:27.999161 16152 raft_consensus.cc:493] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:27.999683 16256 raft_consensus.cc:385] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:28.000196 16256 raft_consensus.cc:740] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.006197 16152 raft_consensus.cc:515] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.008668 16152 leader_election.cc:290] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:28.010648 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee293a1e4364c2e8c2d72fa2c0c6ab9" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:28.018153 16107 raft_consensus.cc:2468] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 0.
I20260709 12:29:28.010459 16239 raft_consensus.cc:2804] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:28.019361 16239 raft_consensus.cc:493] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:28.019798 16239 raft_consensus.cc:3060] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.019694 15923 leader_election.cc:304] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
I20260709 12:29:28.011129 16152 raft_consensus.cc:493] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:28.021823 16152 raft_consensus.cc:515] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.024061 16286 raft_consensus.cc:2804] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:28.024938 16286 raft_consensus.cc:493] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:28.025537 16286 raft_consensus.cc:3060] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.026197 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6798d36196f40d882bab16f264150ed" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
W20260709 12:29:28.028046 15923 leader_election.cc:343] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:28.024205 16152 leader_election.cc:290] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:28.013967 15921 leader_election.cc:304] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: cd106f11cb4c43f19e686df9f9a365f9
I20260709 12:29:28.029724 16239 raft_consensus.cc:515] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.031760 16152 raft_consensus.cc:2804] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:28.032248 16152 raft_consensus.cc:493] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:28.032586 16152 raft_consensus.cc:3060] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.032627 16031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6798d36196f40d882bab16f264150ed" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:28.014487 16033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee293a1e4364c2e8c2d72fa2c0c6ab9" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:28.034953 16033 raft_consensus.cc:2468] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 0.
I20260709 12:29:28.016366 16264 raft_consensus.cc:515] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
W20260709 12:29:28.036531 15921 leader_election.cc:343] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:28.037083 15921 leader_election.cc:304] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f742bdd975884e759c43e1311e22ac23; no voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7
I20260709 12:29:28.014402 16256 consensus_queue.cc:260] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.040105 16264 leader_election.cc:290] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
W20260709 12:29:28.010880 16070 leader_election.cc:343] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:28.041901 16070 leader_election.cc:343] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:28.043305 16070 leader_election.cc:304] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7; no voters: f742bdd975884e759c43e1311e22ac23
I20260709 12:29:28.049583 16283 raft_consensus.cc:2804] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:28.050169 16283 raft_consensus.cc:493] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:29:28.050503 15998 leader_election.cc:343] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:28.050100 16286 raft_consensus.cc:515] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.050907 16283 raft_consensus.cc:3060] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.044286 16284 raft_consensus.cc:2749] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:28.045948 16239 leader_election.cc:290] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:28.010115 16032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:28.056547 16032 raft_consensus.cc:2468] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 0.
I20260709 12:29:28.048092 16258 tablet_bootstrap.cc:654] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:28.048694 16280 raft_consensus.cc:493] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:28.055025 15998 leader_election.cc:304] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7; no voters: f742bdd975884e759c43e1311e22ac23
I20260709 12:29:28.060954 16283 raft_consensus.cc:515] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.062785 16264 raft_consensus.cc:2804] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:28.063398 16264 raft_consensus.cc:493] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:28.063836 16264 raft_consensus.cc:3060] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.066186 16286 leader_election.cc:290] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 election: Requested vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:28.066272 16283 leader_election.cc:290] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:28.066843 16152 raft_consensus.cc:515] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.074127 16280 raft_consensus.cc:515] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.076442 16152 leader_election.cc:290] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 election: Requested vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:28.077188 16152 raft_consensus.cc:493] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:28.083359 16152 raft_consensus.cc:515] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.087754 16152 leader_election.cc:290] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:28.090111 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d60dd6f617e4b6ca66f0573bb2aaa82" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:28.092031 16070 leader_election.cc:304] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7; no voters: f742bdd975884e759c43e1311e22ac23
I20260709 12:29:28.093060 16283 raft_consensus.cc:2804] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:28.076442 16280 leader_election.cc:290] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:28.093534 16283 raft_consensus.cc:493] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:28.093689 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4daed21d1e44c07a6b92afea49cbb6d" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:28.094758 16283 raft_consensus.cc:3060] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.100150 15956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abfa827185b748668746d469553068cb" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:28.102886 16032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ae049343d64dad9f168e68d13b4687" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:28.106156 15956 raft_consensus.cc:2468] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 0.
I20260709 12:29:28.107146 16264 raft_consensus.cc:515] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.108600 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ae049343d64dad9f168e68d13b4687" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:28.109277 15957 raft_consensus.cc:3060] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.109694 16264 leader_election.cc:290] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:28.109917 15958 raft_consensus.cc:3060] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:28.100356 15921 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.194:33263, user_credentials={real_user=slave}} blocked reactor thread for 61919.7us
W20260709 12:29:28.115638 15998 leader_election.cc:343] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:28.117539 16033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4daed21d1e44c07a6b92afea49cbb6d" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:28.118662 16033 raft_consensus.cc:3060] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.118860 16030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abfa827185b748668746d469553068cb" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:28.114171 16256 ts_tablet_manager.cc:1434] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.120s	user 0.012s	sys 0.014s
I20260709 12:29:28.120064 16030 raft_consensus.cc:2468] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 0.
I20260709 12:29:28.121623 16256 tablet_bootstrap.cc:492] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23: Bootstrap starting.
I20260709 12:29:28.106274 16032 raft_consensus.cc:3060] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.127518 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:28.131672 16258 tablet_bootstrap.cc:492] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7: No bootstrap required, opened a new log
I20260709 12:29:28.133009 16258 ts_tablet_manager.cc:1403] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7: Time spent bootstrapping tablet: real 0.167s	user 0.016s	sys 0.005s
I20260709 12:29:28.135371 16031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88e98f796f438f82b5615a926aa4c1" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:28.136060 16029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abfa827185b748668746d469553068cb" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:28.136746 15956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d60dd6f617e4b6ca66f0573bb2aaa82" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:28.136992 16029 raft_consensus.cc:2468] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 0.
I20260709 12:29:28.137261 16030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee293a1e4364c2e8c2d72fa2c0c6ab9" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:28.137460 15956 raft_consensus.cc:3060] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.137955 16030 raft_consensus.cc:3060] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.138450 15955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:28.136191 16031 raft_consensus.cc:3060] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.140735 16105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee293a1e4364c2e8c2d72fa2c0c6ab9" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:28.141372 16105 raft_consensus.cc:3060] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.138998 16258 raft_consensus.cc:359] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.142714 16258 raft_consensus.cc:385] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:28.141521 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88e98f796f438f82b5615a926aa4c1" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:28.143031 16258 raft_consensus.cc:740] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.145464 15921 leader_election.cc:304] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:28.145588 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abfa827185b748668746d469553068cb" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:28.146381 16107 raft_consensus.cc:2468] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 0.
W20260709 12:29:28.144297 15996 leader_election.cc:343] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:28.148142 16257 ts_tablet_manager.cc:1434] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.262s	user 0.101s	sys 0.110s
I20260709 12:29:28.148963 16288 raft_consensus.cc:2804] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:28.149587 16288 raft_consensus.cc:493] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:28.150720 16288 raft_consensus.cc:3060] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.152173 16173 raft_consensus.cc:493] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:28.152819 16173 raft_consensus.cc:515] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.155356 16173 leader_election.cc:290] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:28.157084 16258 consensus_queue.cc:260] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.145064 16070 leader_election.cc:304] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
W20260709 12:29:28.159019 15923 leader_election.cc:343] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:28.160897 15955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa801227038b4ca9a3bb2a634b0d39ef" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:28.165105 15955 raft_consensus.cc:2468] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 0.
I20260709 12:29:28.165474 16258 ts_tablet_manager.cc:1434] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 12:29:28.167542 16105 raft_consensus.cc:2468] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 1.
I20260709 12:29:28.170037 16257 tablet_bootstrap.cc:492] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:28.174122 15996 leader_election.cc:304] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
W20260709 12:29:28.174419 16070 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.4.193:45651, user_credentials={real_user=slave}} blocked reactor thread for 43664.1us
I20260709 12:29:28.175292 16264 raft_consensus.cc:2804] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:28.175712 16264 raft_consensus.cc:493] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:28.176087 16264 raft_consensus.cc:3060] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.177047 16280 raft_consensus.cc:2804] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:28.176512 15956 raft_consensus.cc:2468] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 1.
I20260709 12:29:28.177597 16280 raft_consensus.cc:493] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:28.164149 16031 raft_consensus.cc:2468] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 1.
I20260709 12:29:28.178069 16280 raft_consensus.cc:3060] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.180085 16107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa801227038b4ca9a3bb2a634b0d39ef" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:28.180943 16107 raft_consensus.cc:2468] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 0.
I20260709 12:29:28.184628 15923 leader_election.cc:304] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
I20260709 12:29:28.181821 16257 tablet_bootstrap.cc:654] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:28.190217 15921 leader_election.cc:304] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: cd106f11cb4c43f19e686df9f9a365f9
I20260709 12:29:28.202472 16284 raft_consensus.cc:2804] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:28.203022 16284 raft_consensus.cc:697] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Becoming Leader. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Running, Role: LEADER
I20260709 12:29:28.194996 16280 raft_consensus.cc:515] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.190271 16289 raft_consensus.cc:2804] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:28.204387 16033 raft_consensus.cc:2468] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 1.
I20260709 12:29:28.204748 16289 raft_consensus.cc:697] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Becoming Leader. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Running, Role: LEADER
I20260709 12:29:28.182895 15958 raft_consensus.cc:2468] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 1.
I20260709 12:29:28.191136 15996 leader_election.cc:304] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: cd106f11cb4c43f19e686df9f9a365f9
I20260709 12:29:28.206209 16280 leader_election.cc:290] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:28.200812 16032 raft_consensus.cc:2468] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 1.
I20260709 12:29:28.156307 16283 raft_consensus.cc:515] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.203943 16284 consensus_queue.cc:237] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.203878 16030 raft_consensus.cc:2468] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 1.
I20260709 12:29:28.208299 16106 raft_consensus.cc:2393] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f8085f7497804cc096027b14b64f95b7 in current term 1: Already voted for candidate cd106f11cb4c43f19e686df9f9a365f9 in this term.
I20260709 12:29:28.205783 16289 consensus_queue.cc:237] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.210217 16173 raft_consensus.cc:2804] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:28.186024 15957 raft_consensus.cc:2468] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 1.
I20260709 12:29:28.213912 16283 leader_election.cc:290] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:28.196022 16288 raft_consensus.cc:515] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
W20260709 12:29:28.219337 15904 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:28.196832 16264 raft_consensus.cc:515] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.217056 16070 leader_election.cc:304] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:28.220293 16030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abfa827185b748668746d469553068cb" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:28.221032 16030 raft_consensus.cc:3060] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.221343 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abfa827185b748668746d469553068cb" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:28.222007 16283 raft_consensus.cc:2804] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:28.201078 16256 tablet_bootstrap.cc:654] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:28.223018 16283 raft_consensus.cc:697] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 1 LEADER]: Becoming Leader. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Running, Role: LEADER
I20260709 12:29:28.224135 16283 consensus_queue.cc:237] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.222432 16106 raft_consensus.cc:2393] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f742bdd975884e759c43e1311e22ac23 in current term 1: Already voted for candidate cd106f11cb4c43f19e686df9f9a365f9 in this term.
I20260709 12:29:28.227246 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abfa827185b748668746d469553068cb" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:28.228159 15958 raft_consensus.cc:2393] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd106f11cb4c43f19e686df9f9a365f9 in current term 1: Already voted for candidate f742bdd975884e759c43e1311e22ac23 in this term.
I20260709 12:29:28.229373 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa801227038b4ca9a3bb2a634b0d39ef" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:28.230437 15957 raft_consensus.cc:3060] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.229465 16030 raft_consensus.cc:2468] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 1.
I20260709 12:29:28.232235 15921 leader_election.cc:304] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: cd106f11cb4c43f19e686df9f9a365f9
I20260709 12:29:28.232276 16264 leader_election.cc:290] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:28.233855 16287 raft_consensus.cc:2804] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:28.234409 16287 raft_consensus.cc:697] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Becoming Leader. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Running, Role: LEADER
I20260709 12:29:28.235168 15998 leader_election.cc:304] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8085f7497804cc096027b14b64f95b7; no voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23
I20260709 12:29:28.235196 16030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abfa827185b748668746d469553068cb" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:28.235327 16287 consensus_queue.cc:237] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.236696 16030 raft_consensus.cc:2393] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd106f11cb4c43f19e686df9f9a365f9 in current term 1: Already voted for candidate f742bdd975884e759c43e1311e22ac23 in this term.
I20260709 12:29:28.239131 16264 raft_consensus.cc:2749] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:28.243127 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:28.247828 16288 leader_election.cc:290] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 election: Requested vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:28.251169 16032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:28.252238 16032 raft_consensus.cc:2393] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd106f11cb4c43f19e686df9f9a365f9 in current term 1: Already voted for candidate f8085f7497804cc096027b14b64f95b7 in this term.
I20260709 12:29:28.265724 16256 tablet_bootstrap.cc:492] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23: No bootstrap required, opened a new log
I20260709 12:29:28.266666 16256 ts_tablet_manager.cc:1403] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23: Time spent bootstrapping tablet: real 0.145s	user 0.015s	sys 0.003s
I20260709 12:29:28.266676 16070 leader_election.cc:304] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
I20260709 12:29:28.259006 16173 raft_consensus.cc:697] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Becoming Leader. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:28.271152 15854 catalog_manager.cc:5697] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "cd106f11cb4c43f19e686df9f9a365f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:28.272295 16283 raft_consensus.cc:2804] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:28.273397 16256 raft_consensus.cc:359] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.274561 16256 raft_consensus.cc:385] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:28.274883 16256 raft_consensus.cc:740] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.274238 15852 catalog_manager.cc:5697] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 reported cstate change: term changed from 0 to 1, leader changed from <none> to f742bdd975884e759c43e1311e22ac23 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "f742bdd975884e759c43e1311e22ac23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:28.275669 16256 consensus_queue.cc:260] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.276764 16283 raft_consensus.cc:697] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 LEADER]: Becoming Leader. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Running, Role: LEADER
W20260709 12:29:28.277889 16070 leader_election.cc:343] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:28.277783 16283 consensus_queue.cc:237] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.279175 16256 ts_tablet_manager.cc:1434] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:29:28.279403 16257 tablet_bootstrap.cc:492] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:28.280272 16257 ts_tablet_manager.cc:1403] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.124s	user 0.016s	sys 0.034s
I20260709 12:29:28.283167 16070 leader_election.cc:304] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9; no voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7
I20260709 12:29:28.276463 15852 catalog_manager.cc:5697] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 reported cstate change: term changed from 0 to 1, leader changed from <none> to f742bdd975884e759c43e1311e22ac23 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "f742bdd975884e759c43e1311e22ac23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:28.285775 16070 leader_election.cc:304] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9; no voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7
I20260709 12:29:28.287030 16280 raft_consensus.cc:2749] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:28.287766 16280 raft_consensus.cc:2749] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:28.283237 16173 consensus_queue.cc:237] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.289191 16105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa801227038b4ca9a3bb2a634b0d39ef" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:28.289991 16105 raft_consensus.cc:3060] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.290453 16257 raft_consensus.cc:359] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.291356 16257 raft_consensus.cc:385] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:28.291669 16257 raft_consensus.cc:740] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.292398 16257 consensus_queue.cc:260] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.294799 16257 ts_tablet_manager.cc:1434] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:29:28.295683 16257 tablet_bootstrap.cc:492] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:28.288460 15957 raft_consensus.cc:2468] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 1.
I20260709 12:29:28.300108 15996 leader_election.cc:304] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:28.301573 16292 raft_consensus.cc:2804] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:28.302194 16292 raft_consensus.cc:697] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Becoming Leader. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:28.303303 16292 consensus_queue.cc:237] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.315347 15856 catalog_manager.cc:5697] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8085f7497804cc096027b14b64f95b7 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "f8085f7497804cc096027b14b64f95b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:28.322269 16105 raft_consensus.cc:2468] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 1.
I20260709 12:29:28.346809 16257 tablet_bootstrap.cc:654] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:28.356238 15852 catalog_manager.cc:5697] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195). New cstate: current_term: 1 leader_uuid: "cd106f11cb4c43f19e686df9f9a365f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:28.362879 15854 catalog_manager.cc:5697] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 reported cstate change: term changed from 0 to 1, leader changed from <none> to f742bdd975884e759c43e1311e22ac23 (127.2.4.193). New cstate: current_term: 1 leader_uuid: "f742bdd975884e759c43e1311e22ac23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:28.363929 16257 tablet_bootstrap.cc:492] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:28.364616 16257 ts_tablet_manager.cc:1403] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.069s	user 0.019s	sys 0.003s
I20260709 12:29:28.367784 16257 raft_consensus.cc:359] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.368489 16257 raft_consensus.cc:385] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:28.368762 16257 raft_consensus.cc:740] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.369531 16257 consensus_queue.cc:260] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.371835 16257 ts_tablet_manager.cc:1434] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:28.372849 16257 tablet_bootstrap.cc:492] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9: Bootstrap starting.
I20260709 12:29:28.373323 15856 catalog_manager.cc:5697] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8085f7497804cc096027b14b64f95b7 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "f8085f7497804cc096027b14b64f95b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:28.379231 16257 tablet_bootstrap.cc:654] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:28.385182 16257 tablet_bootstrap.cc:492] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9: No bootstrap required, opened a new log
I20260709 12:29:28.385757 16257 ts_tablet_manager.cc:1403] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.003s
I20260709 12:29:28.388860 16257 raft_consensus.cc:359] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.389618 16257 raft_consensus.cc:385] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:28.389914 16257 raft_consensus.cc:740] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Initialized, Role: FOLLOWER
I20260709 12:29:28.390574 16257 consensus_queue.cc:260] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.392643 16257 ts_tablet_manager.cc:1434] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:29:28.393525 16292 raft_consensus.cc:493] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:28.394251 16292 raft_consensus.cc:515] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.396111 16292 leader_election.cc:290] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:28.397200 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6798d36196f40d882bab16f264150ed" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:28.397466 16105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6798d36196f40d882bab16f264150ed" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:28.398103 15957 raft_consensus.cc:2468] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 0.
I20260709 12:29:28.398391 16105 raft_consensus.cc:2468] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 0.
I20260709 12:29:28.399554 15996 leader_election.cc:304] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:28.400959 16292 raft_consensus.cc:2804] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:28.401331 16292 raft_consensus.cc:493] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:28.401676 16292 raft_consensus.cc:3060] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.405805 16283 raft_consensus.cc:493] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:28.406488 16283 raft_consensus.cc:515] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.408905 16283 leader_election.cc:290] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:28.409866 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d60dd6f617e4b6ca66f0573bb2aaa82" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:28.409473 16292 raft_consensus.cc:515] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.410267 16030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d60dd6f617e4b6ca66f0573bb2aaa82" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:28.410665 15957 raft_consensus.cc:2393] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd106f11cb4c43f19e686df9f9a365f9 in current term 1: Already voted for candidate f8085f7497804cc096027b14b64f95b7 in this term.
I20260709 12:29:28.412457 16070 leader_election.cc:304] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9; no voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7
I20260709 12:29:28.413378 16283 raft_consensus.cc:3060] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.413194 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6798d36196f40d882bab16f264150ed" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:28.413117 16292 leader_election.cc:290] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:28.414353 15957 raft_consensus.cc:3060] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.414723 16105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6798d36196f40d882bab16f264150ed" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:28.415313 16105 raft_consensus.cc:3060] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.422896 16283 raft_consensus.cc:2749] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:28.422861 15957 raft_consensus.cc:2468] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 1.
I20260709 12:29:28.423727 16105 raft_consensus.cc:2468] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 1.
I20260709 12:29:28.424971 15996 leader_election.cc:304] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:28.425938 16292 raft_consensus.cc:2804] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:28.426487 16292 raft_consensus.cc:697] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Becoming Leader. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:28.427448 16292 consensus_queue.cc:237] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.435988 15854 catalog_manager.cc:5697] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8085f7497804cc096027b14b64f95b7 (127.2.4.194). New cstate: current_term: 1 leader_uuid: "f8085f7497804cc096027b14b64f95b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:28.483386 16283 raft_consensus.cc:493] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:28.484113 16283 raft_consensus.cc:515] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.486291 16283 leader_election.cc:290] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:28.487396 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88e98f796f438f82b5615a926aa4c1" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:28.487664 16030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88e98f796f438f82b5615a926aa4c1" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:28.488456 16030 raft_consensus.cc:2393] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd106f11cb4c43f19e686df9f9a365f9 in current term 1: Already voted for candidate f742bdd975884e759c43e1311e22ac23 in this term.
I20260709 12:29:28.490031 16070 leader_election.cc:304] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9; no voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7
I20260709 12:29:28.490904 16283 raft_consensus.cc:3060] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.497524 16283 raft_consensus.cc:2749] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:28.597457 16284 consensus_queue.cc:1048] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.610469 16284 consensus_queue.cc:1048] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:28.613258 16284 raft_consensus.cc:493] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:28.613844 16284 raft_consensus.cc:515] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:28.616611 16284 leader_election.cc:290] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:28.617300 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9" is_pre_election: true
I20260709 12:29:28.617969 16030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:28.618420 16106 raft_consensus.cc:2393] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f742bdd975884e759c43e1311e22ac23 in current term 1: Already voted for candidate cd106f11cb4c43f19e686df9f9a365f9 in this term.
I20260709 12:29:28.619326 16030 raft_consensus.cc:2393] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f742bdd975884e759c43e1311e22ac23 in current term 1: Already voted for candidate f8085f7497804cc096027b14b64f95b7 in this term.
I20260709 12:29:28.621703 15921 leader_election.cc:304] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f742bdd975884e759c43e1311e22ac23; no voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7
I20260709 12:29:28.622833 16284 raft_consensus.cc:3060] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:28.631863 16284 raft_consensus.cc:2749] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:28.657027 16284 consensus_queue.cc:1048] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.669978 16288 consensus_queue.cc:1048] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:28.700495 16284 consensus_queue.cc:1048] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:28.717715 16288 consensus_queue.cc:1048] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:28.749655 16292 consensus_queue.cc:1048] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.763805 16264 consensus_queue.cc:1048] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.783386 16283 consensus_queue.cc:1048] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:28.799336 16283 consensus_queue.cc:1048] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.807540 16283 consensus_queue.cc:1048] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.814069 16264 consensus_queue.cc:1048] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.829394 16264 consensus_queue.cc:1048] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:28.839200 16324 consensus_queue.cc:1048] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:28.847699 16292 consensus_queue.cc:1048] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:28.864146 16264 consensus_queue.cc:1048] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:29:29.226414 15904 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:29.801298 16320 raft_consensus.cc:493] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:29.802059 16320 raft_consensus.cc:515] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:29.804111 16320 leader_election.cc:290] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:29.805470 16032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7" is_pre_election: true
I20260709 12:29:29.805291 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23" is_pre_election: true
I20260709 12:29:29.806322 16032 raft_consensus.cc:2468] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 1.
I20260709 12:29:29.806495 15958 raft_consensus.cc:2468] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 1.
I20260709 12:29:29.807817 16070 leader_election.cc:304] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:29.808630 16320 raft_consensus.cc:2804] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:29.809868 16320 raft_consensus.cc:493] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:29.810302 16320 raft_consensus.cc:3060] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:29.819080 16320 raft_consensus.cc:515] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:29.820919 16320 leader_election.cc:290] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 2 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:29.821990 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:29.822228 16032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:29.822659 15958 raft_consensus.cc:3060] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:29.822932 16032 raft_consensus.cc:3060] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:29.830235 15958 raft_consensus.cc:2468] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 2.
I20260709 12:29:29.830432 16032 raft_consensus.cc:2468] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 2.
I20260709 12:29:29.831568 16070 leader_election.cc:304] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
I20260709 12:29:29.832412 16320 raft_consensus.cc:2804] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:29.832924 16320 raft_consensus.cc:697] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 2 LEADER]: Becoming Leader. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Running, Role: LEADER
I20260709 12:29:29.833870 16320 consensus_queue.cc:237] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:29.843521 15853 catalog_manager.cc:5697] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 reported cstate change: term changed from 0 to 2, leader changed from <none> to cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195). New cstate: current_term: 2 leader_uuid: "cd106f11cb4c43f19e686df9f9a365f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:30.148524 16106 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2ae049343d64dad9f168e68d13b4687"
dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
mode: GRACEFUL
new_leader_uuid: "f742bdd975884e759c43e1311e22ac23"
 from {username='slave'} at 127.0.0.1:48628
I20260709 12:29:30.149278 16106 raft_consensus.cc:606] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 LEADER]: Received request to transfer leadership to TS f742bdd975884e759c43e1311e22ac23
I20260709 12:29:30.161379 16106 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4daed21d1e44c07a6b92afea49cbb6d"
dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
mode: GRACEFUL
new_leader_uuid: "f742bdd975884e759c43e1311e22ac23"
 from {username='slave'} at 127.0.0.1:48628
I20260709 12:29:30.162056 16106 raft_consensus.cc:606] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 1 LEADER]: Received request to transfer leadership to TS f742bdd975884e759c43e1311e22ac23
I20260709 12:29:30.177583 15958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9c88e98f796f438f82b5615a926aa4c1"
dest_uuid: "f742bdd975884e759c43e1311e22ac23"
mode: GRACEFUL
new_leader_uuid: "f8085f7497804cc096027b14b64f95b7"
 from {username='slave'} at 127.0.0.1:33802
I20260709 12:29:30.178476 15958 raft_consensus.cc:606] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Received request to transfer leadership to TS f8085f7497804cc096027b14b64f95b7
I20260709 12:29:30.185024 16106 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a"
dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
mode: GRACEFUL
new_leader_uuid: "f8085f7497804cc096027b14b64f95b7"
 from {username='slave'} at 127.0.0.1:48628
I20260709 12:29:30.185935 16106 raft_consensus.cc:606] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 2 LEADER]: Received request to transfer leadership to TS f8085f7497804cc096027b14b64f95b7
I20260709 12:29:30.192546 16030 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6798d36196f40d882bab16f264150ed"
dest_uuid: "f8085f7497804cc096027b14b64f95b7"
mode: GRACEFUL
new_leader_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
 from {username='slave'} at 127.0.0.1:43196
I20260709 12:29:30.193356 16030 raft_consensus.cc:606] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Received request to transfer leadership to TS cd106f11cb4c43f19e686df9f9a365f9
I20260709 12:29:30.265733 15958 raft_consensus.cc:1275] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Refusing update from remote peer cd106f11cb4c43f19e686df9f9a365f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:29:30.267306 16320 consensus_queue.cc:1048] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:30.280462 16030 raft_consensus.cc:1275] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Refusing update from remote peer cd106f11cb4c43f19e686df9f9a365f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:29:30.283736 16320 consensus_queue.cc:1048] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:30.296738 16320 raft_consensus.cc:993] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9: : Instructing follower f8085f7497804cc096027b14b64f95b7 to start an election
I20260709 12:29:30.297320 16324 raft_consensus.cc:1081] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 2 LEADER]: Signalling peer f8085f7497804cc096027b14b64f95b7 to start an election
I20260709 12:29:30.299417 16030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a"
dest_uuid: "f8085f7497804cc096027b14b64f95b7"
 from {username='slave'} at 127.0.0.1:43192
I20260709 12:29:30.300132 16030 raft_consensus.cc:493] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:30.300551 16030 raft_consensus.cc:3060] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:30.308501 16030 raft_consensus.cc:515] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:30.310729 16030 leader_election.cc:290] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 3 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:30.311946 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:30.312474 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c13ce3b5a040a394b6cb64be5ec33a" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:30.312737 15958 raft_consensus.cc:3060] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:30.313198 16106 raft_consensus.cc:3055] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:30.313572 16106 raft_consensus.cc:740] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Running, Role: LEADER
I20260709 12:29:30.314495 16106 consensus_queue.cc:260] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:30.315997 16106 raft_consensus.cc:3060] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:30.320081 15958 raft_consensus.cc:2468] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 3.
I20260709 12:29:30.321280 15996 leader_election.cc:304] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:30.322183 16264 raft_consensus.cc:2804] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:30.322618 16264 raft_consensus.cc:697] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 3 LEADER]: Becoming Leader. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:30.323313 16264 consensus_queue.cc:237] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:30.324113 16106 raft_consensus.cc:2468] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 3.
I20260709 12:29:30.331853 15853 catalog_manager.cc:5697] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 reported cstate change: term changed from 2 to 3, leader changed from cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195) to f8085f7497804cc096027b14b64f95b7 (127.2.4.194). New cstate: current_term: 3 leader_uuid: "f8085f7497804cc096027b14b64f95b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:30.468581 16292 raft_consensus.cc:993] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7: : Instructing follower cd106f11cb4c43f19e686df9f9a365f9 to start an election
I20260709 12:29:30.469290 16292 raft_consensus.cc:1081] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Signalling peer cd106f11cb4c43f19e686df9f9a365f9 to start an election
I20260709 12:29:30.471799 16106 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6798d36196f40d882bab16f264150ed"
dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
 from {username='slave'} at 127.0.0.1:48616
I20260709 12:29:30.472703 16106 raft_consensus.cc:493] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:30.473134 16106 raft_consensus.cc:3060] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:30.482002 16106 raft_consensus.cc:515] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:30.483987 16106 leader_election.cc:290] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 2 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:30.485029 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6798d36196f40d882bab16f264150ed" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:30.485733 15958 raft_consensus.cc:3060] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:30.485491 16030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6798d36196f40d882bab16f264150ed" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:30.486475 16030 raft_consensus.cc:3055] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:30.486815 16030 raft_consensus.cc:740] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:30.487869 16030 consensus_queue.cc:260] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:30.489542 16030 raft_consensus.cc:3060] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:30.492923 16320 raft_consensus.cc:993] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9: : Instructing follower f742bdd975884e759c43e1311e22ac23 to start an election
I20260709 12:29:30.493782 16324 raft_consensus.cc:1081] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 LEADER]: Signalling peer f742bdd975884e759c43e1311e22ac23 to start an election
I20260709 12:29:30.494724 15958 raft_consensus.cc:2468] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 2.
I20260709 12:29:30.495786 15958 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f2ae049343d64dad9f168e68d13b4687"
dest_uuid: "f742bdd975884e759c43e1311e22ac23"
 from {username='slave'} at 127.0.0.1:33792
I20260709 12:29:30.496337 15958 raft_consensus.cc:493] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:30.496271 16070 leader_election.cc:304] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
I20260709 12:29:30.497080 15958 raft_consensus.cc:3060] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:30.497545 16324 raft_consensus.cc:2804] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:30.498207 16324 raft_consensus.cc:697] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [term 2 LEADER]: Becoming Leader. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Running, Role: LEADER
I20260709 12:29:30.498529 16030 raft_consensus.cc:2468] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 2.
I20260709 12:29:30.499095 16324 consensus_queue.cc:237] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:30.505308 16287 raft_consensus.cc:993] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23: : Instructing follower f8085f7497804cc096027b14b64f95b7 to start an election
I20260709 12:29:30.505940 16335 raft_consensus.cc:1081] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Signalling peer f8085f7497804cc096027b14b64f95b7 to start an election
I20260709 12:29:30.508843 16030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9c88e98f796f438f82b5615a926aa4c1"
dest_uuid: "f8085f7497804cc096027b14b64f95b7"
 from {username='slave'} at 127.0.0.1:43188
I20260709 12:29:30.509673 16030 raft_consensus.cc:493] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:30.509470 15958 raft_consensus.cc:515] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:30.510284 16030 raft_consensus.cc:3060] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:30.510262 15853 catalog_manager.cc:5697] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 reported cstate change: term changed from 1 to 2, leader changed from f8085f7497804cc096027b14b64f95b7 (127.2.4.194) to cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195). New cstate: current_term: 2 leader_uuid: "cd106f11cb4c43f19e686df9f9a365f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:30.513424 16106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ae049343d64dad9f168e68d13b4687" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:30.514328 16106 raft_consensus.cc:3055] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:30.514659 16106 raft_consensus.cc:740] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Running, Role: LEADER
I20260709 12:29:30.515802 16106 consensus_queue.cc:260] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:30.517138 16106 raft_consensus.cc:3060] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:30.517544 16032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ae049343d64dad9f168e68d13b4687" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:30.518797 16032 raft_consensus.cc:3060] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:30.521245 16030 raft_consensus.cc:515] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:30.523764 16030 leader_election.cc:290] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 2 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:30.524982 15957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88e98f796f438f82b5615a926aa4c1" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:30.525777 15958 leader_election.cc:290] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 2 election: Requested vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:30.525867 15957 raft_consensus.cc:3055] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:30.526340 15957 raft_consensus.cc:740] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Running, Role: LEADER
I20260709 12:29:30.525372 16106 raft_consensus.cc:2468] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 2.
I20260709 12:29:30.527253 15957 consensus_queue.cc:260] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:30.527691 15923 leader_election.cc:304] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
I20260709 12:29:30.528721 15957 raft_consensus.cc:3060] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:30.528877 16335 raft_consensus.cc:2804] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:30.529590 16335 raft_consensus.cc:697] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [term 2 LEADER]: Becoming Leader. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Running, Role: LEADER
I20260709 12:29:30.530483 16335 consensus_queue.cc:237] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:30.537943 15957 raft_consensus.cc:2468] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 2.
I20260709 12:29:30.538789 16105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88e98f796f438f82b5615a926aa4c1" candidate_uuid: "f8085f7497804cc096027b14b64f95b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:30.539469 16105 raft_consensus.cc:3060] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:30.539616 15996 leader_election.cc:304] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:30.540496 16324 raft_consensus.cc:993] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9: : Instructing follower f742bdd975884e759c43e1311e22ac23 to start an election
I20260709 12:29:30.541086 16324 raft_consensus.cc:1081] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 1 LEADER]: Signalling peer f742bdd975884e759c43e1311e22ac23 to start an election
I20260709 12:29:30.541764 16292 raft_consensus.cc:2804] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:30.542364 16292 raft_consensus.cc:697] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [term 2 LEADER]: Becoming Leader. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:30.543331 16292 consensus_queue.cc:237] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:30.539654 15856 catalog_manager.cc:5697] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 reported cstate change: term changed from 1 to 2, leader changed from cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195) to f742bdd975884e759c43e1311e22ac23 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "f742bdd975884e759c43e1311e22ac23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:30.547961 16032 raft_consensus.cc:2468] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 2.
I20260709 12:29:30.550678 16105 raft_consensus.cc:2468] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8085f7497804cc096027b14b64f95b7 in term 2.
I20260709 12:29:30.552974 15957 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4daed21d1e44c07a6b92afea49cbb6d"
dest_uuid: "f742bdd975884e759c43e1311e22ac23"
 from {username='slave'} at 127.0.0.1:33792
I20260709 12:29:30.563110 15957 raft_consensus.cc:493] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:30.563553 15957 raft_consensus.cc:3060] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:30.566263 15856 catalog_manager.cc:5697] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 reported cstate change: term changed from 1 to 2, leader changed from f742bdd975884e759c43e1311e22ac23 (127.2.4.193) to f8085f7497804cc096027b14b64f95b7 (127.2.4.194). New cstate: current_term: 2 leader_uuid: "f8085f7497804cc096027b14b64f95b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:30.571301 15957 raft_consensus.cc:515] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:30.573421 15957 leader_election.cc:290] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 2 election: Requested vote from peers f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263), cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947)
I20260709 12:29:30.574945 16032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4daed21d1e44c07a6b92afea49cbb6d" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:30.575688 16032 raft_consensus.cc:3060] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:30.575711 16105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4daed21d1e44c07a6b92afea49cbb6d" candidate_uuid: "f742bdd975884e759c43e1311e22ac23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
I20260709 12:29:30.576606 16105 raft_consensus.cc:3055] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:30.576954 16105 raft_consensus.cc:740] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Running, Role: LEADER
I20260709 12:29:30.578619 16105 consensus_queue.cc:260] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:30.579919 16105 raft_consensus.cc:3060] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:30.582926 16032 raft_consensus.cc:2468] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 2.
I20260709 12:29:30.584292 15921 leader_election.cc:304] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f742bdd975884e759c43e1311e22ac23, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:30.585242 16335 raft_consensus.cc:2804] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:30.585871 16335 raft_consensus.cc:697] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [term 2 LEADER]: Becoming Leader. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Running, Role: LEADER
I20260709 12:29:30.586819 16335 consensus_queue.cc:237] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:30.587523 16105 raft_consensus.cc:2468] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f742bdd975884e759c43e1311e22ac23 in term 2.
I20260709 12:29:30.597031 15856 catalog_manager.cc:5697] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 reported cstate change: term changed from 1 to 2, leader changed from cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195) to f742bdd975884e759c43e1311e22ac23 (127.2.4.193). New cstate: current_term: 2 leader_uuid: "f742bdd975884e759c43e1311e22ac23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:30.749230 16105 raft_consensus.cc:1275] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 3 FOLLOWER]: Refusing update from remote peer f8085f7497804cc096027b14b64f95b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:30.750712 16343 consensus_queue.cc:1048] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:30.760686 15957 raft_consensus.cc:1275] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 3 FOLLOWER]: Refusing update from remote peer f8085f7497804cc096027b14b64f95b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:30.762605 16264 consensus_queue.cc:1048] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:30.916419 16032 raft_consensus.cc:1275] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Refusing update from remote peer f742bdd975884e759c43e1311e22ac23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:30.918278 16287 consensus_queue.cc:1048] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:30.930294 16105 raft_consensus.cc:1275] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Refusing update from remote peer f742bdd975884e759c43e1311e22ac23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:30.931977 16287 consensus_queue.cc:1048] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:30.994336 16105 raft_consensus.cc:1275] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Refusing update from remote peer f8085f7497804cc096027b14b64f95b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:30.996271 16264 consensus_queue.cc:1048] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:31.008960 15957 raft_consensus.cc:1275] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Refusing update from remote peer f8085f7497804cc096027b14b64f95b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:31.010831 16343 consensus_queue.cc:1048] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:31.014907 16030 raft_consensus.cc:1275] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Refusing update from remote peer f742bdd975884e759c43e1311e22ac23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:31.017009 16335 consensus_queue.cc:1048] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:31.023602 15957 raft_consensus.cc:1275] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Refusing update from remote peer cd106f11cb4c43f19e686df9f9a365f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:31.025210 16320 consensus_queue.cc:1048] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:31.032425 16105 raft_consensus.cc:1275] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Refusing update from remote peer f742bdd975884e759c43e1311e22ac23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:31.035012 16335 consensus_queue.cc:1048] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:31.049219 16030 raft_consensus.cc:1275] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Refusing update from remote peer cd106f11cb4c43f19e686df9f9a365f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:31.051956 16356 consensus_queue.cc:1048] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:33.198474 15854 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36370:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:29:33.227770 15854 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36370:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:29:33.258955  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:29:33.463562 15923 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:33947, user_credentials={real_user=slave}} blocked reactor thread for 84216.9us
W20260709 12:29:33.531818 15998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.195:33947, user_credentials={real_user=slave}} blocked reactor thread for 46854.9us
I20260709 12:29:33.542309 16412 tablet.cc:1724] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:33.548904 16405 log.cc:927] Time spent T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:29:33.586799 15854 catalog_manager.cc:5789] TS cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195:33947) does not have the latest schema for tablet b6798d36196f40d882bab16f264150ed (table soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]). Expected version 1 got 0
I20260709 12:29:33.640458 16418 tablet.cc:1724] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.643807 16425 tablet.cc:1724] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.659584 16410 tablet.cc:1724] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:33.668906 15996 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.4.193:45651, user_credentials={real_user=slave}} blocked reactor thread for 191205us
I20260709 12:29:33.678012 16423 tablet.cc:1724] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.686446 16427 tablet.cc:1724] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.693809 16417 tablet.cc:1724] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.712833 16424 tablet.cc:1724] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.722406 16410 tablet.cc:1724] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.729218 15854 catalog_manager.cc:5789] TS f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651) does not have the latest schema for tablet 2ee293a1e4364c2e8c2d72fa2c0c6ab9 (table soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]). Expected version 1 got 0
I20260709 12:29:33.708316 16429 tablet.cc:1724] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.762094 16430 tablet.cc:1724] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.763187 16426 tablet.cc:1724] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.768594 15854 catalog_manager.cc:5789] TS f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263) does not have the latest schema for tablet 9c88e98f796f438f82b5615a926aa4c1 (table soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]). Expected version 1 got 0
I20260709 12:29:33.708314 16428 tablet.cc:1724] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.782845 15854 catalog_manager.cc:5789] TS f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263) does not have the latest schema for tablet 53c13ce3b5a040a394b6cb64be5ec33a (table soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9]). Expected version 1 got 0
I20260709 12:29:33.749076 16030 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72091658e1c047c08f409622ac3910b3"
dest_uuid: "f8085f7497804cc096027b14b64f95b7"
mode: GRACEFUL
new_leader_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
 from {username='slave'} at 127.0.0.1:39316
I20260709 12:29:33.783793 16030 raft_consensus.cc:606] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 2 LEADER]: Received request to transfer leadership to TS cd106f11cb4c43f19e686df9f9a365f9
I20260709 12:29:33.811251 16420 raft_consensus.cc:993] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7: : Instructing follower cd106f11cb4c43f19e686df9f9a365f9 to start an election
I20260709 12:29:33.811774 16420 raft_consensus.cc:1081] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 2 LEADER]: Signalling peer cd106f11cb4c43f19e686df9f9a365f9 to start an election
I20260709 12:29:33.815425 16107 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72091658e1c047c08f409622ac3910b3"
dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
 from {username='slave'} at 127.0.0.1:48616
I20260709 12:29:33.816051 16107 raft_consensus.cc:493] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:33.816470 16107 raft_consensus.cc:3060] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:33.820819 16412 tablet.cc:1724] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.822657 16425 tablet.cc:1724] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.838109 16423 tablet.cc:1724] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.823216 16427 tablet.cc:1724] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.842303 16107 raft_consensus.cc:515] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:33.823107 16417 tablet.cc:1724] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.845381 16107 leader_election.cc:290] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 3 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:33.847415 16030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72091658e1c047c08f409622ac3910b3" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:33.848325 16030 raft_consensus.cc:3055] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:33.848702 16030 raft_consensus.cc:740] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f8085f7497804cc096027b14b64f95b7, State: Running, Role: LEADER
I20260709 12:29:33.850178 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72091658e1c047c08f409622ac3910b3" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:33.850994 15958 raft_consensus.cc:3060] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:33.852864 16030 consensus_queue.cc:260] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:33.856451 16030 raft_consensus.cc:3060] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:33.858358 16426 tablet.cc:1724] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.871021 15958 raft_consensus.cc:2468] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 3.
I20260709 12:29:33.901904 16427 tablet.cc:1724] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.902395 16429 tablet.cc:1724] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.905102 16430 tablet.cc:1724] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.913869 16418 tablet.cc:1724] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.914939 16428 tablet.cc:1724] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.919540 16410 tablet.cc:1724] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.929513 16030 raft_consensus.cc:2468] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 3.
I20260709 12:29:33.932507 16430 tablet.cc:1724] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.941908 15958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d37749a2508249b2afaf4467cc56d190"
dest_uuid: "f742bdd975884e759c43e1311e22ac23"
mode: GRACEFUL
new_leader_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
 from {username='slave'} at 127.0.0.1:38694
I20260709 12:29:33.942770 16070 leader_election.cc:304] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f742bdd975884e759c43e1311e22ac23; no voters: 
I20260709 12:29:33.962855 15958 raft_consensus.cc:606] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 2 LEADER]: Received request to transfer leadership to TS cd106f11cb4c43f19e686df9f9a365f9
I20260709 12:29:33.942479 16429 tablet.cc:1724] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:33.964453 16400 raft_consensus.cc:2804] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:33.965087 16400 raft_consensus.cc:697] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 3 LEADER]: Becoming Leader. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Running, Role: LEADER
I20260709 12:29:33.966162 16400 consensus_queue.cc:237] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:33.969846  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:29:33.983608  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:34.012741 15856 catalog_manager.cc:5697] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 reported cstate change: term changed from 2 to 3, leader changed from f8085f7497804cc096027b14b64f95b7 (127.2.4.194) to cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "cd106f11cb4c43f19e686df9f9a365f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:34.024113 15853 catalog_manager.cc:6242] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350: soft_deleted_table [id=0a39c38576144c768f16e8a76a06b6a9] alter complete (version 1)
I20260709 12:29:34.250765 16376 raft_consensus.cc:993] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23: : Instructing follower cd106f11cb4c43f19e686df9f9a365f9 to start an election
I20260709 12:29:34.251421 16411 raft_consensus.cc:1081] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 2 LEADER]: Signalling peer cd106f11cb4c43f19e686df9f9a365f9 to start an election
I20260709 12:29:34.253283 16107 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d37749a2508249b2afaf4467cc56d190"
dest_uuid: "cd106f11cb4c43f19e686df9f9a365f9"
 from {username='slave'} at 127.0.0.1:48604
I20260709 12:29:34.253857 16107 raft_consensus.cc:493] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:34.254179 16107 raft_consensus.cc:3060] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:34.259779 16107 raft_consensus.cc:515] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:34.262089 16107 leader_election.cc:290] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 3 election: Requested vote from peers f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651), f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263)
I20260709 12:29:34.263067 15958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37749a2508249b2afaf4467cc56d190" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f742bdd975884e759c43e1311e22ac23"
I20260709 12:29:34.263522 16030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37749a2508249b2afaf4467cc56d190" candidate_uuid: "cd106f11cb4c43f19e686df9f9a365f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f8085f7497804cc096027b14b64f95b7"
I20260709 12:29:34.264060 15958 raft_consensus.cc:3055] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:34.264400 16030 raft_consensus.cc:3060] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:34.264534 15958 raft_consensus.cc:740] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f742bdd975884e759c43e1311e22ac23, State: Running, Role: LEADER
I20260709 12:29:34.265573 15958 consensus_queue.cc:260] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:34.266954 15958 raft_consensus.cc:3060] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:34.271734 16030 raft_consensus.cc:2468] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 3.
I20260709 12:29:34.272653 15958 raft_consensus.cc:2468] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd106f11cb4c43f19e686df9f9a365f9 in term 3.
I20260709 12:29:34.273305 16070 leader_election.cc:304] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd106f11cb4c43f19e686df9f9a365f9, f8085f7497804cc096027b14b64f95b7; no voters: 
I20260709 12:29:34.274473 16400 raft_consensus.cc:2804] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:34.275013 16400 raft_consensus.cc:697] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 3 LEADER]: Becoming Leader. State: Replica: cd106f11cb4c43f19e686df9f9a365f9, State: Running, Role: LEADER
I20260709 12:29:34.276094 16400 consensus_queue.cc:237] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } }
I20260709 12:29:34.285436 15856 catalog_manager.cc:5697] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 reported cstate change: term changed from 2 to 3, leader changed from f742bdd975884e759c43e1311e22ac23 (127.2.4.193) to cd106f11cb4c43f19e686df9f9a365f9 (127.2.4.195). New cstate: current_term: 3 leader_uuid: "cd106f11cb4c43f19e686df9f9a365f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd106f11cb4c43f19e686df9f9a365f9" member_type: VOTER last_known_addr { host: "127.2.4.195" port: 33947 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:34.409842 16030 raft_consensus.cc:1275] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 3 FOLLOWER]: Refusing update from remote peer cd106f11cb4c43f19e686df9f9a365f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:34.411664 16400 consensus_queue.cc:1048] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:34.428305 15958 raft_consensus.cc:1275] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23 [term 3 FOLLOWER]: Refusing update from remote peer cd106f11cb4c43f19e686df9f9a365f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:34.430160 16409 consensus_queue.cc:1048] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:34.684247 15958 raft_consensus.cc:1275] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 3 FOLLOWER]: Refusing update from remote peer cd106f11cb4c43f19e686df9f9a365f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:34.685581 16409 consensus_queue.cc:1048] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f742bdd975884e759c43e1311e22ac23" member_type: VOTER last_known_addr { host: "127.2.4.193" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:34.696635 16030 raft_consensus.cc:1275] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 3 FOLLOWER]: Refusing update from remote peer cd106f11cb4c43f19e686df9f9a365f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:34.698405 16409 consensus_queue.cc:1048] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8085f7497804cc096027b14b64f95b7" member_type: VOTER last_known_addr { host: "127.2.4.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:34.985013  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:34.992419  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:34.992887  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:35.994143  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:35.999696  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:36.000207  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:37.001333  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:37.007099  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:37.007575  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:38.008755  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:38.015980  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:38.016564  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:39.017966  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:39.023175  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:39.023651  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:40.024794  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:40.033762  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:40.034314  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:41.035562  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:41.041019  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:41.041594  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:42.043277  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:42.052891  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:42.053380  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:43.054855  2067 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:43.060269  2067 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:43.060890  2067 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:44.137429  2067 tablet_server.cc:179] TabletServer@127.2.4.193:0 shutting down...
W20260709 12:29:44.166383 15996 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 15 similar messages]
W20260709 12:29:44.173532 15996 consensus_peers.cc:597] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 -> Peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Couldn't send request to peer f742bdd975884e759c43e1311e22ac23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.204025  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:44.205169  2067 tablet_replica.cc:333] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.206445  2067 raft_consensus.cc:2243] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.207429  2067 raft_consensus.cc:2272] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.211526  2067 tablet_replica.cc:333] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.212581  2067 raft_consensus.cc:2243] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.214042  2067 raft_consensus.cc:2272] T f2ae049343d64dad9f168e68d13b4687 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.217595  2067 tablet_replica.cc:333] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.218454  2067 raft_consensus.cc:2243] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.219698  2067 raft_consensus.cc:2272] T abfa827185b748668746d469553068cb P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.223148  2067 tablet_replica.cc:333] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.224004  2067 raft_consensus.cc:2243] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.225279  2067 raft_consensus.cc:2272] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.228725  2067 tablet_replica.cc:333] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.229529  2067 raft_consensus.cc:2243] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.230273  2067 raft_consensus.cc:2272] T fa801227038b4ca9a3bb2a634b0d39ef P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.234285  2067 tablet_replica.cc:333] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.235213  2067 raft_consensus.cc:2243] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.236114  2067 raft_consensus.cc:2272] T d37749a2508249b2afaf4467cc56d190 P f742bdd975884e759c43e1311e22ac23 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.239425  2067 tablet_replica.cc:333] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.240123  2067 raft_consensus.cc:2243] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.240728  2067 raft_consensus.cc:2272] T 3630e85b83b04193a66ec5acd5611eeb P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.243971  2067 tablet_replica.cc:333] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.244714  2067 raft_consensus.cc:2243] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.245658  2067 raft_consensus.cc:2272] T 9c88e98f796f438f82b5615a926aa4c1 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.250488  2067 tablet_replica.cc:333] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.252094  2067 raft_consensus.cc:2243] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.253489  2067 raft_consensus.cc:2272] T 53c13ce3b5a040a394b6cb64be5ec33a P f742bdd975884e759c43e1311e22ac23 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:44.254746 16070 consensus_peers.cc:597] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 -> Peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Couldn't send request to peer f742bdd975884e759c43e1311e22ac23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:44.257555 16070 consensus_peers.cc:597] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 -> Peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Couldn't send request to peer f742bdd975884e759c43e1311e22ac23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.258201  2067 tablet_replica.cc:333] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.259516  2067 raft_consensus.cc:2243] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.260434  2067 raft_consensus.cc:2272] T 87b688925a5c41e1b41d0a4d8652e834 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.264385  2067 tablet_replica.cc:333] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.265146  2067 raft_consensus.cc:2243] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.265970  2067 raft_consensus.cc:2272] T b6798d36196f40d882bab16f264150ed P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.269239  2067 tablet_replica.cc:333] T 84514ec125fd474ba0c033b11a66d145 P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.270159  2067 raft_consensus.cc:2243] T 84514ec125fd474ba0c033b11a66d145 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.270846  2067 raft_consensus.cc:2272] T 84514ec125fd474ba0c033b11a66d145 P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.274282  2067 tablet_replica.cc:333] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.275490  2067 raft_consensus.cc:2243] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.276898  2067 raft_consensus.cc:2272] T 11a00e1f25ac46bf8df46d1a30c6f75b P f742bdd975884e759c43e1311e22ac23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.280923  2067 tablet_replica.cc:333] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.281950  2067 raft_consensus.cc:2243] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.283679  2067 raft_consensus.cc:2272] T dfdf24842682422dab37b9ccc8ab0c24 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:44.285874 15996 consensus_peers.cc:597] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 -> Peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Couldn't send request to peer f742bdd975884e759c43e1311e22ac23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.288043  2067 tablet_replica.cc:333] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.289050  2067 raft_consensus.cc:2243] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.289848  2067 raft_consensus.cc:2272] T 72091658e1c047c08f409622ac3910b3 P f742bdd975884e759c43e1311e22ac23 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.293262  2067 tablet_replica.cc:333] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.294045  2067 raft_consensus.cc:2243] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.295243  2067 raft_consensus.cc:2272] T 85fc54f820da47d0951fabd0d60156d0 P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.299105  2067 tablet_replica.cc:333] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.299955  2067 raft_consensus.cc:2243] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.301427  2067 raft_consensus.cc:2272] T e4daed21d1e44c07a6b92afea49cbb6d P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.304754  2067 tablet_replica.cc:333] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23: stopping tablet replica
I20260709 12:29:44.305439  2067 raft_consensus.cc:2243] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.306296  2067 raft_consensus.cc:2272] T 2a65d88797ab489db65de91babc2828f P f742bdd975884e759c43e1311e22ac23 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:44.335227 15996 consensus_peers.cc:597] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 -> Peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Couldn't send request to peer f742bdd975884e759c43e1311e22ac23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.374614  2067 tablet_server.cc:196] TabletServer@127.2.4.193:0 shutdown complete.
W20260709 12:29:44.376863 15996 consensus_peers.cc:597] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 -> Peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Couldn't send request to peer f742bdd975884e759c43e1311e22ac23. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:45651: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:44.419132 15996 consensus_peers.cc:597] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 -> Peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Couldn't send request to peer f742bdd975884e759c43e1311e22ac23. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:45651: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.433178  2067 tablet_server.cc:179] TabletServer@127.2.4.194:0 shutting down...
I20260709 12:29:44.487147  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:44.488090  2067 tablet_replica.cc:333] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.489269  2067 raft_consensus.cc:2243] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.490566  2067 raft_consensus.cc:2272] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.493779  2067 tablet_replica.cc:333] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.494462  2067 raft_consensus.cc:2243] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.495497  2067 raft_consensus.cc:2272] T 9c88e98f796f438f82b5615a926aa4c1 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.498327  2067 tablet_replica.cc:333] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.499080  2067 raft_consensus.cc:2243] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.499682  2067 raft_consensus.cc:2272] T f2ae049343d64dad9f168e68d13b4687 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.502511  2067 tablet_replica.cc:333] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.503206  2067 raft_consensus.cc:2243] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.504036  2067 raft_consensus.cc:2272] T e4daed21d1e44c07a6b92afea49cbb6d P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.506731  2067 tablet_replica.cc:333] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.507449  2067 raft_consensus.cc:2243] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.508431  2067 raft_consensus.cc:2272] T 2a65d88797ab489db65de91babc2828f P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.511119  2067 tablet_replica.cc:333] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.511932  2067 raft_consensus.cc:2243] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.512916  2067 raft_consensus.cc:2272] T 53c13ce3b5a040a394b6cb64be5ec33a P f8085f7497804cc096027b14b64f95b7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.515658  2067 tablet_replica.cc:333] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.516331  2067 raft_consensus.cc:2243] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.517376  2067 raft_consensus.cc:2272] T 87b688925a5c41e1b41d0a4d8652e834 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.519979  2067 tablet_replica.cc:333] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.520632  2067 raft_consensus.cc:2243] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.521186  2067 raft_consensus.cc:2272] T 3630e85b83b04193a66ec5acd5611eeb P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.523723  2067 tablet_replica.cc:333] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.524374  2067 raft_consensus.cc:2243] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.525393  2067 raft_consensus.cc:2272] T fa801227038b4ca9a3bb2a634b0d39ef P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.527798  2067 tablet_replica.cc:333] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.528412  2067 raft_consensus.cc:2243] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.529119  2067 raft_consensus.cc:2272] T d37749a2508249b2afaf4467cc56d190 P f8085f7497804cc096027b14b64f95b7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.532024  2067 tablet_replica.cc:333] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.532738  2067 raft_consensus.cc:2243] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.533367  2067 raft_consensus.cc:2272] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.536022  2067 tablet_replica.cc:333] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.536644  2067 raft_consensus.cc:2243] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.537272  2067 raft_consensus.cc:2272] T abfa827185b748668746d469553068cb P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.540093  2067 tablet_replica.cc:333] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.540707  2067 raft_consensus.cc:2243] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.541318  2067 raft_consensus.cc:2272] T b6798d36196f40d882bab16f264150ed P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.544330  2067 tablet_replica.cc:333] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.545018  2067 raft_consensus.cc:2243] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.545583  2067 raft_consensus.cc:2272] T 11a00e1f25ac46bf8df46d1a30c6f75b P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.548203  2067 tablet_replica.cc:333] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.548882  2067 raft_consensus.cc:2243] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.550043  2067 raft_consensus.cc:2272] T 84514ec125fd474ba0c033b11a66d145 P f8085f7497804cc096027b14b64f95b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.552747  2067 tablet_replica.cc:333] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.553390  2067 raft_consensus.cc:2243] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.554046  2067 raft_consensus.cc:2272] T 85fc54f820da47d0951fabd0d60156d0 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.556720  2067 tablet_replica.cc:333] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.557366  2067 raft_consensus.cc:2243] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.558264  2067 raft_consensus.cc:2272] T 72091658e1c047c08f409622ac3910b3 P f8085f7497804cc096027b14b64f95b7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.560982  2067 tablet_replica.cc:333] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7: stopping tablet replica
I20260709 12:29:44.561583  2067 raft_consensus.cc:2243] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.562487  2067 raft_consensus.cc:2272] T dfdf24842682422dab37b9ccc8ab0c24 P f8085f7497804cc096027b14b64f95b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.616611  2067 tablet_server.cc:196] TabletServer@127.2.4.194:0 shutdown complete.
I20260709 12:29:44.682801  2067 tablet_server.cc:179] TabletServer@127.2.4.195:0 shutting down...
W20260709 12:29:44.701810 16070 consensus_peers.cc:597] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 -> Peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Couldn't send request to peer f742bdd975884e759c43e1311e22ac23. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:45651: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:44.718868 16070 consensus_peers.cc:597] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 -> Peer f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263): Couldn't send request to peer f8085f7497804cc096027b14b64f95b7. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33263: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.734423  2067 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:44.735481  2067 tablet_replica.cc:333] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.736693  2067 raft_consensus.cc:2243] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.737504  2067 raft_consensus.cc:2272] T 9d60dd6f617e4b6ca66f0573bb2aaa82 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.741410  2067 tablet_replica.cc:333] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.742215  2067 raft_consensus.cc:2243] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:44.742348 16070 consensus_peers.cc:597] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 -> Peer f742bdd975884e759c43e1311e22ac23 (127.2.4.193:45651): Couldn't send request to peer f742bdd975884e759c43e1311e22ac23. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.193:45651: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.742980  2067 raft_consensus.cc:2272] T 9c88e98f796f438f82b5615a926aa4c1 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.745580  2067 tablet_replica.cc:333] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.746269  2067 raft_consensus.cc:2243] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.746783  2067 raft_consensus.cc:2272] T f2ae049343d64dad9f168e68d13b4687 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.749073  2067 tablet_replica.cc:333] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.749789  2067 raft_consensus.cc:2243] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.750279  2067 raft_consensus.cc:2272] T dfdf24842682422dab37b9ccc8ab0c24 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.752722  2067 tablet_replica.cc:333] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.753393  2067 raft_consensus.cc:2243] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.753959  2067 raft_consensus.cc:2272] T 2ee293a1e4364c2e8c2d72fa2c0c6ab9 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.756390  2067 tablet_replica.cc:333] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.756996  2067 raft_consensus.cc:2243] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.757510  2067 raft_consensus.cc:2272] T abfa827185b748668746d469553068cb P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.760027  2067 tablet_replica.cc:333] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.760618  2067 raft_consensus.cc:2243] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.761740  2067 raft_consensus.cc:2272] T b6798d36196f40d882bab16f264150ed P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.764086  2067 tablet_replica.cc:333] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.764673  2067 raft_consensus.cc:2243] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.765125  2067 raft_consensus.cc:2272] T 11a00e1f25ac46bf8df46d1a30c6f75b P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.767443  2067 tablet_replica.cc:333] T 84514ec125fd474ba0c033b11a66d145 P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.768014  2067 raft_consensus.cc:2243] T 84514ec125fd474ba0c033b11a66d145 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.768471  2067 raft_consensus.cc:2272] T 84514ec125fd474ba0c033b11a66d145 P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.770982  2067 tablet_replica.cc:333] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.771891  2067 raft_consensus.cc:2243] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.772655  2067 raft_consensus.cc:2272] T e4daed21d1e44c07a6b92afea49cbb6d P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.775852  2067 tablet_replica.cc:333] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.776571  2067 raft_consensus.cc:2243] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.777216  2067 raft_consensus.cc:2272] T 2a65d88797ab489db65de91babc2828f P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:44.777603 16070 consensus_peers.cc:597] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 -> Peer f8085f7497804cc096027b14b64f95b7 (127.2.4.194:33263): Couldn't send request to peer f8085f7497804cc096027b14b64f95b7. Status: Network error: Client connection negotiation failed: client connection to 127.2.4.194:33263: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.779835  2067 tablet_replica.cc:333] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.780584  2067 raft_consensus.cc:2243] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.781071  2067 raft_consensus.cc:2272] T 87b688925a5c41e1b41d0a4d8652e834 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.783807  2067 tablet_replica.cc:333] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.784577  2067 raft_consensus.cc:2243] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.785391  2067 raft_consensus.cc:2272] T 53c13ce3b5a040a394b6cb64be5ec33a P cd106f11cb4c43f19e686df9f9a365f9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.788306  2067 tablet_replica.cc:333] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.788954  2067 raft_consensus.cc:2243] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.789973  2067 raft_consensus.cc:2272] T 72091658e1c047c08f409622ac3910b3 P cd106f11cb4c43f19e686df9f9a365f9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.792254  2067 tablet_replica.cc:333] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.792876  2067 raft_consensus.cc:2243] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.793370  2067 raft_consensus.cc:2272] T 85fc54f820da47d0951fabd0d60156d0 P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.795703  2067 tablet_replica.cc:333] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.796252  2067 raft_consensus.cc:2243] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.796769  2067 raft_consensus.cc:2272] T fa801227038b4ca9a3bb2a634b0d39ef P cd106f11cb4c43f19e686df9f9a365f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.799034  2067 tablet_replica.cc:333] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.799573  2067 raft_consensus.cc:2243] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.800479  2067 raft_consensus.cc:2272] T d37749a2508249b2afaf4467cc56d190 P cd106f11cb4c43f19e686df9f9a365f9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.802785  2067 tablet_replica.cc:333] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9: stopping tablet replica
I20260709 12:29:44.803510  2067 raft_consensus.cc:2243] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.804419  2067 raft_consensus.cc:2272] T 3630e85b83b04193a66ec5acd5611eeb P cd106f11cb4c43f19e686df9f9a365f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.854998  2067 tablet_server.cc:196] TabletServer@127.2.4.195:0 shutdown complete.
I20260709 12:29:44.903746  2067 master.cc:562] Master@127.2.4.254:37151 shutting down...
I20260709 12:29:44.943691  2067 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.944577  2067 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.944918  2067 tablet_replica.cc:333] T 00000000000000000000000000000000 P ef17addbd962435aa4431544d3522350: stopping tablet replica
I20260709 12:29:44.971688  2067 master.cc:584] Master@127.2.4.254:37151 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (23066 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (44883 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (457744 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.RestartTserver

 1 FAILED TEST
I20260709 12:29:45.047912  2067 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2263 messages since previous log ~34 seconds ago
I20260709 12:29:45.048182  2067 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2263 messages since previous log ~34 seconds ago
I20260709 12:29:45.048362  2067 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2259 messages since previous log ~34 seconds ago
I20260709 12:29:45.048558  2067 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 7 messages since previous log ~105 seconds ago
I20260709 12:29:45.048725  2067 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 7 messages since previous log ~105 seconds ago
I20260709 12:29:45.048906  2067 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 7 messages since previous log ~129 seconds ago
I20260709 12:29:45.049060  2067 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~53 seconds ago
I20260709 12:29:45.049252  2067 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~53 seconds ago
I20260709 12:29:45.049423  2067 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/master/catalog_manager.cc:4755: suppressed but not reported on 1 messages since previous log ~313 seconds ago
I20260709 12:29:45.049736  2067 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 12 messages since previous log ~0 seconds ago
I20260709 12:29:45.050079  2067 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~21 seconds ago
